Compare commits

...
2177 Commits
Author SHA1 Message Date
Niko1221 dea6920b81 hip-rdna3 on Windows: Strix Halo (gfx1151) keeps the expert arena unregistered by default; strata.exe finds rocm\bin for --version (delay-loaded ROCm math DLLs, bundled by package_windows.py) 2026-10-20 17:09:46 +02:00
Niko1221 9ae93f7b1f Control 0.1.46.1: version bump for the Windows long-path hotfix 2026-10-20 11:18:52 +02:00
Niko1221 9f13d1e78f Merge main (Docker commit fix) into the long-path hotfix 2026-10-20 11:15:54 +02:00
Niko1221 a0abdf7588 Engine install: survive Windows MAX_PATH (short staging folder, extended-length paths, plain file errors, deep-folder warning)
On a fresh Windows PC (long paths off) the AMD engine packages could not be unpacked: staging under .updates/staging/<12 hex> pushed the
longest hipBLASLt file past 259 characters, and the failure surfaced as 'local file operation failed (FileNotFoundError)'.

- staging is now <components>/.s/<4 hex>/engine/<folder>; Control stages the same way
- unpack, delete and verify go through extended-length paths (strata_control/longpath.py)
- OSError messages name the file and say so when the path is too long
- a plain warning (shown as a warn line by launch) when the install folder is too deep for the engine's own file loading
- tests: real 0.2.1 package path lengths for a 20 character user name
2026-10-20 11:15:20 +02:00
Niko1221 c093ac263f docker/build.sh: STRATA_BUILD_COMMIT overrides the commit it passes 2026-10-20 10:39:48 +02:00
Niko1221 436fbd0a6e Docker: pass the source commit into the terminal stage (no git in the image)
package_terminal.py reads STRATA_BUILD_COMMIT first and only then asks git, and no longer crashes when git is missing. build.sh passes the commit as a build arg, the Dockerfile turns it into that env.
2026-10-20 10:39:15 +02:00
Niko1221 5bda973a43 Docker: make docker/publish.sh executable (the image workflow failed with Permission denied) 2026-10-20 10:29:54 +02:00
Niko1221 90767252f4 Release notes 0.2.1: final two-card batch A/B result, drop TODOs 2026-10-20 09:40:48 +02:00
Niko1221 a7bdbfdabc Release notes 0.2.1: the #3237 drive probe is on by default (owner OK) 2026-10-20 06:27:42 +02:00
Niko1221 9944d2f7f1 Merge branch 'f-021-int' into f-021-rel 2026-10-20 05:56:29 +02:00
Niko1221 b42ec662c7 Release notes 0.2.1: the R9700 queue fix with the two-queue numbers 2026-10-20 05:56:26 +02:00
Niko1221 b35a9816c7 amd: open two GPU queues instead of five at start (keeps the fixed prompt speed, no decode loss on R9700) 2026-10-20 04:53:45 +02:00
Niko1221 d33bb832ca Release notes 0.2.1: engine 0.1.46 and feed sequence 102 (proposed), file-name and Docker TODOs resolved, tuning tables per package 2026-10-20 02:37:08 +02:00
Niko1221 3e7c5d8e1c build_hip_linux.sh: pick up lib/rocm_sysdeps when the ROCm tree has it (the gfx1151 halo package needed a patched copy in 0.2.0-beta) 2026-10-20 02:31:50 +02:00
Niko1221 894d1be1f4 Version bump for 0.2.1 (proposed): app 0.2.1, engine and Control 0.1.46
Cargo.toml and Cargo.lock, desktop and host package.json, PRODUCT_VERSION, CMake project and Metal STRATA_VERSION, build_hip_linux PKGVER default, About comment, tests that pin the number. tauri.conf.json has no version key (Tauri takes it from Cargo.toml). Download names in the READMEs, AMD_HIP.md and UPDATES.md follow.
2026-10-20 02:17:37 +02:00
Niko1221 93aa62ed64 Release notes 0.2.1: fill the iD, iE, zc and bimodal parts (measured defaults, fixes, known limits) 2026-10-20 02:14:44 +02:00
Niko1221 83027df587 Merge branch 'f-021-iD' into f-021-int 2026-10-20 02:13:21 +02:00
Niko1221 484e06b961 Linux file tier: a start-up drive probe overrides the room rule's file cache on a fast drive (#3237)
When the room rule picks the file cache but the cache holds under half of the read set, the engine now
times O_DIRECT preads (8 threads) against first touches of a mapping (3 threads) on random 1 MiB
pieces of the model file, once, and reads unbuffered when the direct path runs at 4 GB/s or more and
is 1.3x (under 30% held) / 1.5x (under 50%) faster than the cached path. The reporter's PCIe 5 NVMe
(8.18 vs 4.93 GB/s, 13% held) lands on unbuffered (2x prompts, 3-4x decode in their A/B). A 1.7-2.5
GB/s NVMe (P1004) stays on the cache, where it decodes 1.1-1.6x faster; a ratio alone could not
separate the two, hence the absolute floor. STRATA_FILE_TIER_PROBE=0 keeps the room rule alone.
Default change: proposed, not validated on a fast drive here. Decision in platform::file_tier_prefers_direct
(platform_memory_test).
2026-10-20 01:32:52 +02:00
Niko1221 ed4595ad69 Merge branch 'f-021-bimodal' into f-021-int 2026-10-20 01:20:25 +02:00
Niko1221 06d7604af7 Merge branch 'f-021-zc' into f-021-int 2026-10-20 01:07:32 +02:00
Niko1221 ecf2bdcb7f Linux: io prefetch + decode gate by default for experts.bin packs when the page cache has room for 55% of the experts
Zero-copy does not apply to experts.bin (one contiguous blob the kernels already read from the mapping), but the io_fill_ prefetch path does. RTX 3060, IQ3_XXS 43 GB pack, 32 GB cgroup: decode 1.38/1.21/1.33x (5/5 pairs), prompts 1.04/1.06x. At 16 GB the cold 4K prompt was 4-6% slower, so it stays off below the room test and the log says how to force it. Identity gates 10/10, exact_pp 4K/20K identical.
2026-10-20 01:07:19 +02:00
Niko1221 3f1fa67f14 amd: open the GPU queues in a fixed order at start (R9700 fast/slow prompt starts)
On gfx1200/gfx1201 under Linux the compute stream and the expert staging stream
landed on the same compute pipe in about half of the starts, which made 4K prompts
13% slower (p620, 1,980 vs 2,290 tok/s). Five queues (four normal, one highest) are
now opened one after the other on the main thread before any other thread starts.
STRATA_HIP_QUEUE_PRIME=0 turns it off. Answers are unchanged.
2026-10-20 01:04:48 +02:00
Niko1221 881a42adab Merge branch 'f-021-notes' into f-021-int 2026-10-20 00:48:22 +02:00
Niko1221 3616148bbf Merge branch 'f-021-iE' into f-021-int
# Conflicts:
#	apps/host/src/locales/de.json
#	apps/host/src/locales/en.json
#	apps/host/src/locales/es.json
#	apps/host/src/locales/ja.json
#	apps/host/src/locales/ko.json
#	apps/host/src/locales/pt.json
#	apps/host/src/locales/ru.json
#	apps/host/src/locales/zh.json
2026-10-20 00:48:09 +02:00
Niko1221 24e56539c5 Release notes 0.2.1 (draft): downloads table, what is new, measured defaults, fixes, privacy, known limits, TODO markers for the running streams 2026-10-20 00:47:45 +02:00
Niko1221 8e957cd53b SYCL: build.sh stops an AOT Battlemage build before the compile when ocloc is the one that crashes on the int8 kernels (Compute Runtime 26.22.38646.x, IGC 2.36.3); I8_ESIMD=OFF / STRATA_SYCL_I8_ESIMD builds without them; docs say docker build --pull (#3219) 2026-10-20 00:47:08 +02:00
Niko1221 4f6ec303b5 Merge branch 'f-021-tfix' into f-021-int 2026-10-20 00:44:55 +02:00
Niko1221 1f52a00373 apps/host staging-scripts-syntax test: use Git bash with forward-slash paths on Windows instead of the WSL launcher 2026-10-20 00:44:41 +02:00
Niko1221 e68a032a58 math test: the KaTeX render check skips when katex is not installed in a shared node_modules (#3268) 2026-10-20 00:43:04 +02:00
Niko1221 10715d856d gfx906 (HIP wave64) builds again: no Qwen3.6 engine (cuBLAS calls the compat layer does not map), GGML_USE_HIP on the W4A4 target, the trellis overlay sources and the moe_f16_kq stub in the HIP source list; all targets compile for gfx906 (#3172 #3271) 2026-10-20 00:41:38 +02:00
Niko1221 e45a9bfd3a Merge branch 'f-021-split' into f-021-int 2026-10-20 00:19:12 +02:00
Niko1221 34e7c5cb1a Merge branch 'f-021-iA' into f-021-int
# Conflicts:
#	apps/host/server/native-control.mjs
2026-10-20 00:12:03 +02:00
Niko1221 8aaf188aa9 Your own engine settings are kept: user_args, a reserve of 0 on a compute-only card, control vectors survive a load (#3215 #3270 #3216)
The app and setup rewrote the run config's "args" on every Load: the resource preference forced
--vram-reserve-mib to 1536 (2048 with pictures, a fifth of the card on "work") whatever the file said, and
a setup run (started by the Load when the context or pictures differ) dropped every option it did not
choose itself, the control vector options included.

- run config: "user_args" holds the options set by hand; "args_written" is the args as setup/the app last
  wrote them, so a hand edit of "args" is noticed and moved into user_args (strata_control/user_args.py,
  apps/host/server/native-user-args.mjs: the same rule)
- every start (serve/server.py engine_args), every setup run (setup.carry_user_args), every host write and load
  and a saved tuning put the person's options over the generated ones; the same option: theirs wins
- old files without args_written: setup keeps the options the old args had, the new ones lack and setup.py never names
  (plus the control vector group); a reserve that is not setup's own 700
- --vram-reserve-mib: any number the person set (Expert setting, user_args, the key, a value in args) is used as it
  is, 0 included; the safety default (1536/1024, 2048 with pictures, 20% on "work") stays for a card that drives a
  display or is not known to be headless; a card the probe says drives no display keeps 512 (700 with pictures);
  a 0.2.0-beta file where the app wrote the number itself is told from a changed one
- hardware: drives_display per card (nvidia-smi display_active; the Windows adapter with a desktop resolution for AMD)
- setup --vram-reserve-mib, the web settings page and `start --vram-reserve-mib` record the value as the person's
- docs/TROUBLESHOOTING.md: what is kept and what the app still chooses
2026-10-20 00:11:11 +02:00
Niko1221 22703894ec Merge branch 'f-021-iB' into f-021-int
# Conflicts:
#	apps/host/server/native-control.mjs
2026-10-20 00:06:49 +02:00
Niko1221 b1713b3ff2 docs: dual-socket CPU pool measurements (workers across sockets won on one box, lost on another), IOMMU check, no numactl pinning (#3201) 2026-10-20 00:06:38 +02:00
Niko1221 0875029d10 docs: two-card gfx906 tip, --pipeline-windows 2 (+36% decode) and --trim-stage-weights (+5% prompt) measured on 2x MI50, off by default (#3173) 2026-10-20 00:06:38 +02:00
Niko1221 952cce7752 GPU choice (0.2.1): Automatic skips cards the engine cannot run, one card sent as gpu, settings-file GPUs kept, readable dropdown, Monitor opens on assigned cards (#3203 #3195 #3256 #3221 #3211) 2026-10-20 00:06:22 +02:00
Niko1221 30c64f09d7 Test fixes after the 0.2.1 merges: the gfx1151 table parse stops before the iGPU set, the benchmark's model-card link is not a model-host fetch, docker/build.sh copies bench/prompts/standard-v1 for the Terminal package, the Docker script tests use Git bash with unix paths on Windows 2026-10-20 00:05:48 +02:00
Niko1221 cbc0e329c6 Trellis MMQ overlay follows the ggml-cuda it patches: the generated callbacks and the loader pass the precision argument when mmq.cuh has it (llama.cpp b11xxx), identical output on the pinned commit, off with a message for an unknown signature (#3217) 2026-10-20 00:03:13 +02:00
Niko1221 f680a80d88 Batch groups: PROPOSED NVIDIA two-card default is one serial group (measured +12..16% on 2x A4000); owner decides 2026-10-20 00:03:09 +02:00
Niko1221 d8a6a0e012 App settings: the Allow browser origins help says it does nothing in the app, that the app's API already allows pages of any site, and where cors_origins really goes; text in all 8 languages and in DETAILS (#3208) 2026-10-19 23:52:04 +02:00
Niko1221 3cc18d7bb7 Chat: LaTeX math is drawn with KaTeX (inline $..$ and \(..\), display $$..$$ and \[..\]); KaTeX and its woff2 fonts ship with the app and load only when an answer has a formula; money and code stay text (#3268) 2026-10-19 23:49:59 +02:00
Niko1221 1994f3acb2 FP4-Local: setup installs the pinned torch, transformers and safetensors the converter imports (requirements-nvfp4-local.txt) right before a prepare that runs; the tool names a missing package up front instead of after 20 minutes (#3227) 2026-10-19 23:44:24 +02:00
Niko1221 c911065a14 Windows installer: ask for the install folder (another drive when C: is short), keep the earlier folder in app updates and silent installs; docs name what stays on C: (#3186 #3190) 2026-10-19 23:43:47 +02:00
Niko1221 40c605f7bc Windows with a non-UTF-8 code page (Russian): PowerShell output is read as UTF-8 (Documents folder, save dialog, fonts, GPU names), Python runs in UTF-8 mode from the launchers and the host's children, and the engine and picture reader get a UTF-8 manifest so a Cyrillic model folder arrives intact (#3202) 2026-10-19 23:38:49 +02:00
Niko1221 96a9107a27 Downloads: the MTP fetch shows progress in the app and survives a flaky link (runs again, fewer connections, one writer per folder); a cancel and a killed setup end the whole tree on Windows (snapshot + the fetch ends with its parent); Gyro's low-RAM pack gets --compat-bf16 (#3257 #3214 #3193) 2026-10-19 23:38:38 +02:00
Niko1221 4e76d6c9a6 HIP prefill: no start-up hipBLAS warm-up when the cards are of more than one architecture (#3204)
f3be9743 warms the first BF16 and FP16 products of every device at start. On a layer split across a
gfx1151 and a gfx1201 card it slowed prompts about 25% (hc read of the gfx1151 stage 2.7 -> 9.0 s per
32K block; STRATA_PREFILL_WARM=0 restores it). Two cards of one architecture lose nothing (2x R9700,
interleaved pairs). The warm-up now runs only when every visible card has the same gcnArchName;
STRATA_PREFILL_WARM=1 asks for it anyway. Policy in warm_policy.hpp with warm_policy_test.
2026-10-19 23:37:11 +02:00
Niko1221 8441cfd0e2 App: a start that stopped for no VRAM left for the expert cache is tried once more with less kept free (#3192); a model on the person's own GGUF folder keeps that folder, so a reload does not download it (#3199) 2026-10-19 23:31:43 +02:00
Niko1221 9cc2a9b6aa docs: build from git on Linux (clone, setup, rebuild, trying a pull request, engine env, the monitor from a checkout, plain cmake) (#3191) 2026-10-19 23:27:09 +02:00
Niko1221 9b8436df4e tools: gguf-py is also looked up in the install folder (STRATA_INSTALL_DIR, the app's folder), so tools run by hand from the Control package find what setup.py downloaded; the error names STRATA_GGUF_PY (#3269) 2026-10-19 23:26:32 +02:00
Niko1221 c6a20b7e85 Linux: say it when a finite RLIMIT_RTTIME (PipeWire without rtkit) is the reason for an engine death by signal 9, warn at start, troubleshooting entry (#3238) 2026-10-19 23:25:48 +02:00
Niko1221 88b29d481a CPU pool: the LP-E cores (/sys/devices/cpu_lowpower/cpus) are efficiency cores, in the engine and in setup (#3262)
On Meteor Lake and Arrow Lake-H the LP-E cores are in neither the cpu_core nor the cpu_atom
list, so the pool and setup counted them as P-cores: a Core Ultra 7 255H read as 8P+8E, the
more-E-than-P default plan did not apply and 15 workers ran (-35%). A CPU in no list now falls
back to its capacity. Tests: pool_plan_test (helper), tools/test_setup_hybrid.py (fake /sys).
2026-10-19 23:25:46 +02:00
Niko1221 2efa4e067a Merge branch 'f-021-minipc' into f-021-int 2026-10-19 22:20:52 +02:00
Niko1221 eb7eac250b Radeon 780M: the matrix-core prompt set is the default on gfx1103 (integrated, Linux): prompts x1.6 at 1K and x2.0 at 4K, decode unchanged, same greedy ids; STRATA_IGPU_PROMPT_SET=0 turns it off; gfx1150/1151 and Windows stay off; docs note the MES reset issue; per-arch tests 2026-10-19 22:20:33 +02:00
Niko1221 9f5488e3b2 Merge branch 'f-021-arcvis' into f-021-int
# Conflicts:
#	docs/MODELS.md
#	strata_control/vision_reader.py
#	tools/vision/CMakeLists.txt
#	tools/vision/strata_vision.cpp
2026-10-19 21:42:12 +02:00
Niko1221 cac4816afd Intel Arc pictures: on by default for the Alchemist cards (CPU reader), Battlemage stays opt-in; SYCL GPU picture reader (strata-vision-sycl) for --vision gpu; reader rule in vision_reader.py; tests and docs 2026-10-19 21:40:50 +02:00
Niko1221 4ed8eb02fa hipBLASLt tuning table on by default on Strix Halo (gfx1151) only, the owner's choice; opt-in elsewhere 2026-10-19 21:37:42 +02:00
Niko1221 b203b5f71e arch_defaults: UMA arena is not a default (measured slower, resets not fixed): constant off, STRATA_ARENA_PIN_GIB=none stays opt-in; docs say the iGPU prompt set is opt-in with the 780M numbers 2026-10-19 21:37:25 +02:00
Niko1221 468e3a6252 Merge branch 'f-021-hip' into f-021-int 2026-10-19 21:34:18 +02:00
Niko1221 c6aa4223d9 AMD packages carry the hipBLASLt tuning tables; the engine start picks the one for the card
Each AMD engine package gets tuning/<arch>-hipblaslt-<version>.txt for the architectures it covers and the
hipBLASLt version it bundles, and records hipblaslt_version in BUILD.json (packaging/stage_tuning.py for the
Linux build scripts, tools/hip/package_windows.py for the Windows zip). package_engine.py lists the tables in
the manifest and refuses an AMD stage without a version or with a table for another version or architecture.

serve/server.py with_component_engine sets STRATA_HIPBLASLT_TUNING from the package (strata_control/
hipblaslt_tuning.py): a table only for the exact arch and version, a log line for a mismatch, an unreadable
card or a missing version, STRATA_HIPBLASLT_TABLE=on|off (or "hipblaslt_table" in the run config) as the one
switch, and a STRATA_HIPBLASLT_TUNING the person set is kept. setup.py looks in the engine folder's tuning/
first.

A table changes the rounding of the prompt's GEMMs (R9700: residual differs, first-token KL 0.016), so the
automatic choice is off (AUTO_USES_TABLE = False) until the owner decides; "on" uses the table. No gfx1201
table exists for hipBLASLt 1.2.0 and one calibrated for it was not worth shipping (+1% prompt, -2% decode).
2026-10-19 21:33:47 +02:00
Niko1221 c23a78694e Merge branch 'f-021-macvis' into f-021-int 2026-10-19 21:27:28 +02:00
Niko1221 509466a668 Mac pictures: signing smoke checks strata-vision, docs and the 7 READMEs say the Mac reads pictures
sign-and-notarize.sh starts strata-vision --version in the signed engine folders (it already signs every Mach-O).
MAC.md carries the M4 numbers (0.45 s on the GPU, 50.7 s on the CPU for a 260-token picture) and the rule.
2026-10-19 21:26:49 +02:00
Niko1221 a10777a671 Merge branch 'f-021-R' into f-021-int 2026-10-19 21:26:01 +02:00
Niko1221 bcf67d1774 Merge branch 'f-021-bench' into f-021-int 2026-10-19 21:23:46 +02:00
Niko1221 c8e5d2f3ce Docs: BENCHMARK_PROTOCOL (implemented, still draft: field mapping, Metal, versions, runner reading, SECURITY link), COMMUNITY_BENCHMARKS, TERMINAL; BENCH.bat, bench.sh and strata_bench.py point to strata bench run 2026-10-19 21:23:20 +02:00
Niko1221 6d41b37e76 strata bench run: the standard benchmark protocol (standard-v1 is the default suite); fixtures and COMMIT file in the Terminal archive; tests with a mock server 2026-10-19 21:23:20 +02:00
Niko1221 e1ef9df21c Standard benchmark: frozen fixtures (bench/prompts/standard-v1) and the tool that builds them 2026-10-19 21:23:19 +02:00
Niko1221 dab7ee72d0 arch_defaults: the integrated-GPU matrix-core prompt set is opt-in again (bit-changing, owner's call); the Radeon 780M keeps its expert arena unregistered by default on Linux (UMA mode); per-arch selection tests 2026-10-19 21:21:50 +02:00
Niko1221 6b28dbbd1e Merge branch 'f-021-dock' into f-021-int 2026-10-19 21:21:22 +02:00
Niko1221 e0e0c0545a Docker: DOCKER.md records the rocm-gfx120x card test of the 0.2.0-beta packages (Radeon AI PRO R9700) 2026-10-19 21:20:52 +02:00
Niko1221 8acee5a055 Docker: build.sh and publish.sh exit 0 after a good run; DOCKER.md records the 0.2.1 image tests
- The EXIT trap of build.sh (--ctx-only or --ctx given) and publish.sh (--work given) ended with a failing test, so dash reported exit 1 after a
  successful run. Both traps now use if/fi. Test: build.sh --ctx-only exits 0 and its context holds docs/TERMINAL_LAUNCH.md.
- docs/DOCKER.md: image sizes, the 2026-10-19 test of the images built from the 0.2.0-beta packages (cuda13 and cuda12 on an RTX A4000, ROCm images
  built and checked, wrong-card message).
2026-10-19 21:16:34 +02:00
Niko1221 dca9770ef4 Task board: Schedule | Board view on Tasks, derivation with tests, loop record, sidebar count, all 8 locales 2026-10-19 21:05:20 +02:00
Niko1221 ff18fbb868 Task board host side: dismiss mark, newest problem only, one plain line per run, pending approvals of every chat 2026-10-19 21:05:20 +02:00
Niko1221 cf4749276d Mac pictures: strata-vision with Metal built in, Mac setup and launch turn pictures on
tools/vision builds one program for the Mac: ggml's Metal backend inside, --gpu reads on the GPU, without it on the CPU. The reader
choice (strata_control/vision_reader.py) gives a Mac the GPU reader whatever its memory (the CPU reader needs ~50 s per picture:
ggml has no BF16 vector kernel on ARM). setup.py's Mac path downloads the mmproj and writes the vision section; strata launch,
the MCP server and metal/setup_mac.py default pictures on. packaging/build_vision_mac.sh builds the program for the engine package.
2026-10-19 20:59:29 +02:00
Niko1221 578bb6aa7a Docker: publish the images from 0.2.1 (workflow, signed-feed fetch, local script)
- .github/workflows/docker-images.yml: manual only (workflow_dispatch) with the release tag as input; a plan job lists the flavors the
  release has an engine package for; one job per flavor builds the image and, only with push on, pushes ghcr.io/niko1221/strata:<version>-<flavor>
  and latest-<flavor> with GITHUB_TOKEN (packages: write).
- docker/fetch-engines.py: downloads the engine package from the release and checks it against the signed update feed (feed signature, archive
  size and SHA-256, manifest hash, package signature, every file), the same code as `strata update`. Flavors cuda13, cuda12, rocm-gfx120x,
  rocm-gfx110x, rocm-gfx1151 (the halo package), sycl (only if the feed lists one).
- docker/publish.sh: the same steps by hand (fetch, build, start check, push only with --push).
- Fix: docker/build.sh did not copy docs/TERMINAL_LAUNCH.md, which package_terminal.py needs; no image could be built from a clean tree.
- Dockerfile: OCI source/url/documentation labels (ghcr links the package to the repository).
- Compose files and docs/DOCKER.md name the 0.2.1 tags on ghcr.io; DOCKER.md has a maintainers section. Tests for the scripts, the workflow and the fetch.
2026-10-19 20:57:22 +02:00
Niko1221 1073929c80 [Other] serve.test_parallel: split in two files, each about a minute
Not a hang: 42 tests against a fake engine that paces real time took 127 s in one module and passed. The yield/failure tests move to serve/test_parallel_yield.py; the shared fake-engine helpers are FakeEngineCase.
2026-10-19 20:54:34 +02:00
Niko1221 e5dc00c81e docs: integrated GPUs (mini PCs) - the matrix-core prompt set is on by default, numbers, off switches, Windows untested 2026-10-19 20:38:55 +02:00
Niko1221 1c3c6d3351 hip: STRATA_ARENA_PIN_GIB=none leaves the expert arena unregistered (UMA mode for an integrated GPU: no KFD userptr, resident through the working-set lock); opt-in, not default 2026-10-19 20:38:55 +02:00
Niko1221 ce968a2fac hip: integrated Radeons (gfx1103/gfx1150/gfx1151, UMA flag) take the matrix-core prompt set by default; STRATA_IGPU_DEFAULTS=0 and each switch =0 turn it off; unit test with mock devices 2026-10-19 20:38:55 +02:00
Niko1221 40d1a3e7d7 [TUI] strata connect --write: a test with no keyboard, and an ended input counts as no 2026-10-19 20:38:37 +02:00
Niko1221 a269d1b983 [Other] engine hash tests: mock the release probe, keep the engine folder in a temp dir
get_prebuilt has probed the release with a HEAD request since 0.2.0; the refusal tests reached GitHub, got a 404 and compiled instead of refusing. The code is right, the test was stale.
2026-10-19 20:38:04 +02:00
Niko1221 9d2cb4f6d2 [Other] --data-dir never moves the model library without a choice
A --data-dir that names a new folder used to move Models, packs and mtp into it and rewrite the saved data_dir. Now the terminal asks (keep or move), --yes and a missing keyboard keep the old library in place (still found as a root), --move-data moves, and the saved default changes only on a move, --set-default-data-dir, or a PC with no models. Tests use temp folders.
2026-10-19 20:37:29 +02:00
Niko1221 74e25e165b Merge branch 'f-021-hostA' into f-021-int
# Conflicts:
#	apps/host/src/locales/de.json
#	apps/host/src/locales/en.json
#	apps/host/src/locales/es.json
#	apps/host/src/locales/ja.json
#	apps/host/src/locales/ko.json
#	apps/host/src/locales/pt.json
#	apps/host/src/locales/ru.json
#	apps/host/src/locales/zh.json
2026-10-19 20:30:04 +02:00
Niko1221 f5690d51a8 web-ask test: one data folder per bridge so the run exits; the scheduled-task case checks the refusal and the report 2026-10-19 20:25:20 +02:00
Niko1221 e2071c2f8b Downloads: 16 connections, several pieces per request, ModelScope length probe, faster MTP fetch
A single connection to Hugging Face's CDN stalls to 1-3 MB/s for 10-30 s again and again on the
owner's route; eight connections hide it, but a request per 32 MiB piece paid TCP slow start every
time. A request now covers up to four neighbouring pieces (the piece list and resumes are unchanged),
a try that moved bytes forward does not count as a failed one, and the default is 16 connections.
A server whose HEAD gives no length (ModelScope) is asked for bytes 0-0 so it gets the connections
too. mtp_fetch reuses the CDN address, reads the 28 shard headers side by side and lets the
connections run ahead of a slow range.
2026-10-19 20:07:33 +02:00
Niko1221 f2345ebd66 The model page and first-run page show the engine's own pick for this PC (memory tiers, Gyro-M as the other choice); the fit counts the draft layer (F) 2026-10-19 19:56:02 +02:00
Niko1221 671f4573da Fit checks and slot sizing count the draft layer's ~1 GB of graphics memory while MTP is on 2026-10-19 19:56:01 +02:00
Niko1221 3d0a79ca23 SAFETY.md: update checks only on a button press, web search asks once per chat in Manual, starter list refused 2026-10-19 19:54:54 +02:00
Niko1221 f523a94227 Chat footer speed: tool-call writing time counts as decoding time, so tok/s is output tokens over decode time 2026-10-19 19:54:48 +02:00
Niko1221 267f35f777 Chat with no model: plain words from the host, nothing sent before the engine's first state is known; tests for every state 2026-10-19 19:54:48 +02:00
Niko1221 71bdfda0b3 Offline web search: a pretend-offline mode for the search plugin and a test that the model gets the plain sentence 2026-10-19 19:54:48 +02:00
Niko1221 7662bf85c4 Starter model list: with the real engine on, the host refuses to download from it (unpinned files); it is display only 2026-10-19 19:54:47 +02:00
Niko1221 44a56865f5 Manual mode asks once per chat before the first web search or page; Settings, Tools switch to skip the question (off by default) 2026-10-19 19:54:37 +02:00
Niko1221 7f037f4451 Terminal: 'already downloaded' for a model on disk; strata monitor finds a running launch without --url (K) 2026-10-19 19:48:18 +02:00
Niko1221 5c9884cdee A build for another make of graphics card says so in plain words with the right download and is not unpacked (I) 2026-10-19 19:40:57 +02:00
Niko1221 5b4db785ca Start-up line: an installed engine with no model loaded is not 'No Strata engine' (J), and the wrong-make notice wiring 2026-10-19 19:40:57 +02:00
Niko1221 321f159032 Offline: the update check says there is no internet connection in plain words; offline start and load test 2026-10-19 19:40:05 +02:00
Niko1221 54a02dc32c Settings: opening Engine and Update makes no network call; the last check is shown and a new one runs only on a button press 2026-10-19 19:40:05 +02:00
Niko1221 40bf5f5c40 Merge branch 'f-021-settings' into f-021-int 2026-10-19 19:34:47 +02:00
Niko1221 bc54d945f6 Merge branch 'f-021-crash' into f-021-int 2026-10-19 19:34:46 +02:00
Niko1221 7a2bfc52ae Models: a per-model draft layer (MTP) switch, picture-off and custom context that stick
The installed app listed every model's pictures as "off", so each Load of a picture-capable model ran
setup again, which writes the arguments (--mtp, --spec-min-p, --vision, --max-context) and the vision
section from its own choices. Hand edits and the Read pictures tick came back each time.

- host: images comes from the run config's vision section, so a prepared model loads straight away
- host: Read pictures off no longer downloads/keeps the reader; --vision is left out of the engine arguments
- host: Draft layer (MTP) setting; "MTP per request" and it are the app's own and no longer frozen
- host: a context size set by hand shows in the Context list
- "mtp": false in the run config (strata_control/draft_layer.py) is applied at every start by
  serve/server.py, kept by setup, written by `setup --mtp off` and `strata launch --mtp off`
- strata_install accepts any context size, not only the listed ones
- docs: draft layer off, and what the app owns in the run config
2026-10-19 19:29:39 +02:00
Niko1221 e88ee39d1d Docs: state exactly which refusal checks ran (Linux packages) and the Windows PATH note for --version 2026-10-19 19:18:52 +02:00
Niko1221 a5ee74ffa3 Desktop: a failed start on Windows and Mac leaves a log, a crash file and a message box instead of 0xc0000409
Reported crash (0.2.0-beta, Windows 11 26300): strata-desktop.exe exits with 0xc0000409 at 0x31a315 and no log. That offset is the
fast-fail in Rust's abort after a panic (panic = "abort"); a Tauri setup error is a panic ("Failed to setup app"), and the shell had no
log on Windows, no console (windows_subsystem), so the reason was lost.

- the run log opens first in main (%LOCALAPPDATA%\Strata\logs, the folder Save logs already reads) and a panic hook writes the
  message and place to it and to desktop-crash-<time>.log, and shows a plain message box before the abort
- setup no longer returns an error to Tauri: a host that does not start or a window that cannot be made is reported the same way
  and ends the app with exit code 1; the build error is no longer an expect
- the host's output goes to the run log (it was lost without a console)
- plain next step in the box for a WebView2 problem and for a blocked host
- STRATA_NO_DIALOG (scripted runs) and STRATA_DESKTOP_TEST_PANIC (tests) for proving the net
2026-10-19 19:17:43 +02:00
Niko1221 74652692e1 Release matrix tests: Windows rdna3 (RX 7000, Strix Halo) and Linux halo builds, no arch in two hip builds of a platform 2026-10-19 18:32:56 +02:00
Niko1221 85f77658f3 Docs: RX 7000 and Strix Halo installers (Windows rdna3, Linux strixhalo) in the READMEs, AMD_HIP, STRIX_HALO and the 0.2.0 release notes 2026-10-19 18:27:13 +02:00
Niko1221 f57c0adcb3 Merge branch 'f-020-reset' into f-020-int 2026-10-19 17:12:10 +02:00
Niko1221 3fa1b524d7 Portable monitor bundle: carries consoleinfo.py and overview.py (manifest schema 6), the plain Monitor runs when launch is not bundled, smokes accept the Overview's /metrics read, test derives the bundle's import closure 2026-10-19 16:35:51 +02:00
Niko1221 76e51103c7 Merge branch 'f-020-ci' into f-020-int 2026-10-19 16:26:40 +02:00
Niko1221 fcca2ccb84 CI smokes: serve smoke expects the server-package preflight, server smoke accepts OK (skipped=N), nested-telemetry fixture deep enough for Python 3.12+ 2026-10-19 16:23:26 +02:00
Niko1221 eac694ae4c Issue and PR reset bot: intake check, /reopen, 5-day sweep, with node tests 2026-10-19 16:10:26 +02:00
Niko1221 4caf857c76 README translations: the new app and Terminal screenshots with translated captions 2026-10-19 16:03:04 +02:00
Niko1221 e5d987a553 Merge branch 'f-020-shots' into f-020-int 2026-10-19 16:02:32 +02:00
Niko1221 dde39e8a8f README app screenshot: crop the chat footer (its speed figure is wrong, see the 0.2.1 plan) 2026-10-19 16:02:32 +02:00
Niko1221 03b75b8da9 README: new app and Terminal screenshots for 0.2.0-beta 2026-10-19 15:52:41 +02:00
Niko1221 3d1fb48090 Release notes 0.2.0: which file to download, memory tiers, faster checked downloads, the chat fixes, signed Mac 2026-10-19 15:29:09 +02:00
Niko1221 042cd201c9 Merge branch 'f-020-tr2' into f-020-int 2026-10-19 15:04:28 +02:00
Niko1221 b961c7aa8c README: use the app's own UI strings in the ja, pt-BR and zh-CN translations 2026-10-19 15:04:16 +02:00
Niko1221 6ec549d2dd README de/fr/es: use the app's own labels, French keeps English UI names; tidy colon asides 2026-10-19 15:03:27 +02:00
Niko1221 4ff5972fcb README: new Japanese, Brazilian Portuguese and Simplified Chinese translations of the English README 2026-10-19 15:02:13 +02:00
Niko1221 04a2e6a3f1 README: new German, French and Spanish translations of the 0.2.0 README 2026-10-19 15:01:57 +02:00
Niko1221 c301209636 README: the Windows AMD app is for RX 9000 / R9700 cards; other AMD cards point to AMD_HIP.md 2026-10-19 14:57:39 +02:00
Niko1221 78bba245a5 README: file-per-card download table, what-you-need before install, shorter blocks; drop the Mac damaged workaround 2026-10-19 14:57:15 +02:00
Niko1221 18f7b90ca6 Feed: the app entry may carry a pre-release version (0.2.0-beta); the strict key rejected the whole release feed; UPDATES: the release steps for the engine-less update installers, --url-prefix=, Control version 2026-10-19 14:01:29 +02:00
Niko1221 1b8e88b9e5 feed_tool: a Linux or Mac package packed on Windows keeps its programs executable (mode from the file header); test 2026-10-19 12:19:17 +02:00
Niko1221 ef59c0b555 Merge f-020-int (final 0.2.0-beta source 4ea817163) into f-020-rel 2026-10-19 11:53:08 +02:00
Niko1221 4ea8171636 Merge branch 'f-020-dl' into f-020-int 2026-10-19 11:51:12 +02:00
Niko1221 3c3d2881b4 Merge f-020-int into f-020-rel 2026-10-19 11:35:31 +02:00
Niko1221 604e23cd9a Engine-less app update: tauri.update.conf.json drops engine-bundle so one latest.json entry per OS fits every installer variant; tests for the overlay, for the engine-less update on an installed AMD engine, and for the release engine selection (CUDA 13/12 by driver and card, HIP by gfx, Metal) 2026-10-19 11:35:23 +02:00
Niko1221 41cc4b2e89 Plans: no step counter in the progress lines (owner: 'Step 2 of 5' removed like the goal's) 2026-10-19 11:35:11 +02:00
Niko1221 3d9c21509e Merge f-020-int into f-020-rel 2026-10-19 11:32:19 +02:00
Niko1221 779bc30501 Merge branch 'f-020-gui' into f-020-int 2026-10-19 11:29:58 +02:00
Niko1221 d5bcc3bd8d GUI fixes: wait for the assistant to start, no step counter on goals, files open without a chosen folder, results open in the sidebar 2026-10-19 11:29:40 +02:00
Niko1221 116adbf7d6 tester README (Linux NVIDIA): the driver line names the CUDA 13 engine's minimum, 580.65 2026-10-19 10:53:25 +02:00
Niko1221 19d5cd87fa install.ps1: name a discrete graphics card before the built-in one in the line it prints 2026-10-19 10:53:24 +02:00
Niko1221 2ca90e48ec binary_deps: a Windows DLL found only in a sub-folder counts as bundled only when BUILD.json lists that folder in lib_dirs (the RX 9000 package's rocblas.dll under rocm/bin made a plain start die with 0xC0000135) 2026-10-19 10:53:24 +02:00
Niko1221 adfb7ecb2a packaging: the Control python launcher uses no external command (dirname failed inside the AppImage on newer systems); the engine start error names a missing Windows DLL (0xC0000135) and says how to fix it 2026-10-19 10:52:51 +02:00
Niko1221 a122ba0bdf Mac signing: lessons of the first real run (entitlements plist LF everywhere, TMPDIR's trailing slash no longer makes a // path, the Terminal python check finds the python3 symlink) 2026-10-19 10:50:55 +02:00
Niko1221 04221c3e2b Download big model files over several connections
download() reads a file of 256 MiB or more in 32 MiB ranged pieces over 8 connections
(STRATA_DL_CONNECTIONS=1 restores one stream), writes each piece at its own offset,
and keeps the finished pieces in <name>.part.chunks so a stopped download continues
without fetching them again. The redirect to the signed CDN address is resolved once
and again before it expires or after a 401/403/410. A server without Range support
or without a length is read in one stream as before; an older single-stream .part
continues after its last whole piece. The progress line and the .part size the app
reads are unchanged, the SHA-256 pin is still checked on the whole file, and the
.done mark is the same. tools/mtp_fetch.py reads its ranges in parallel too.
2026-10-19 10:38:35 +02:00
Niko1221 fdb5e48616 Merge branch 'f-020-macrel' into f-020-int 2026-10-19 09:33:22 +02:00
Niko1221 af78949b17 build-feed writes install-manifest.txt from the Control packs (what install.sh and install.ps1 read); test of the release path; docs 2026-10-19 09:28:23 +02:00
Niko1221 935407d883 Mac in-app update: sign-and-notarize.sh --updater-archive packs the notarised app; feed_tool set-app-platform verifies the Tauri signature and writes the update entry
tauri build makes Strata.app.tar.gz and its .sig before sign-and-notarize.sh re-signs and notarises the app, so the published Mac update would be the unsigned app. The script now writes Strata.app.tar.gz from the stapled app on request; the Tauri key stays on the PC (npx tauri signer sign), and set-app-platform checks the .sig against tauri.conf.json's pubkey over the exact bytes and records url, signature, sha256 and size.
2026-10-19 09:18:13 +02:00
Niko1221 899d1b1b3a metal: restore the stubs for gemm_iq_f16_grouped, native_q8_0_sm60_pack/_release and swiglu_split_f16 (CUDA-only paths the Metal engine links but never runs)
The Metal engine did not link from f-020-int: prefill.cpp and native_head.cpp reference these four entry points. All answer 'nothing done'.
2026-10-19 09:17:56 +02:00
Niko1221 02a8193da7 test_strata_mcp: the Windows AMD recommend test no longer reads this PC's installed engine (hermetic; fails on a PC with a CUDA engine since e107297a8) 2026-10-19 09:07:01 +02:00
Niko1221 c7bfd131b5 Merge branch 'f-020-int' into f-020-golden 2026-10-19 08:51:07 +02:00
Niko1221 6f2a3e758d Merge branch 'f-020-sha' into f-020-int 2026-10-19 08:41:46 +02:00
Niko1221 e193161de2 Pin SHA-256 and size for every model file setup downloads; stop when the pinned version is gone
Qwen3.8-Flash-Next (ISTA), Swift 1.5, the Coder and the picture reader get fixed
hashes (REPO_PINS, checked in download()). A 404 on the pinned commit stops instead
of taking the current file; STRATA_ALLOW_UNPINNED=1 is the announced tester opt-out
(setup.py and tools/mtp_fetch.py). Swift moves to the commit that has its IQ3_S.
Tests for pins, mismatch, 404, opt-out and offline use; SAFETY.md updated.
2026-10-19 08:41:24 +02:00
Niko1221 06b6b23e51 Merge branch 'f-020-int' into f-020-golden 2026-10-19 08:38:10 +02:00
Niko1221 553c25ca7c Merge branch 'f-020-tiers' into f-020-int 2026-10-19 08:36:15 +02:00
Niko1221 8ac47258bb tests: wizard tests pick IQ3_XXS by hand now that IQ3_S is pre-selected at 64 GB 2026-10-19 08:35:13 +02:00
Niko1221 dbd1cb9af8 setup golden: re-record the run configs after pictures-on-by-default (91baead75) and the memory tiers (aa82a8863); classification in phaseA-tests/golden-020.md 2026-10-19 08:34:26 +02:00
Niko1221 94f095c893 tests: launch TUI pins follow the 64 GB tier (IQ3_S) 2026-10-19 08:33:19 +02:00
Niko1221 a4c1644a37 Merge branch 'f-020-tiers' into f-020-int 2026-10-19 08:30:20 +02:00
Niko1221 aa82a88639 Recommend by the owner's memory tiers (RAM + VRAM): 40 GB Swift 1.5 IQ2_XS with Gyro-M named, 48 GB IQ3_XXS, 64 GB IQ3_S; MODELS.md Pick by memory 2026-10-19 08:26:54 +02:00
Niko1221 9962aae5b7 Merge branch 'f-020-safety' into f-020-int 2026-10-19 08:16:03 +02:00
Niko1221 cbbdd16b43 SAFETY/README: no 'nothing leaves your PC' overclaim (web search is on by default); neutral wording for Mac signing 2026-10-19 08:16:03 +02:00
Niko1221 25492e4a6b Docs: add SAFETY.md (what Strata downloads, when it uses the internet, what stays local, how downloads are checked, what the agent may do) and link it 2026-10-19 08:15:11 +02:00
Niko1221 7e3a9fec71 tests: DOCKER.md no longer carries TODO(owner); check the not-published line instead 2026-10-19 08:13:25 +02:00
Niko1221 2a6650a364 README: memory tiers per the owner (40 GB Swift 1.5 or Gyro-M with its speed note, 64 GB IQ3_S original or Swift), low-RAM line, Gyro-S vs IQ2_XS 2026-10-19 08:11:47 +02:00
Niko1221 5e7b9a8301 Docs: plain-wording pass on the 0.2.0 text (fewer mid-sentence bolds, direct phrasing) 2026-10-19 08:01:47 +02:00
Niko1221 2b209b406d Docs consistency pass: pictures on by default on NVIDIA and AMD (Intel Arc off for now), seven monitor views, area prefix for issue and PR titles in AGENTS.md 2026-10-19 08:00:26 +02:00
Niko1221 6ff75059a9 Merge f-020-docs3 into f-020-int (0.2.0 release wiring and docs)
# Conflicts:
#	packaging/UPDATES.md
2026-10-19 07:59:35 +02:00
Niko1221 e46be1fe0a Merge f-020-docs2 into f-020-int (0.2.0 release wiring and docs) 2026-10-19 07:59:18 +02:00
Niko1221 f86ca9d1c5 Merge f-020-docs into f-020-int (0.2.0 release wiring and docs) 2026-10-19 07:59:17 +02:00
Niko1221 477fd36bce Merge f-020-release into f-020-int (0.2.0 release wiring and docs) 2026-10-19 07:59:17 +02:00
Niko1221 193cbb1fbf README redesign: logo header, clearer install per system, memory rule, Gyro for low memory, library analogy, support section
Header uses the existing brand lockup. Performance tables get bars and plain labels, with the engine versions left to DETAILS. What you need is built around 32 GB of RAM plus VRAM and links the platform docs. Troubleshooting moves below How does it work.
2026-10-19 07:55:50 +02:00
Niko1221 76e4fef398 Intel Arc pictures stay opt-in until the SYCL engine's first-picture GPU fault is fixed; UPDATES: publish as a normal release 2026-10-19 07:48:06 +02:00
Niko1221 dac635445b CONTRIBUTING.md: a Titles section (area prefix for issues and pull requests) 2026-10-19 07:42:35 +02:00
Niko1221 046bdc83d9 Titles start with their area: Area dropdown and title prefix in the issue forms, a benchmark form, a PR template line, a title-area workflow 2026-10-19 07:42:27 +02:00
Niko1221 667d3cf7d7 Intel Arc: pictures on by default through the portable CPU reader (setup_intel, shared /dev/shm for the embeddings) 2026-10-19 07:28:38 +02:00
Niko1221 4794b13779 strata launch --help states the picture rule; strata monitor is the seven-view monitor (dead six-view fallback removed), tests 2026-10-19 07:28:38 +02:00
Niko1221 09b8b54d19 Release app config: the engine travels inside every installer, signed updater artifacts overlay, macOS 15 minimum 2026-10-19 07:28:37 +02:00
Niko1221 98b865c56f Release wiring: release feed key and Tauri updater key, GitHub Releases feed address in the installers, versions 0.1.45.3 and 0.2.0-beta 2026-10-19 07:23:44 +02:00
Niko1221 91baead754 Pictures on by default on every make of card: MCP install, Qwen3.6, Unsloth and the original Flash-Next families follow the one reader rule (CPU under 24 GB, GPU from 24 GB); the CPU reader keeps no VRAM reserve 2026-10-19 07:23:44 +02:00
Niko1221 80a10f31c3 docs: details, batching and tester readmes for 0.2.0
DETAILS: pictures on by default with the 24 GB reader rule, the engine label of the prompt-chunk planner (before 0.1.45.3, not 0.1.46), the per-architecture sets measured on 3060, A4000 and P100, the N7 and ROUTE_PRIOR decisions, a pointer to the app and Terminal troubleshooting. Links to private phaseA-tests notes removed from DETAILS and BATCHING. Tester readmes: AMD pictures are on (CPU under 24 GB, GPU on a 24 GB 7900 XTX), the RX 7900 text reconciled with the community runs, Mac needs macOS 15 and has no picture reader yet.
2026-10-19 07:16:00 +02:00
Niko1221 b27c46cbd6 docs: Mac, AMD, Intel, older GPUs and multi-GPU for 0.2.0
MAC: the .dmg and the Terminal bring the Metal engine; pictures only with the source build. AMD_HIP: the 24 GB
picture rule and the measured R9700 GPU reader. INTEL: the grouped XMX label is engine 0.1.45.3 (not 0.1.48), plus
the A750 prompt-hang fix (fca3c5958; STRATA_PF_DMA_PAGEABLE=1 restores the old copy). INTEL_ARC: maintainers now test
on a B70 and an A750; where the Intel engine comes from. OLDER_GPUS: the P100 sm_60 defaults (STRATA_SM60_PERF).
MULTI_GPU: two models on two cards, one make per engine. MODELS and TROUBLESHOOTING: the Mac and Intel Arc have no
picture reader yet.
2026-10-19 07:15:14 +02:00
Niko1221 1c4970b9ac 0.2.0 release notes: the split, installers, Terminal, pictures, new defaults with measured numbers, tester fixes, known limits 2026-10-19 07:09:40 +02:00
Niko1221 26613a44f6 AGENTS, AI_SETUP, INSTALL: the app and the Terminal first, install from source as the fallback
Agents install the Terminal with the one line and run strata launch --model NAME --yes --no-tui (flags checked against
strata launch --help). Pictures follow the 24 GB rule on every make; the Mac has no picture reader yet. INSTALL gains
The app and The Terminal sections, update paths for both, and a Mac source section. AGENTS points to CONTRIBUTING.md.
2026-10-19 07:08:46 +02:00
Niko1221 afe92c1727 docs: models, NVFP4, Gyro, Qwen3.6 and troubleshooting for 0.2.0
Pictures are on by default (CPU reader under 24 GB, GPU reader from 24 GB, every make; no Mac reader yet).
Gyro and NVFP4-QAD read pictures; Gyro needs engine 0.1.45.1+ and is correct on RTX 50 since the SwiGLU guard.
The three NVFP4 versions get their app names. A Qwen3.6-35B-A3B section, the app and Terminal ways to add a model,
and engine versions on the older speed tables. Troubleshooting: no-engine warning, needs a newer engine / Update
engine, Mac xattr steps, strata doctor, Save logs. Private test-notes references removed.
2026-10-19 07:07:54 +02:00
Niko1221 ad262c80aa README: the app installers and the one-line Terminal install first; build from source moves lower
Pictures on by default (CPU reader under 24 GB, GPU reader from 24 GB, every make; none on the Mac yet), the Mac app,
the Terminal in a short section, the Anthropic base URL without /v1, two models on two cards, one make per engine.
The speed tables stay and say which engine measured them. A Contributing line links CONTRIBUTING.md.
2026-10-19 07:05:43 +02:00
Niko1221 14ec155ec7 0.2.0 text, group 3: Terminal docs (installed command, seven views, release address, connect --check json), Docker, packaging, SECURITY, issue and PR templates, test requests, historical Terminal pages 2026-10-19 07:04:59 +02:00
Niko1221 e9e08adb37 Add CONTRIBUTING.md and the draft standard benchmark protocol (owner text, unchanged) 2026-10-19 06:58:45 +02:00
Niko1221 fd0e3766a3 Merge f-split-tuiqa (95f380769) into f-split-int: TUI QA fixes (launch with flags, discovery, monitor menu, connect checks, layouts, no browser, warnings), picture reader CPU under 24 GB / GPU from 24 GB 2026-10-19 00:03:56 +02:00
Niko1221 95f3807699 Merge f-split-int d6aa6504e into f-split-tuiqa: launch warns and asks (setup stops only for NEED_ENGINE unless launch said go on), the app warns; one picture-reader rule in strata_control/vision_reader.py (CPU reader under 24 GB, GPU reader from 24 GB, every make; pictures stay on) for setup, launch and the app catalog 2026-10-19 00:03:31 +02:00
Niko1221 a2367ca55b Mac: one script to sign (Developer ID, hardened runtime) and notarise the app dmg and the Terminal archive; Mac README text for notarised and not notarised builds 2026-10-19 00:01:30 +02:00
Niko1221 e657d310e7 models import: a folder with several models says which ones it holds and asks for one file, in plain words 2026-10-18 23:33:52 +02:00
Niko1221 64af23dafc Terminal shows the app's version (0.2.0-beta) with the build number beside it; the archive README no longer contradicts itself, ships the launch doc it links and gives free-memory guidance for the system it is for; launch doc updated 2026-10-18 23:28:01 +02:00
Niko1221 bccb0f4280 Terminal wizard: the Save choices stay on screen at 60x20 and 40x15, the Advanced command has its own scroll area, key hints wrap and say what Enter does; request filter explains itself and matches LIVE/DONE; plain-word setting descriptions; tuning default explained; monitor bundle carries nav.py (T09, T11) 2026-10-18 23:23:09 +02:00
Niko1221 adc4a6822b Advanced command preview follows the wizard's context, memory format and picture choice (T09 part 1) 2026-10-18 23:00:01 +02:00
Niko1221 3d2e7f166e strata connect --check --output json runs the same checks and returns each result: ok is false and the exit code 1 when a check failed or the server is unreachable; the app list says Vendor tested (T07) 2026-10-18 22:58:57 +02:00
Niko1221 545fccf007 Terminal monitor: one seven-view model (arrows and 1-7 reach Connect, wrap, hints say 1-7), the selected tab always marked and the names kept at narrow widths, Overview titled Overview, Connect separates vendor-tested history from this server's live result, light theme for Connect and pop-ups (T06, T08, T10) 2026-10-18 22:57:36 +02:00
Niko1221 d6aa6504e0 Merge f-amd-vision (d5fda4d2f) into f-split-int: GPU picture reader for AMD (strata-vision-hip) with automatic CPU fallback; pictures on by default on AMD; packaging for Linux and Windows 2026-10-18 22:53:23 +02:00
Niko1221 d5fda4d2f3 build_windows.bat: STRATA_VISION_HIP_ARCHS 2026-10-18 22:50:16 +02:00
Niko1221 8667a3dbf7 Merge f-split-int (a3024ed10) into f-split-qa1: qa2's s3237-s3238 kept, my 21 strings renumbered s3260-s3280 2026-10-18 22:46:39 +02:00
Niko1221 b7b65edeb1 build_hip_linux.sh: VISION_HIP_DEFS 2026-10-18 22:46:29 +02:00
Niko1221 4440a303f8 Terminal: no browser opens by itself (launch config forbids it, --open is the only way), and the start message promises a browser only with --open 2026-10-18 22:46:10 +02:00
Niko1221 e107297a8f Mixed makes of graphics cards: only the cards the engine can drive are offered, never a cross-make split, one plain line when the other make has the bigger card, two models at once said to be unsupported (launch, setup, app card list) 2026-10-18 22:44:12 +02:00
Niko1221 6b8cfcd1a3 strata-vision-hip (Windows): hipblas.dll is delay-loaded and rocm\bin is put on the DLL search path first, so the reader starts from any PATH 2026-10-18 22:41:18 +02:00
Niko1221 d060783fbb Terminal: pictures are on by default when the model and the engine can read them (card for CUDA and HIP, CPU otherwise); --vision off turns them off 2026-10-18 22:39:16 +02:00
Niko1221 a3024ed10a Merge f-split-qa2c (10f3435b0) into f-split-int: Linux drag-drop through the page widget's GTK drag signals; export header test 2026-10-18 22:39:06 +02:00
Niko1221 c4345a7c03 Terminal: no engine for this computer is a warning, not a refusal: ask (default yes, --yes skips), download the model, end with the engine problem and the next step, exit 0 2026-10-18 22:33:43 +02:00
Niko1221 10f3435b00 Test: the saved Monitor picture with a model loaded is headed by the logo and the model and time line (NativeMonitor is not used by any screen) 2026-10-18 22:30:01 +02:00
Niko1221 35698ffe70 Linux: dropping a file on the window works under WebKitGTK 2.52: the shell reads the file:// addresses from the GTK drag signals, accepts the drag itself, takes the leave with every button up as the drop and hands the paths to the host 2026-10-18 22:28:58 +02:00
Niko1221 e6ab4c024d No recognised engine: the app still downloads the model and warns in its card; loading names the next step
setup.py control_engine_step stops only for strata launch (STRATA_NEED_ENGINE=1); the app warns and continues. The Models card and the first-run
download card show the owner's plain warning before and during the download; the load refusal says the same with Settings > Engine.
2026-10-18 22:28:15 +02:00
Niko1221 23b08c7620 Terminal: one discovery of existing installs and model files for the screens, models list, own-files import and launch; Gyro-S/M GGUFs are known; installed models show what is really missing (T03, T04, T05) 2026-10-18 22:22:31 +02:00
Niko1221 e23a56f2f8 ENGINE-COMPONENT.md: the picture readers in an AMD package 2026-10-18 22:19:49 +02:00
Niko1221 b0b8860d13 build_windows.bat: the HIP reader's host compiler gets the ROCm device-library path 2026-10-18 22:19:18 +02:00
Niko1221 1486151d4d Terminal: strata launch with flags opens its run screen instead of crashing, and a crash of the screens exits non-zero with a plain sentence (T02) 2026-10-18 22:11:21 +02:00
Niko1221 97cb031bf7 docs: AMD pictures run on the card by default, with the CPU reader as the automatic fallback 2026-10-18 22:04:25 +02:00
Niko1221 c59505e638 build_windows.bat: STRATA_ROCM_KEEP uses the ROCm venv as it is 2026-10-18 22:02:17 +02:00
Niko1221 34bd640a19 Merge f-split-qa2b (b3348da49) into f-split-int: the saved Monitor screenshot shows the logo alone in its top line 2026-10-18 22:01:44 +02:00
Niko1221 0f5511cfdf Merge f-split-qa2 follow-ups into f-split-int: drop guard (no navigation to a dropped file), shell-to-host drop hand-over 2026-10-18 22:01:44 +02:00
Niko1221 b3348da493 Monitor: the saved screenshot starts with the Strata logo alone, without the Monitor headline 2026-10-18 22:01:21 +02:00
Niko1221 b31c13875e AMD picture reader: the no-GPU/arch refusal only in the HIP program; the app's CPU choice starts the CPU reader program 2026-10-18 21:55:51 +02:00
Niko1221 b2e3008bfb build_hip_linux.sh: kernel data collected once 2026-10-18 21:51:33 +02:00
Niko1221 55ca7b955a launch: the Pictures screen says the AMD picture reader runs on the graphics card; build_hip_linux.sh collects libraries from the build outputs 2026-10-18 21:51:10 +02:00
Niko1221 979da9d2bc Terminal launch list: Qwen3.6 (own engine path) stays out of Flash-Next's rows 2026-10-18 21:44:49 +02:00
Niko1221 536c7f019e Windows HIP zip: build_windows.bat builds the two picture readers and package_windows.py ships them 2026-10-18 21:44:45 +02:00
Niko1221 de9101935d setup: AMD pictures on by default (the HIP reader with the CPU reader behind it, else the CPU reader), on Windows too; source build compiles the HIP reader; package_engine carries strata-vision-hip 2026-10-18 21:43:23 +02:00
Niko1221 f82fffbe25 picture reader choice for AMD engine folders and the server's fallback from the GPU reader to the CPU reader (one log line) 2026-10-18 21:39:13 +02:00
Niko1221 b808d6e442 Linux and macOS: dropping a file on the window works. WebKit gives the page the drag but not the files, so the shell keeps its drop handler, hands the paths to the host, and the page fetches each file once; a file dropped outside the chat no longer takes the window to it 2026-10-18 21:36:51 +02:00
Niko1221 75127c794e strata-vision: fix the escaped newlines 2026-10-18 21:34:28 +02:00
Niko1221 96658aebae packaging: build_hip_linux.sh builds the HIP engine package with both picture readers 2026-10-18 21:32:58 +02:00
Niko1221 fb14c354b4 strata-vision: HIP build (strata-vision-hip) for AMD GPUs; the reader refuses a GPU that is not in its build; RUNPATH $ORIGIN 2026-10-18 21:31:30 +02:00
Niko1221 43ea31544b native poll: a read that never answers is dropped after 60 s 2026-10-18 21:31:21 +02:00
Niko1221 a1788c54be Read pictures is on unless the person turned it off: an installed model with no saved choice starts checked 2026-10-18 21:30:17 +02:00
Niko1221 7f7bfa962b Merge f-split-qa3 (283258664) into f-split-int: web search back-off and Bing fallback, turn guard with a final message, /loop stops by count/time/condition, Anthropic base URL without /v1 2026-10-18 21:25:06 +02:00
Niko1221 2832586641 Chat stream: translate the app's own delta lines before the delta branch, so the answer-text code stays as merged 2026-10-18 21:17:43 +02:00
Niko1221 a04244a4be Merge f-gyro-swiglu (8837cb6f0) into f-split-int: the fused gate/up SwiGLU path takes only 80-dot rows; Gyro's trellis types fall back to the full-warp kernel (correct Gyro output on sm_120) 2026-10-18 21:13:13 +02:00
Niko1221 714ca744dc Merge f-split-int (c157ea373) into f-split-qa3 2026-10-18 21:10:12 +02:00
Niko1221 e5476b0e12 Merge f-split-int (c157ea373) into f-split-qa1: my new strings renumbered after s3236 (s3237-s3255) 2026-10-18 21:09:33 +02:00
Niko1221 fef5dfec03 Storage list full width; version list title keeps its spaces 2026-10-18 21:08:41 +02:00
Niko1221 c157ea373f Merge f-split-qa2 into f-split-int: drag-drop in the native window, files-edited count only from tool changes in the workspace, answer-text guard, dialog focus, skill enabled flag, number field checks, folder picker release, small-window composer, Monitor screenshot feedback 2026-10-18 21:07:47 +02:00
Niko1221 40b41da916 Turn guard: a search-heavy /plan that keeps failing is stopped by the app, and a full context never ends in silence
A /plan on a small model searched again after every refused search and rewrote its plan after each one until the context reached 100%,
then the turn ended with no answer and no message. The bridge now counts what the turn shows (turn-guard.mjs): 3 failed searches in a
row, 10 searches in a plan turn (25 otherwise), a plan written from 1. again 4 times, a local model's context at 93%. The turn is stopped the
way Stop stops it but ends as a normal turn: the plan it had is still proposed (the plan card says why it stopped early), any other turn
ends with one plain translated sentence with a next step (frames carry the English sentence as `key`, the page translates it, 8 languages),
and the Harness's own /compact is asked for so the next message has room. A turn that ends with no words at all always gets a message:
"I ran out of room in this chat..." at 85% of the window or more, otherwise "I stopped without writing an answer...". Settings -> Tools says
results may come from Bing when DuckDuckGo is busy. Tests: counting (pure) and three end-to-end runs through the real Harness with the scripted
model (searchloop / bloat) and the real search code on a pretend web that refuses (bridge option searchMock), no network.
2026-10-18 21:07:38 +02:00
Niko1221 31030b170a storage and version: one Downloaded models list from the real files, Check folder unloads nothing, app version shown as Strata
- tools/strata_mcp.py: disk usage from the files a model really uses (every part of a multi-part GGUF, the picture reader, packs, draft layer),
  a file named by several flags or shared by several models counted once; Delete's plan stays separate (removable_gb)
- Settings > Storage: one list and one total (optimized models and other local models), shared files as a line of their own
- Check folder on the same folder no longer unloads the loaded models (O01); an actual change still asks first
- Settings shows the app version (0.2.0-beta) as Strata with the engine beside it; Control's 0.1.45 is no longer shown as Strata
  and no longer goes into the update check as the app version
2026-10-18 20:55:06 +02:00
Niko1221 b23dad1d5f scripts/qa2-flows.mjs: browser checks for the QA fixes (dialog focus, number fields, small window, picture drop, screenshot) 2026-10-18 20:54:29 +02:00
Niko1221 8837cb6f0f gu_swiglu: the 80-dots-a-row guard for every format (Gyro's trellis gate/up rows went through the 16-lane kernel)
launch_gu_swiglu (STRATA_GU_SWIGLU, on by default on compute capability 12.0 cards since 147-verify) runs
16-lane gate/up rows that assume 80 dots a row, but its guard rejected only the K-quants. The APR trellis
types (TQ2_T / TQK6 gate/up: qk 128, ipb 8, 160 dots at n_embd 2560) went through it: half of each row,
Gyro-S answered with loops, garbage or nothing on every RTX 50 / RTX PRO Blackwell card. They now take
launch_gu's full-warp kernel, as launch_gu itself always did for them; 80-dot formats are unchanged.

l620 (RTX PRO 5000, Gyro-S through serve/server.py, the owner's QA prompts, greedy and the app's sampling):
before 0/8 correct, after 8/8 ("STRATA_OK 42", "4", the follow-up sentence, "PING"), the same as
STRATA_SM120_PERF=0 / STRATA_GU_SWIGLU=0; Gyro-S fix vs STRATA_SM120_PERF=0: identity gate 10/10, exact_pp
4K / 20K IDENTICAL (KL 0).
2026-10-18 20:54:11 +02:00
Niko1221 707eebd1dd Web search: spaced and backed-off requests, a session cache, Bing as the second keyless source, one plain message when every source refuses
The default DuckDuckGo search failed with "limiting searches from this computer" after two or three quick searches (reproduced on a
Linux box: two searches 1.2 s apart answered, the third got HTTP 202 and every DuckDuckGo page, html and lite, refused for many minutes
while it was polled). Now: browser-shaped requests (Referer, Origin, Sec-Fetch-*), one request at a time per source 2.5 to 4 s apart with
random jitter and at most 8 a minute, a 30 minute session cache per question, a per-source cooldown after a refusal that doubles from
1 to 10 minutes (Retry-After honoured) during which that source is not asked, no lite-page follow-up after a refusal, one retry for a
timeout or 5xx. When DuckDuckGo refuses, Bing's result page is used (links decoded from its redirects). When every source refuses the
tool says, in one plain sentence, that web search is limited, to try again in a minute or add a key (Exa, Perplexity or DeepSeek) in
Settings -> Tools -> Web search, and tells the model not to retry. Tests use a mocked rate-limited search and a real Bing page fixture.
2026-10-18 20:53:37 +02:00
Niko1221 f4cb8095c4 Locales: the new plain sentences (number range, Add skill from another computer, screenshot saved, folder window) 2026-10-18 20:52:35 +02:00
Niko1221 2325b2ee26 Desktop: pictures can be dropped on the chat in the Windows window (the shell's own drop handler is off, the page gets the drop); the drop box is animated 2026-10-18 20:52:05 +02:00
Niko1221 7572afc5bf Monitor: saving the screenshot says so (it is in the Downloads folder) or says it failed 2026-10-18 20:51:04 +02:00
Niko1221 4794a8067f Test fix for the message box cap 2026-10-18 20:49:39 +02:00
Niko1221 95ece0bcd2 Chat: the message box stops at 30% of the window height and scrolls inside, so Send and the toolbar stay in view in a small window 2026-10-18 20:49:29 +02:00
Niko1221 e0aa3b872a Windows folder window: the host waits for the window to report it is opening and otherwise asks for the path in words, so the button is released; Add skill from another computer says why it cannot 2026-10-18 20:48:17 +02:00
Niko1221 1237d9a5fa downloads and models: live progress, parts counted, choices kept, clear NVFP4 names, More versions list
- native state: the host's 'native' events are applied at once; the light poll (1 s while a job runs, none while hidden) is no longer
  held by a slow forced read; a read that never answers is dropped after 20 s
- install progress: every file of the model is counted (parts, picture reader beside the model folders, NVFP4 packs by their pinned sizes);
  the bar no longer sits at 99 % while the picture reader downloads; the screen names the part (part 2 of 4) and lists the parts
- a config is 'ready' only when every part of a multi-part GGUF is whole
- Models page keeps version, context, computer use, picture reading and expert choices across tabs; Read pictures is on for a version that
  is not downloaded yet; the running download is named
- version list: smaller rows, a Downloaded tag, NVFP4 / NVFP4-NVIDIA / NVFP4-QAD named by who made them with one line each
- first-run page: the four recommended versions, then a More versions list (the Models page list)
2026-10-18 20:47:43 +02:00
Niko1221 337c70d3c2 Response settings: a number outside its range (Max output tokens -1) is refused with a plain sentence in the dialog and by the host 2026-10-18 20:46:09 +02:00
Niko1221 e6aa6f317f Skills: a skill created with Skill enabled off is saved off (and keeps its trigger phrases) 2026-10-18 20:44:33 +02:00
Niko1221 969d7da7df Narrow window: a dialog opened from the menu closes the menu and takes the keyboard (focus also moves in when animation frames are held back) 2026-10-18 20:44:33 +02:00
Niko1221 c0abfdc358 /loop stops by itself: a count, a time or a stop condition in words is read when the loop is created and enforced by the scheduler
"/loop 1m say hi, stop after 3 answers" kept firing and the model answered "stopped" every minute until /loop stop. The count
("stop after 3 answers", "three replies", "5 times"), the time ("for the next 2 hours", "stop after 30 minutes") and a stop
condition ("stop when it passes", "until the file exists") are now parsed out of the line, left out of the message the model gets,
and enforced by a new scheduler (lib/loop-runner.ts): after the last allowed answer nothing is sent again, and the person is told in
one plain sentence why it ended. The scheduler also ends the loop when an answer says it is done or stopped (a bare "Stopped."
always; "Done ..." and similar when a limit was stated; DONE as the first word when a stop condition was given in words), and after
three failed answers in a row. A busy chat waits without using up the count. Tests drive the scheduler with a scripted model and a
fake clock. Strings in 8 languages.
2026-10-18 20:41:28 +02:00
Niko1221 323a11e721 Connect app: the third tab and strata connect give the server root (no /v1); the server also accepts /v1/v1/messages
Messages-API clients (coding CLIs, the SDKs) add /v1/messages to their base URL themselves. The Connect dialog showed the OpenAI-style
base that ends in /v1, so those apps posted /v1/v1/messages and got a 404. The third tab now shows the root address with a
one-line note (the base-URL variable for coding CLIs), the other tabs keep /v1. strata connect cuts a pasted /v1 back to the root for
every app and says so for coding CLIs, and strata launch and the monitor's Connect view list the Messages-API address. The server (serve
and the app's host, plus the shared listener) reads /v1/v1/messages and /v1/v1/messages/count_tokens as the real paths; nothing else
is aliased. Tested with the real Python SDK client against the mock-engine server.
2026-10-18 20:40:33 +02:00
Niko1221 f4078faf4f Answers: reasoning wrapper tags and a stray leading tag fragment never show in the answer or the Markdown export (UI side only) 2026-10-18 20:36:25 +02:00
Niko1221 e09eccc839 Files edited: only report what the turn itself wrote (a cut-short first look no longer makes the rest of the folder new files); never look in app, engine, component, AppData or temp folders 2026-10-18 20:33:55 +02:00
Niko1221 83f2a2c73a model list: Qwen3.6-35B-A3B has its three sizes (the first-run page showed no versions)
The catalog only walked setup.FAMILIES; Qwen3.6 lives in OWN_PATH_FAMILIES, so the app's Qwen3.6 page had an empty list.
strata_models now lists its sizes (download, memory, minimum engine 0.1.45, pictures) and strata_install plans them.
2026-10-18 20:33:08 +02:00
Niko1221 66ad879d18 Merge f-149-default (bac1a3e0e) into f-split-int: Linux defaults for the GGUF read in place (zero-copy page-cached experts, io prefetch with the usefulness/room gate), cgroup-aware RAM check, docs 2026-10-18 17:22:54 +02:00
Niko1221 76ceb365ca Merge f-split-launchfix (8f2f34ec4) into f-split-int: strata launch refuses at once when no usable engine exists, before any model download 2026-10-18 16:52:18 +02:00
Niko1221 8f2f34ec4e strata launch: settle the engine before any model download
strata update exits 0 when the feed has no engine for this PC (or only one for another card family), so launch went on and downloaded the model before failing with the generic engine line. ensure_engine now re-checks the engine after the update step and refuses at once with the update's own plain sentence plus one next step; setup.py in Control mode stops at the engine step for a build of another card family, an old driver, or when launch asks for an engine (STRATA_NEED_ENGINE=1), before the download step. The app without any engine keeps its model-first order.
2026-10-18 16:50:19 +02:00
Niko1221 bac1a3e0e9 Merge f-149-prefetch (docs) into f-149-default 2026-10-18 16:31:49 +02:00
Niko1221 24dc4a1dd6 docs: the 0.2.0 Linux file-tier defaults - the measured A/B against the 0.1.4x defaults 2026-10-18 16:31:49 +02:00
Niko1221 44fbe69c66 Merge f-149-prefetch (GGUF-only defaults) into f-149-default 2026-10-18 15:58:23 +02:00
Niko1221 c31367979d 149: the Linux file-tier defaults only for the GGUF read in place
An experts.bin pack (the NVFP4 packs) keeps STRATA_COLD_OVERLAP, STRATA_IO_PREFETCH and STRATA_IO_PF_GATE
off unless they are set: the defaults were measured on the GGUF read in place only. The startup line and
docs/DETAILS.md say so.
2026-10-18 15:58:22 +02:00
Niko1221 240304928b Merge f-149-prefetch (82c80ac4d) into f-149-default (from f-split-int fb7c62963) 2026-10-18 15:39:49 +02:00
Niko1221 82c80ac4d7 149: Linux defaults - zero-copy page-cached experts, io prefetch and its decode gate; cgroup RAM check counts the clean page cache
- STRATA_COLD_OVERLAP defaults to 4 on Linux (0 elsewhere), STRATA_IO_PREFETCH to 1 on Linux for the
  file tier (mapped GGUF in place or experts.bin), STRATA_IO_PF_GATE to 1 with io prefetch. Off switches:
  STRATA_COLD_OVERLAP=0, STRATA_IO_PREFETCH=0, STRATA_IO_PF_GATE=0. One startup line names them.
- cgroup_available_bytes: every clean file page of the group (memory.stat file, active + inactive; else
  inactive_file) is reclaimable, less min(half of it, limit / 64), as MemAvailable counts page cache on a
  bare host. A container whose own reads left active cache was judged short of RAM and its file tier
  switched to unbuffered reads (an 18 GiB RAM tier in a 32 GiB group: 1.2 GiB available instead of ~11).
  Unit tests with mock memory.stat / memory.max values.
- The per-request cold-reads line only with STRATA_COLD_OVERLAP or STRATA_IO_STATS set.
- docs/DETAILS.md: the 0.2.0 Linux file-tier defaults.
2026-10-18 15:39:41 +02:00
Niko1221 fb7c62963e Merge f-152-users (2e111557b) into f-split-int: CPU worker counts per physical core with a cap of 32, host socket, quota; warning above the recommendation; setup caps what it writes 2026-10-18 15:38:16 +02:00
Niko1221 8b1139d026 149: STRATA_IO_PF_GATE=1 - the decode's read-ahead only where the page cache has room and it pays (opt-in)
With STRATA_IO_PREFETCH=1, the decode's read-ahead (pread on the I/O threads, the look-ahead two layers
deep) and its pread demand reads run only while the gate is open; the prompt path is never gated.
Checked every 16 decode windows:
- room: (the tightest cgroup v2 limit - the group's non-file memory, else MemAvailable) / the experts'
  bytes; closes below STRATA_IO_PF_GATE_LO 0.55, opens above STRATA_IO_PF_GATE_HI 0.65;
- pays: closes when fewer than 128 MB were read ahead in 16 windows, or under STRATA_IO_PF_GATE_USEFUL
  (0.10) of those bytes were used; then a probe every 256 windows.
Closed, the look-ahead predicts one layer and warms with the madvise hint, as without io prefetch. The
decode thread (and the fetch threads it starts) is marked by FileExpertSource::DecodeScope. The serve
log's file-tier line says the room, the share used, the state and the flips. A request's wait for the
adaptive round in flight at its start is logged when >= 1 ms.

5 pairs (Q2_0 GGUF in place, 12 GB card emulated, cold SSD, cgroup RAM caps), mode 4 + IO_PREFETCH +
gate vs mode 4 alone: 32 GB 1.16 / 1.21 / 1.70 decode, 1.86 / 1.98 prompts; 16 GB (closed) 1.00 / 1.01 /
1.10, 1.54 / 1.78 prompts; an 18 GiB RAM tier 0.91 / 0.84 / 0.86 (its IO_PREFETCH start re-checks the
RAM inside the cgroup and switches the file tier to unbuffered - not the gate's doing).
Identity gates vs b829a3189 (fixed flags, Q2_0 and IQ3_S, default and mode 4 + IO_PREFETCH + gate with
--mmap-experts): 10/10 and exact_pp 4K / 20K IDENTICAL in all four.
2026-10-18 15:19:59 +02:00
Niko1221 7b06ea51bd Merge f-split-fb3 into f-split-int (locale keys re-added after the card-list strings) 2026-10-18 15:02:48 +02:00
Niko1221 2e111557b0 docs: how many CPU threads the engine starts and when a configured count is warned about 2026-10-18 14:59:31 +02:00
Niko1221 c47fbb3f22 Merge f-split-fb5 (cd1b3bd31) into f-split-int: the card list preselects the card a loaded model runs on; sidebar shows version and GPU 2026-10-18 14:57:18 +02:00
Niko1221 cd1b3bd31a Models: the card list preselects the card a loaded model runs on; the sidebar model card says the version and the card (Q2_0 · GPU 2) 2026-10-18 14:46:30 +02:00
Niko1221 0c7872ddc2 Merge f-split-pics3 (ea1e28436) into f-split-int: NVFP4-QAD reads pictures; setup accepts a prepared model without its checkpoint 2026-10-18 14:45:12 +02:00
Niko1221 ea1e28436f TUI test: the model without pictures is a stand-in now that Gyro reads them 2026-10-18 14:43:02 +02:00
Niko1221 261145baaf Merge f-split-fb4 (201d674e9) into f-split-int: cloudflared discovery, settings file, Gyro minimum engine, two models on two GPUs 2026-10-18 14:42:15 +02:00
Niko1221 201d674e94 Models: GPU numbering matches the other screens, friendly model names, fit figures less what the kept models use
Owner-approved for 0.2.0. The card list says (GPU n) with the same numbers as Monitor and the picker; a loaded
model is named from the catalog (Qwen3.8-Flash-Next Q2_0) in the picker, the Loaded list, the sidebar card and the
Monitor, the raw file id only as a fallback; with Run multiple models on, This computer and the card rows show what is
left after the models that stay loaded (a short muted line says so), in 8 languages. The picker rows no longer
print the raw id beside the name.
2026-10-18 14:41:11 +02:00
Niko1221 4eeccd9c00 Task handover: markdown bubble, clean chat title, long step lists capped 2026-10-18 14:39:45 +02:00
Niko1221 2dbd302038 setup: a prepared NVFP4-QAD model needs none of the checkpoint's safetensors; NVFP4-QAD reads pictures
ready_needed() drops the checkpoint's tensors/* once the model is prepared (prepared_done): the engine reads the prepared files only. have_model, ready_fetch, the already-downloaded search in other Strata folders and the disk need now use it, so a prepared model whose 95 GB of safetensors were deleted is ready instead of failing with 'not enough free disk' and a 102 GB download.
NVFP4-QAD (FP4-Local) reads pictures: vision True in setup.py and the MCP catalog; the config's vision section uses prepared/dense.gguf as the text model with the shared mmproj, like the GPTQ pack.
Tests for both; the launch test for a model without pictures now stands one in.
2026-10-18 14:32:19 +02:00
Niko1221 6149cec03e cloudflared: only Strata's own tunnel (same port or same token) blocks a start; other running connectors are listed in plain words with their PID
The coordinator's rule: the owner PC and a tester's system service run connectors for unrelated tunnels, and
quick tunnels must keep working. Running connectors are listed (pid; the command line is only matched,
never shown). A refusal only when one is clearly Strata's own: --url at Strata's port, or the same token.
Terminal (strata launch) follows the same rule. Checked with a real cloudflared on r9700.
2026-10-18 14:24:08 +02:00
Niko1221 5eea92bb09 Helper run no longer ends early when the helper reports while the chat model is still answering (the "interrupted before it finished" error) 2026-10-18 14:17:53 +02:00
Niko1221 6ad1ad69ab Settings file check: the app's own ui_* notes are not unknown keys (found by checking the configs the app wrote on r9700) 2026-10-18 14:14:18 +02:00
Niko1221 c1440388fb Two models on two GPUs in the app: say which card a loaded model uses, prefer a free card for the second, explain a failed start on a shared card, show an unload refusal beside its button, keep a failed instance out of the chat picker
Found running the AMD AppImage on r9700 (Q2_0 on card 1, IQ3_XXS on card 2, Run multiple models):
- the card list ignored loaded models (same text, same pre-selected card); a second model on a used card died
  after two slow engine starts with 'server_exited'. Now the row says 'Already used by <model>', a free card is
  pre-selected while Run multiple models is on, and a failed start there is one plain sentence (no useless retry).
- Unload refused because a chat still used the model (409) showed nothing near the button; it now shows the
  sentence and the next step in the Loaded models card.
- a model that failed to start was listed in the chat picker (disabled); it is hidden.
2026-10-18 14:13:35 +02:00
Niko1221 e2d03ac4fa scripted worker: multi script ends with its own text 2026-10-18 13:41:41 +02:00
Niko1221 b533ae5246 Skills used under an answer; Continue in a new chat on a task (owner-approved mockups), 8 languages, tests 2026-10-18 13:41:17 +02:00
Niko1221 7a70fcccbc Remove the generated Tauri schema files a cargo test run wrote 2026-10-18 13:34:44 +02:00
Niko1221 796937d485 agent README: model choice, reasoning, pictures 2026-10-18 13:34:34 +02:00
Niko1221 e932ad7eb7 Plain next step after a refused picture names the new switch 2026-10-18 13:31:33 +02:00
Niko1221 3084cd93de pics-e2e: on-by-default, OFF, silent server, agent step 2026-10-18 13:29:04 +02:00
Niko1221 14691f6e53 Control launcher: shell built-ins only (no dirname)
Inside the AppImage LD_LIBRARY_PATH holds the bundled libraries, and on Ubuntu 26.04 (uutils coreutils)
dirname fails there: 'libsystemd.so.0: version LIBSYSTEMD_254 not found'. The python/python3 launchers then
computed an empty folder and the app said Control stopped / no engine. Found with the AMD AppImage on r9700.
2026-10-18 13:24:36 +02:00
Niko1221 36e06fcd86 pictures-ui: temp path 2026-10-18 13:23:24 +02:00
Niko1221 65ac8f9106 Pictures: 8-language strings, plain-error rule narrowed, status unit test, real-browser paste/+/drop/off script 2026-10-18 13:22:51 +02:00
Niko1221 138075961f Merge f-148-intel-xmxdef (cfe65f8d1) into f-split-int2 2026-10-18 13:21:18 +02:00
Niko1221 bec4c838ab CPU pool plan: hybrid CPUs with more E-cores than P-cores default to the P-cores but the host's plus half of the E-cores (Core Ultra 7 265: 12-16 workers +12.6% decode over 19; 24 workers -44%) 2026-10-18 13:18:38 +02:00
Niko1221 af26053565 Pictures on by default everywhere: paste, shrink big pictures, one status for composer and Send, plain tool error, per-model off switch wording 2026-10-18 13:12:12 +02:00
Niko1221 cfe65f8d1a docs/sycl: PF_XMX=2 Alchemist default approved by the owner 2026-10-18 (comment and INTEL.md heading only) 2026-10-18 13:11:44 +02:00
Niko1221 ae40bd7516 Merge f-148-intel-xmxdef (12fd8cff6) into f-split-int2 2026-10-18 13:10:38 +02:00
Niko1221 79e94f37f0 OpenRouter reasoning from the model list (supported_parameters), effort sent as reasoning.effort; API pictures on by default 2026-10-18 13:07:03 +02:00
Niko1221 bca49fc5f2 Helper agents on a named model: forgiving model choice (id, provider/id, name, local), did-you-mean, plain list, guidance for every model 2026-10-18 13:05:00 +02:00
Niko1221 d631ed2da7 Gyro: the minimum engine is 0.1.45.1, not 0.1.45; the app says 'needs a newer engine' with the update step
Gyro's expert files are native_experts.txt v5 (Hadamard-folded). The engine reads v5 since the APR port
(f47b0385f); the released 0.1.45 stops at v4 (its binary and source have no v5 reader, every 0.1.45.1+ package
has). GYRO_ENGINE in setup.py and the catalog the app reads (tools/strata_mcp.py: engine in the fallback table,
engine_min on every size) now say 0.1.45.1. The host marks sizes the installed engine cannot read
(needsNewerEngine; an engine built from this checkout or with no readable version is never called too old),
refuses the download and the load with a plain sentence, and Models / first-run setup show the existing
'This model needs a newer engine' notice with Update engine and hold the buttons back.
2026-10-18 13:04:09 +02:00
Niko1221 036c8f6ec3 CPU pool plan: cap 32 (measured on the 32-core EPYC: 24 lost 2-5% decode, 32 and 48 equal, 63 oversubscribed lost 35%) 2026-10-18 13:03:10 +02:00
Niko1221 9319219623 Expert settings: Open settings file (the model's run config in the system text editor), and a plain check of it before every start
The per-model settings already live in one text file (strata-<model>.json: args, env, sampling and the
named keys). Expert settings gets an Open settings file button (Notepad, open -t, or the system's
plain-text program) with the note that changes apply on the next start. The file is checked when the
panel opens, when the window comes back into focus, and before a load: bad JSON, a wrong value, an
unknown key (the nearest real name is offered) are said in plain words with the line, in 8 languages.
A mistake holds the start back instead of an engine that dies on it; an unknown key only warns.
2026-10-18 12:59:05 +02:00
Niko1221 31f225e4fb cloudflared: find a system install (PATH and the usual places), check its version, say which one is used, and never start a second connector next to a running one
The app only looked in Strata's own connector folders, so a cloudflared installed by the system
(/usr/bin, brew, winget, a .deb) was reported as not installed. Now: own folder, component folder,
PATH, then /usr/bin, /usr/local/bin, ~/.local/bin, Homebrew, Program Files. The first copy that runs
--version and is 2022.1.0 or newer is used; the Connector step names it and its place. A cloudflared
already running (system service or started by the person) is said so plainly and not doubled.
Terminal (strata launch) gets the same search and check. Plain messages in 8 languages.
2026-10-18 12:52:12 +02:00
Niko1221 eeb05631fe tests: the Mac catalog test looked for '0 GB' in the whole computer line, which also holds the free-disk figure ('60 GB free'); it now checks the card part only 2026-10-18 12:51:17 +02:00
Niko1221 8a66da38a5 Merge f-split-pics2 (ac658f54a) into f-split-int2 2026-10-18 12:51:01 +02:00
Niko1221 a451dcd913 Merge f-148-intel (fca3c5958) into f-split-int2 2026-10-18 12:38:59 +02:00
Niko1221 ac658f54a6 Test: strata-vision-gguf-test wrapper and Gyro pictures flag 2026-10-18 12:25:37 +02:00
Niko1221 12fd8cff65 sycl: STRATA_PF_XMX=2 is the default on Arc Alchemist (opt-out STRATA_PF_XMX=0); Battlemage unchanged
A750 prompt prefill +30..39% at 4K (mode 2: 79.3 -> 110.4 tok/s, 5 of 5 rounds), +32..35% at 1-2K tokens; decode and cache slots unchanged.
Teacher-forced KL against the default 0.0002 (floor 0.0001), first-token KL 0.0017 (floor 0.0005), tasks inside the default's run-to-run spread.
Docs: INTEL.md has the numbers.
2026-10-18 12:22:23 +02:00
Niko1221 fca3c59586 sycl prefill: DMA an expert straight from the arena only when it is USM host memory (A750 prompt deadlock)
SYCL has no cudaHostRegister, so the expert arena is pageable memory even where the engine calls it pinned. A copy->memcpy of a 2.3 MB blob
from it goes through the Level Zero driver's staging path on the calling thread, which never returned on some prompts (A750, default
flags, a 2,744-token docs excerpt: the thread spun in sched_yield inside libze_intel_gpu with every engine idle, 6 of 6 runs; also seen
in a 5.3K-token needle prompt of the task suite). Only USM host allocations (the --stream-experts pinned mirror) are DMA'd directly now
(answer cached per 2 MiB bucket); every other expert goes through the stager's pinned buffers like an unpinned one.
STRATA_PF_DMA_PAGEABLE=1 restores the old direct copy.
Checks: the repro completes 3 of 3 (0 of 6 before); identical first-token logits and tokens, old vs new: A750 (frozen share, 1.4K, 2.5K and
4K prompts), B70 stock config and B70 without --stream-experts (there the new path is also faster: prefill 680-740 vs 77-456 tok/s); A750 4K
prefill 79.4 vs 79.3 tok/s, B70 stock 1246 vs 1240.
Adds sycl/tools/test_prompt_deadlock.sh with the 2,744-token prompt (passes on the fix, times out on the old build).
2026-10-18 12:19:14 +02:00
Niko1221 35af455aec Gyro catalog test: pictures are on 2026-10-18 12:16:56 +02:00
Niko1221 3fe84ec28d strata-vision reads only the vocabulary and header of the text model; Gyro reads pictures
llama.cpp refuses a GGUF whose tensor directory has a type it does not know, so the picture reader could not open a Gyro model (TQK6 / TQK7, ids 145 and 146) and Gyro was left without pictures. strata-vision now copies the key/value header into a small tensor-less GGUF and loads that vocab-only; the weights are never read or type-checked. Falls back to the file itself when the header holds a value type the copier does not handle.
Gyro-S and Gyro-M read pictures (setup.py, MCP catalog). Adds a C++ test (strata-vision-gguf-test) with a GGUF that has unknown tensor types.
2026-10-18 12:14:19 +02:00
Niko1221 83cde2ca30 Merge f-151-b70 (c765c5f87) into f-split-int2 2026-10-18 11:29:12 +02:00
Niko1221 5ece068f7e CPU expert pool: default worker count from physical cores, host socket, quota and a cap; warning for a configured count above it; setup caps what it writes 2026-10-18 11:20:20 +02:00
Niko1221 196a683d6d docs: the prompt chunk planner (file tier, Linux) - what it does, where it applies, the measured numbers, STRATA_PREFILL_HOST_PLAN=0 2026-10-18 11:14:16 +02:00
Niko1221 fd46e34d5c Merge f-150-planner (8253477bf) into f-split-int2 2026-10-18 11:13:37 +02:00
Niko1221 c765c5f87b sycl: int8 prompt experts default to auto on Battlemage (bmg-g21 / bmg-g31 by architecture id), FP16 elsewhere; STRATA_PREFILL_MMQ=0 off, =1 forced; launch failure or missing kernels fall back to the FP16 chain with one log line; load-time log line; mode-choice unit test; docs 2026-10-18 10:51:21 +02:00
Niko1221 983fa18ad3 docs: B70 follow-up measurements (GDN variant, XMX attention, attention layout, KqSplit, group size); static_assert on the group size 2026-10-18 10:31:41 +02:00
Niko1221 5bcc208a39 sycl: QSA prompt attention: probabilities in [cell][head] order with vector reads in the value phase (default, bit-identical, +0.9% prompt on the B70); STRATA_ATTN_RS=1 reduce-scatter scores measured -1.5% 2026-10-18 09:53:11 +02:00
Niko1221 0e58a652fb tests: iq_multi_parity under STRATA_KQ_SPLIT=1 also without V2K, with the fused gate/up SwiGLU, and with the sub-16 gate/up rows off
The NVIDIA worker's non-bitwise result came from a cherry-pick onto a tree without the dot-count guard in launch_gu (K-quant rows have 160 dots, the sub-16 rows assume 80); the merged tree has it and the Split path is bitwise on every route.
2026-10-18 09:50:49 +02:00
Niko1221 88c1502dc7 pt: s3158 without the word 'demora' (the no-demo catalog test matches 'demo' inside it) 2026-10-18 09:34:41 +02:00
Niko1221 97b680e13e gfx12: the WMMA prompt attention and the Q2_0 fused prompt experts are default-on on Linux only; Windows HIP builds keep them opt-in until tested there
core/gfx12_platform_defaults.hpp holds the per-platform rule (checked in arch_defaults_test); the engine log names it at the first prompt; docs/AMD_HIP.md.
2026-10-18 09:34:37 +02:00
Niko1221 a77d9fab54 Merge f-149-prefetch (2044864db) into f-split-int2 2026-10-18 09:34:20 +02:00
Niko1221 58f81f20e3 Merge f-151-amd (d9070189c) into f-split-int2
Conflicts with 148-amd's gfx12 WMMA default resolved: STRATA_HIP_WMMA unset = gfx12 on every call (148-amd), 0 = off, 1 = forced (also gfx11), auto = gfx12 with n_q >= STRATA_HIP_WMMA_MIN_Q; failed launch falls back to the FP32 kernel; the 96-slot ring follows the same state; the opt-in hint is gone.
2026-10-18 09:34:20 +02:00
Niko1221 8253477bf9 Prefill planner: request-sized chunks. A prompt that fits today's chunk keeps one chunk; a longer one reads in equal chunks of at most the plan's size (no stub tail). Buffers stay sized for today's chunk. 2026-10-18 09:20:24 +02:00
Niko1221 6aa8e25f49 docs: INTEL.md section for the fused int8 expert path (numbers, quality, crossover) 2026-10-18 09:19:38 +02:00
Niko1221 af3770c6b0 Merge f-148-amd (2053388dc, without the IQ2 fused default) into f-split-int2 2026-10-18 09:18:46 +02:00
Niko1221 40a2216b42 Merge f-148-nv into f-split-int2 2026-10-18 09:18:38 +02:00
Niko1221 2e1d62961a Merge f-split-pics into f-split-int2 2026-10-18 09:18:38 +02:00
Niko1221 b6a38b3288 Merge f-split-linuxui (83aa59fdc) into f-split-int2 2026-10-18 09:18:33 +02:00
Niko1221 d9070189c0 docs: --kv-grow on HIP with numbers, STRATA_PREFILL_HELP on two R9700 2026-10-18 08:32:28 +02:00
Niko1221 2725c23f74 NVIDIA sm_86 and sm_60: the measured bitwise decode switches on by default per architecture (nv_defaults; STRATA_SM86_PERF=0 / STRATA_SM60_PERF=0 turn them off) 2026-10-18 08:05:20 +02:00
Niko1221 3980a5ce70 HIP: virtual memory ranges (hipMemAddressReserve/Create/Map/SetAccess, 2 MiB chunks) so --kv-grow can run on ROCm 6+ (STRATA_HIP_VMM=0 turns it off) 2026-10-18 07:58:25 +02:00
Niko1221 2044864dba 149: STRATA_COLD_OVERLAP=4 (zero-copy without the overlap); no residency query for RAM-resident experts
The zero-copy pre-pass ran mincore for every CPU expert of a layer, the RAM copy's too: -2..-9% decode
with an 18 GiB RAM tier. It now skips the RAM-resident and override experts (b32: 0.98/0.97/0.91 ->
1.05/1.10/1.08 vs mode 0, 3 pairs). A file without a residency query (not Linux) never goes zero-copy.
2026-10-18 07:12:43 +02:00
Niko1221 9a01b6e3b8 sycl: int8 expert products: work-group kernels (activation tiles and codebook in local memory), STRATA_PREFILL_MMQ=auto from a measured per-GPU crossover cached per device/driver/format, per-group FP16 fallback, per-device state, clamped down-matrix reads, AOT guard for non-Xe2, docs; 4K +12.5%, 1K +20% with identical tokens 2026-10-18 07:09:40 +02:00
Niko1221 961d37cd55 gfx12: the 96-slot prompt ring follows the matrix-core attention's real state (STRATA_HIP_WMMA=1/auto on a card that has it, not failed) 2026-10-18 06:21:31 +02:00
Niko1221 2053388dc9 docs: AMD_HIP.md - the gfx12 matrix-core prompt attention and the Q2_0 fused prompt experts as defaults, with the 148-amd quality and speed numbers 2026-10-18 06:08:29 +02:00
Niko1221 ebdfaacd1c sycl: int8 DPAS expert products for IQ3_XXS and IQ2_S gate/up and Q2_0 down too (every layer type of the IQ3_S pack but one), thread tile NT=1 with MT 2/4/8, parity test over all six type pairs; STRATA_PREFILL_MMQ=1 prompts +10% (4K), +16% (2K), +20% (1K) with identical tokens 2026-10-18 05:54:34 +02:00
Niko1221 9c4516509f tests: iq_multi_parity runs with STRATA_KQ_SPLIT=1 2026-10-18 05:35:23 +02:00
Niko1221 4908ddaa49 docs: STRATA_HIP_WMMA=auto, the first-prompt warm-up, the gfx1201 hipBLASLt version note; timing: the first chunk's PLE wait on the ple line 2026-10-18 05:35:05 +02:00
Niko1221 d8471050bd Prefill planner: refit the miss share to two rigs (#2824 SATA 2x3060, P2003 NVMe cgroup sweep); read the drive class from the GGUF path 2026-10-18 05:09:14 +02:00
Niko1221 2856a8045e sycl: fused int8 DPAS expert products for the prompt path (ESIMD, Battlemage): IQ3_S gate/up + IQ4_NL down, int8 q8_1 activations, parity test, opt-in STRATA_PREFILL_MMQ=1; first correct version, not yet faster than dequantize + oneMKL 2026-10-18 05:08:50 +02:00
Niko1221 c85cc6dd19 149: io prefetch scheduling, RAM_ADAPT2, cold-read overlap and zero-copy GGUF experts (all opt-in)
- STRATA_IO_PF_SCHED=1 (with STRATA_IO_PREFETCH=1): confidence-scored read-ahead from the router
  lookahead, per-window speculative byte budget, deadlines (late jobs dropped), useful-share feedback;
  new per-request counters (read ahead MB, used before demand, late, not admitted).
- STRATA_RAM_ADAPT2=1 (with STRATA_RAM_ADAPT=n): RAM-tier promotions ranked by expected stall saved per
  byte (drive vs page cache), read on the source's I/O thread, installed at window boundaries; the
  prompt's routing seeds it with a limited weight (Prefill::on_routes).
- STRATA_COLD_OVERLAP: 1 reads a layer's staged blobs in the background while the pool computes the
  ready experts, each landed group computed as it lands; 2 measures R / C_r / C_c per layer; 3 adds
  zero-copy and 4 is zero-copy alone: an expert the page cache holds whole (GGUF in place, Linux mincore)
  is computed from the mapping through a per-job NativeFmt (ExpertJobMulti::fmt) instead of being
  assembled into a stage buffer. Same bytes, same rows, same bits.

Default (all unset) is the old code path: identity gates vs b829a3189 IDENTICAL (Q2_0 and IQ3_S,
fixed flags, 10/10 + exact_pp 4K/20K); mode 3 with --mmap-experts also 10/10 + 4K/20K IDENTICAL.
2026-10-18 05:06:29 +02:00
Niko1221 aef32e224b prefill timing: the hyper-connection write-back on its own line (it was charged to the phase before it, e.g. the wait for the first chunk's PLE rows); hip_prefill_gr_write_bench 2026-10-18 04:57:28 +02:00
Niko1221 83aa59fdc5 setup: say why the page waits while the bundled engine unpacks (8 languages) 2026-10-18 04:42:27 +02:00
Niko1221 f3be974359 prefill: warm the first BF16 and FP16 hipBLAS products on a thread at start (STRATA_PREFILL_WARM=0 skips), STRATA_COLD_TRACE diagnostic 2026-10-18 04:24:19 +02:00
Niko1221 9b75c6a127 desktop: host_command test (loopback bind forced, shutdown secret, plain refusal); Cargo.lock follows the app version 2026-10-18 04:17:21 +02:00
Niko1221 219ae436fa setup_intel: recommend --prefill 8192 for a Battlemage card in the borrowing (over 32K context) config, with the measured numbers and a test 2026-10-18 04:07:31 +02:00
Niko1221 79ac3164a0 gfx12: STRATA_HIP_WMMA=auto (prompt attention on matrix cores chosen per call, FP32 fallback on a failed launch); Split traits for Q4_K/Q5_K/Q8_0 (STRATA_KQ_SPLIT=1, bitwise, opt-in) with iq_multi_parity cases 2026-10-18 03:51:49 +02:00
Niko1221 2c6bfef271 gfx12: the native Q2_0 pack's gate/up take the fused int8 prompt experts by default (STRATA_PF_FUSED=0 keeps MMQ)
R9700 Q2_0: prompt 4K +16.7%, 20K +19.0% (5/5 pairs); teacher-forced KL 0.0004 over 6,022 positions, 0.0016 over 1,760 long-document ones.
2026-10-18 03:47:13 +02:00
Niko1221 99f6d47233 gfx12: the matrix-core prompt attention (STRATA_HIP_WMMA) is the default; STRATA_HIP_WMMA=0 turns it off
Teacher-forced KL against the old kernel on an R9700 (Q2_0, int8 KV, fixed cache so the floor is 0.0000): 0.0003 over 6,024
positions (p99 0.006, top-1 99.8%), 0.0014 over 1,760 positions of 8 long documents (top-1 98.9%); HumanEval, reasoning, agent,
needles, CJK and tool calls equal. gfx11 stays opt-in. The 96-slot prompt ring follows the kernel (qsa_hip_wmma_active()).
The start-up hint about the switch is gone.
2026-10-18 03:47:13 +02:00
Niko1221 577d644480 sycl: STRATA_FOLLOW / STRATA_FOLLOW_LOGPOS test path in the serve decode loop (teacher-forced KL), off unless set; measurement tooling 2026-10-18 03:40:58 +02:00
Niko1221 b1d4686f4a pics-e2e: optional API part, model match and context options 2026-10-18 03:25:01 +02:00
Niko1221 00d2f12a2d desktoplog: drop an unused constructor 2026-10-18 03:24:55 +02:00
Niko1221 e77f1dfc3f Prefill planner: price the host (RAM page cache, displaced experts, drive) when choosing chunk and ring (#2824)
The auto chunk scan only tested the GPU cache.  A lent slot's expert comes back from the RAM copy or the model file for the length of a
prompt, and every chunk streams the whole file set; past the page cache that set is re-read per chunk (28-45 GB per 29K prompt on
2x RTX 3060 / 32 GB / IQ2_XS at 8192-token chunks, ~1 GB at 4096).  prefill_plan.hpp is the pure model (file set, miss share fitted to
#2824, additive stream + compute + drive time); the serve scan and the generate path refine the VRAM pick with it, after the RAM copy
exists.  A set that fits the page cache keeps today's pick.  STRATA_PREFILL_HOST_PLAN=0 turns it off; --prefill N is untouched.
2026-10-18 03:21:24 +02:00
Niko1221 4306182741 tester READMEs: --save-logs, crash log location, the app version from Cargo.toml 2026-10-18 03:21:01 +02:00
Niko1221 1b003bd406 desktop shell: logs, crash log, --save-logs; the window blocks startViewTransition; one app version (Cargo.toml, 0.2.0-beta); banner while the bundled engine unpacks; diagnostics include the desktop logs 2026-10-18 03:19:07 +02:00
Niko1221 e4ed5d1a4e Pictures: untick means does not read, API refusal hint, e2e catalog check skips known false 2026-10-18 03:16:43 +02:00
Niko1221 52f50ba1f1 Pictures: plainer refusal wording, e2e reads answer tokens only 2026-10-18 03:15:43 +02:00
Niko1221 a3cf8a647b Pictures reach models: catalog marks picture-capable models, API models take image_url, import knows picture readers
The catalog no longer forces images:false in the app; a model reads pictures when it is picture-capable and the engine component carries strata-vision.
API models: a model reads pictures when the person ticks 'This model reads pictures' or the server says so (model list modalities, llama-server /props); the chat sends image_url to any OpenAI-style API that does, and says in plain words how to fix every case where it cannot.
Importing a model folder recognises the picture reader next to it and says plainly when a model type is not ours (use a Custom model API).
2026-10-18 03:14:46 +02:00
Niko1221 6c16529a84 ui: Quit dialog can be dismissed after Strata has stopped 2026-10-18 02:21:09 +02:00
Niko1221 01c8381ce6 make-tester-readme --verify: hash only the files the README names; a sentence that merely names a file is not a stale hash 2026-10-18 02:14:23 +02:00
Niko1221 efa888b143 Merge commit 'b1e0fdaa4' into f-split-int 2026-10-18 01:32:22 +02:00
Niko1221 c37a3c415b repack-appimage: write <out>.sha256 from the finished file 2026-10-18 01:30:50 +02:00
Niko1221 dd1a55f99a make-tester-readme: verify by checksum set 2026-10-18 01:30:04 +02:00
Niko1221 51c60366b8 make-tester-readme: plain replacement 2026-10-18 01:29:06 +02:00
Niko1221 1b27554b9c make-tester-readme: templates may have CRLF (archives of a Windows checkout) 2026-10-18 01:28:00 +02:00
Niko1221 c1a5b13fdf sycl: PcieAuto on SYCL, opt-in STRATA_PCIE_FRAC_DEFAULT=auto (single card, native pack): the PCIe share refined from the CPU pool's speed in the first decode windows 2026-10-18 01:27:12 +02:00
Niko1221 9d868f38e8 make-tester-readme: templates may have CRLF (archives of a Windows checkout) 2026-10-18 01:27:11 +02:00
Niko1221 3409d9aaec gfx12: the native Q2_0 pack's gate/up on the fused int8 prompt experts (STRATA_PF_FUSED=1; opt-in until KL-checked)
native_w11_kernel's gate/up path walked 256-value super-blocks only; a Q2_0 row is 64-value blocks, so gate/up of that
format now takes the same one-block-a-stage walk its down path has. R9700 Q2_0: prompt +16.7% (4K) / +19.0% (20K),
5/5 pairs; hip_prefill_fused_iq Q2_0 / Q2_0: fused vs MMQ rel RMS 1.6e-5.
2026-10-18 01:26:43 +02:00
Niko1221 20f92128c5 tester README test as unittest 2026-10-18 01:25:02 +02:00
Niko1221 b1e0fdaa4b sm_120: PF_FUSED default and per-token HC down blocks only on 80+ SM cards
The RTX 5070 check (48 SMs, IQ2_XS pack, experts partly in RAM): the
fused int8 prompt experts were 4.7% slower on 20K prompts (0/6 pairs) with
no decode gain, and one token a block row could not be told apart from
token pairs in the hyper-connection down projection (an earlier run leaned
to pairs).  So on a 12.0 card with fewer than 80 SMs the prompt experts
keep MMQ (STRATA_PF_FUSED=1 still takes the fused kernels) and the down
projection takes token pairs at every even launch (STRATA_GR_DOWN_TG=2);
12.0 cards with 80+ SMs (the RTX PRO 5000 measured) are unchanged.
2026-10-18 01:24:48 +02:00
Niko1221 a0469e9455 tester READMEs: templates with checksums computed from the files, make-tester-readme.sh (+ --verify), test 2026-10-18 01:24:05 +02:00
Niko1221 e1168345f1 desktop: tester README templates (hashes filled at build time), plain_path warning 2026-10-18 01:15:48 +02:00
Niko1221 5c5fdafb94 sycl: STRATA_FOLLOW / STRATA_FOLLOW_LOGPOS test path in the serve decode loop (teacher-forced KL), off unless set; measurement tooling 2026-10-18 01:09:08 +02:00
Niko1221 eaddf15b77 desktop: Mutex import only where used 2026-10-18 01:02:33 +02:00
Niko1221 a8fdceba5b ui: Quit Strata in the menu in browser mode (confirm, stopped screen), 8 languages; host: open the page without changing the mode 2026-10-18 01:01:32 +02:00
Niko1221 8be9e0976f desktop (Linux): the shell owns the host, the window is its own process; a crashed window moves Strata to the browser (--browser, STRATA_UI=browser, --quit, session file) 2026-10-18 00:58:54 +02:00
Niko1221 e01f766118 host: window mode vs browser mode (app-ui.mjs): Quit Strata, the shell's UI route, app updater unavailable in browser mode with a plain message 2026-10-18 00:55:37 +02:00
Niko1221 401169c02f ui: no route cross-fade (View Transitions) on WebKitGTK, which crashes its UI process on startViewTransition (null pointer in AcceleratedBackingStore::update) 2026-10-18 00:47:02 +02:00
Niko1221 b829a3189d docs: the 12.0 defaults with the final A/B (decode +14-16%, prompts +3-10%), PF_FUSED quality, the ple-io rule 2026-10-18 00:27:27 +02:00
Niko1221 b25b309703 setup: --ple-io ram on an SSD too when the RAM can spare the n-gram table (Linux)
A new prompt's first chunk waits for its n-gram rows (65,536 random SSD
reads at 4K tokens) unless the table is in RAM.  RTX PRO 5000 (Linux, NVMe,
251 GB, a fresh engine per prompt, 3 rounds, the same bits): prompt 4K
1298 -> 1059 ms (-18%), 20K 4644 -> 4223 (-9%), 98K 20447 -> 20027 (-2%),
decode +0.8%.  On an R9700 (slow prompt) the read hides behind layer 0 and
nothing changes; under 64-200 GB RAM caps (cgroup, page cache included)
ram mode equalled direct there and cost ~6 s of a cold start.

The rule: not low-RAM mode, Linux (not WSL), and RAM >= the model's own
need + the table + 16 GB (Q2_0: 93 GB, IQ3_S: 107 GB); a smaller PC keeps
the SSD reads as before.  Setup says what it did and how to undo it.
Tests: RotationalDisk gets three SSD cases; the golden configs keep the
Windows behaviour (ple_ram_platform_ok mocked).
2026-10-18 00:06:47 +02:00
Niko1221 959e6c34c4 prefill: fused int8 prompt experts by default on sm_120 (IQ2_S/IQ3_XXS/IQ3_S/IQ4_XS)
Owner decision, the AMD gfx12 rule: on a compute capability 12.0 card the
native packs' prompt experts run on the fused int8 kernels (moe_fused_iq)
for the gate/up formats measured there - the IQ3_S pack's IQ2_S, IQ3_XXS,
IQ3_S and IQ4_XS layers with IQ4_NL / Q2_0 down.  IQ2_XXS / IQ2_XS gate/up
keep MMQ until measured.  This rounds differently from MMQ:

RTX PRO 5000, IQ3_S: prompt 4K +7.0%, 20K +5.2%, 98K +3.4% (3/3 pairs);
teacher-forced KL against MMQ 0.0002 (5998 positions, noise floor 0.0000)
and 0.0010 (1760 positions after ~6K-token documents); first-token KL
against the FP16 prompt path 0.01181 vs MMQ's 0.01236 (48 prompts of
1.5K-24K tokens); HumanEval 156/164, reasoning 26/30, agent 6/6, needles
6/6, CJK 8/8, tool calls 4/5 - the same as MMQ.

STRATA_PF_FUSED=0 keeps MMQ, =1 takes every covered format (as before),
STRATA_SM120_PERF=0 drops the 12.0 default.  Other cards: unchanged.
2026-10-18 00:06:47 +02:00
Niko1221 75afe3f4a0 experts: gate/up, SwiGLU and q8_1 in one launch (bitwise; default on sm_120)
native_gu_swiglu_kernel: a 512-thread block owns 32 hidden columns of a
group; each 16-lane sub-warp computes gate row r and up row r together
(row_dot2_80_sub16: both rows' loads before the applies, each row's sums in
row_dot_80_sub16's order), the block forms h = silu(gate) * up in shared
memory, and warp k writes entry k's q8_1 block with q8_1_store.  The values
are the separate kernels' bit for bit (gate/up are the same floats, the
product is rounded on its own as in swiglu_q8_1_entries_kernel); it saves
the gate/up round trip and one launch per layer.

RTX PRO 5000 (IQ3_S, all experts resident): a layer's gate/up + SwiGLU
35.5 + 2.4 us -> 36.3 us; fixed 4-token rounds 11.92 -> 11.80 ms; default
flags, 6 interleaved pairs: decode +0.6% story, +0.9% code, +0.5% 8K
document, 6/6 pairs each, the same tokens in every case.

On by default for compute capability 12.0 (sm120_series.hpp); elsewhere
opt-in with STRATA_GU_SWIGLU=1; STRATA_GU_SWIGLU=0 turns it off.
2026-10-18 00:06:47 +02:00
Niko1221 fa726b4c9c launch: shorter model-list wording (too big, SSD N GB) so it fits 110 columns; docs and previews for the classic-console look and several cards 2026-10-17 23:53:48 +02:00
Niko1221 1d03976529 monitor: the classic Windows console switches to Cascadia Mono when installed, else a solid-block look with palette colours; several cards and CPUs in the Overview (totals, a Cards strip, sockets); whole card names in view 5 2026-10-17 23:39:05 +02:00
Niko1221 941041768b sm_120: the hyper-connection down projection with one token a block row (bitwise)
Measured on the RTX PRO 5000 against the token pairs of the previous commit,
fixed verify rounds, same text: 2 tokens 9.44 -> 9.24 ms, 3 tokens 10.81 ->
10.56, 4 tokens 11.94 -> 11.90, 5 tokens 14.04 -> 13.70.  Each (row, token)
is still summed by one warp in the same order.  STRATA_GR_DOWN_TG=2 / 0 picks
pairs / no split as before.
2026-10-17 23:05:40 +02:00
Niko1221 44997124ff mmvq_il: the sm_120 2-4 column table only on 80+ SM cards; docs
The shared table was measured on an RTX 5070 (48 SMs) together with an RTX
3060, so a small 12.0 card keeps it; the RTX PRO 5000 table (110 SMs) now
applies to 12.0 cards with 80 or more SMs (il_cc answers 1201 for them).
The 5-8 column table stays for every 12.0 card.  docs/DETAILS.md: the
measured decode gain of the sm_120 defaults, docs/MMVQ_IL_TABLE.md: the
new tables and switches.
2026-10-17 22:49:13 +02:00
Niko1221 3b4c379b68 Merge commit '7bdc1ffb7' into f-split-int 2026-10-17 22:48:56 +02:00
Niko1221 2aeb246cfc Merge commit 'ae31e3cd3' into f-split-int 2026-10-17 22:41:42 +02:00
Niko1221 ae31e3cd36 Docker: smoke-test.sh (the checks run on real cards), executable bits 2026-10-17 22:41:09 +02:00
Niko1221 8d838f770f DOCKER.md: sizes and what was tested 2026-10-17 22:38:47 +02:00
Niko1221 7bdc1ffb72 launch: the card or cards chosen this time go into every launch's own config (--gpu, the Cards step, or the fit rule), so an installed model never keeps its old card and nothing is reinstalled 2026-10-17 22:34:19 +02:00
Niko1221 a00ed935f1 Docker: GPU/GPUS take effect on a volume that already has the model, tuning wording, build id of the source build 2026-10-17 22:15:53 +02:00
Niko1221 404713f234 Dockerfile (source): no NCCL in the picture reader, so the package check passes 2026-10-17 21:59:11 +02:00
Niko1221 4895b1cfa0 Docker: keep only gguf-py and the ggml marker of the baked llama.cpp; root Dockerfile PATH fix 2026-10-17 21:55:59 +02:00
Niko1221 039d2d4609 desktop: cards-e2e.mjs, the Graphics cards choice end to end through the host API 2026-10-17 21:52:31 +02:00
Niko1221 fa6367a98f Merge branch 'f-split-int' into f-split-fit 2026-10-17 21:51:30 +02:00
Niko1221 33d37ddff6 app: the Graphics cards choice on the model page (the recommended option pre-selected, neutral style), passed on install and load 2026-10-17 21:51:30 +02:00
Niko1221 80e6d91a63 Merge commit 'a6759787c' into f-split-int 2026-10-17 21:49:11 +02:00
Niko1221 ba2c2d5fb7 sm_120: four bitwise decode switches on by default on a 12.0 card
Measured on an RTX PRO 5000 Blackwell 72 GB (IQ3_S, every expert in VRAM),
fixed 4-token verify rounds, story / code, the same text as before:
  STRATA_SIDE_MIXER=2   13.67 -> 13.15 ms a round
  STRATA_ATTN_DAILY=1   -> 13.31
  STRATA_GR_DOWN_TG=2   -> 13.34
  STRATA_GR_UP75=1      -> 13.44
  all four              13.70 -> 12.23 ms (2-token rounds 10.37 -> 9.59)
Each is bitwise equal to the kernels it replaces (their parity tests; the
identity gate).  include/strata/core/sm120_series.hpp holds the rule: on for
compute capability 12.0 unless STRATA_SM120_PERF=0; each switch's own
variable still wins in both directions on any card.  The down projection's
token pairs run at every even launch there (Turing keeps 4 and 6 only).
STRATA_ATTN_DAILY / STRATA_ATTN_MERGE_V2 and STRATA_SIDE_MIXER are now
decided per device (a layer split can mix cards).  Other cards: unchanged.
2026-10-17 21:45:38 +02:00
Niko1221 75fabbd941 mmvq_il: 5-8 column interleaved kernels and the sm_120 tables (bitwise)
A verify window of 5+ rows fell from the interleaved small-batch kernels (1-4
columns) back to native_mmvq_multi: on an RTX PRO 5000 Blackwell the dense
projections of a round jumped +2.0 ms from T=4 to T=5 (the main Q5_K head
521 -> 1011 us).  native_mmvq_il_kernel now also runs 5-8 columns from the
cp = 8 interleaved copy (1-2 rows a warp), every value bitwise native_mmvq's
(mmvq_il_parity, T 1-8, every rows choice).

Tables for compute capability 12.0, read off mmvq_il_parity --bench
--emit-table on the RTX PRO 5000 (3% rule): kIlRows_120 for 2-4 columns (it
takes the interleaved kernels in more cells than the shared table, e.g. the
head at 2 columns 507 -> 367 us) and kIlRows58_120 for 5-8 columns (the head
at 5 / 8 columns 899 -> 491 / 1178 -> 667 us).  Other cards keep their
tables and native_mmvq at 5+ columns.

STRATA_MMVQ_IL_WIDE=0 turns the 5-8 column path off; STRATA_MMVQ_IL_120=0
gives a 12.0 card the shared 2-4 column table again.
2026-10-17 21:45:28 +02:00
Niko1221 390336f68d Docker: pinned llama.cpp baked into the image (no GitHub download on first start), file consistency tests 2026-10-17 21:43:37 +02:00
Niko1221 7e490ef63c Docker: launch gets SIGINT back so docker stop ends a download in 1 s; INSTALL.md points to DOCKER.md 2026-10-17 21:26:58 +02:00
Niko1221 da34c60ba7 Merge commit '91cccc1ae' into f-split-int 2026-10-17 21:21:13 +02:00
Niko1221 3641dafe98 Docker: DOCKER.md, MODEL_PATH in compose, device check for AMD and Intel 2026-10-17 21:15:13 +02:00
Niko1221 c7f803d0db Merge commit 'e77e5550d' into f-split-int 2026-10-17 21:06:44 +02:00
Niko1221 5f1bd58e30 Docker: images built from the engine component packages (cuda13, cuda12, HIP gfx120X/gfx110X, SYCL), launch-based entrypoint, compose examples, from-source Dockerfile 2026-10-17 21:00:56 +02:00
Niko1221 f533c7faa6 docs: which cards a model uses; Cards step column widths and previews 2026-10-17 20:35:07 +02:00
Niko1221 31abd50a60 launch: the installed config must carry the chosen cards (a forced split re-runs setup), unified-memory chips count when they are the only device; setup tests follow the fit rule 2026-10-17 20:28:16 +02:00
Niko1221 91cccc1ae1 Merge branch 'f-split-testfix' into f-split-amdtester
# Conflicts:
#	apps/desktop/scripts/stage-harness.mjs
2026-10-17 20:27:24 +02:00
Niko1221 e77e5550df stage-harness: fix a broken string literal in the import check; node --check / bash -n / py compile of every build script in the host tests 2026-10-17 20:26:24 +02:00
Niko1221 15339326e9 model list: gpu_options per model from the shared fit rule (the app reads the choice from here) 2026-10-17 20:16:48 +02:00
Niko1221 1d8d830afa setup: the multi-GPU default follows the fit rule (largest card + RAM first, the split recommended only when the model does not fit one card), goldens re-recorded, setup tests 2026-10-17 20:16:02 +02:00
Niko1221 c198c25e57 launch: the Cards step uses the shared gpu_fit rule (largest card + RAM recommended, the layer split offered with plain numbers, an integrated chip ignored) 2026-10-17 20:13:33 +02:00
Niko1221 ac38793caa gpu_fit: one rule for which graphics cards a model uses (largest card + RAM first, the split offered when it does not fit, an iGPU ignored), with tests 2026-10-17 20:10:22 +02:00
Niko1221 a6759787c3 Merge f-146-int (b7505b72a, 0.1.46 engine: Volta perf PRs + QSA_WARP sm<75 fix) into f-split-146
Conflict in src/prefill/prefill.cpp (step 2b): the split rotates mixed into Xr (APR, had_gu)
before 0.1.45's quantize_scatter (NVFP4 activations keep the gather); 0.1.46 adds the pf_wmma
FP16 gather branch. Both kept: wmma_l gathers FP16 rows (and rotates Xs / the swiglu output Hh
in fp16 when the layer is Hadamard-folded), otherwise the split's rotate then quantize_scatter
or the NVFP4 gather.
2026-10-17 20:09:14 +02:00
Niko1221 b2761df626 stage-harness.mjs: the missing-imports message had raw line breaks inside a string (SyntaxError, no Harness could be staged on any OS) 2026-10-17 19:53:48 +02:00
Niko1221 bfed34873a Merge branch 'f-split-testfix' into f-split-amdtester 2026-10-17 19:53:36 +02:00
Niko1221 b7505b72a4 Merge branch 'f-146-qsawarp' into f-146-int 2026-10-17 19:23:39 +02:00
Niko1221 1b856af242 Installer: Uninstall only uninstalls; the uninstaller offers Also remove my Strata data (off by default) and lists exactly what it deletes 2026-10-17 19:17:44 +02:00
Niko1221 051d0fc9c4 Models tab: Computer use buttons level with the context dropdown, clear of the separator 2026-10-17 19:15:08 +02:00
Niko1221 447b99ba38 Folder chooser fallback in the app: plain message and a field for the path 2026-10-17 19:12:09 +02:00
Niko1221 d85bb21c6b Linux: programs that are not Strata's own get the AppImage's original environment; folder chooser tries zenity, kdialog, portal and never fails silently 2026-10-17 19:11:09 +02:00
Niko1221 d7347cdde6 Start failures: plain, correct messages from the engine's exit code (missing DLL, bad DLL, crash, blocked, file missing); low memory only with evidence; no memory retry for non-memory failures 2026-10-17 19:07:42 +02:00
Niko1221 ad3b340d30 App never adopts an older CLI install: own settings file for Control and the installer, explicit install folder, tests 2026-10-17 19:02:17 +02:00
Niko1221 37f7f03cd2 Engine packages: bundle every imported non-system library, refuse a package a clean PC could not start, PE/ELF/Mach-O import scanner and tests 2026-10-17 18:59:45 +02:00
Niko1221 cbb249d3dd Windows HIP package: ship msvcp140_atomic_wait.dll (strata.exe imports it) and the other VC++ runtime DLLs next to the exes as well as in rocm/bin 2026-10-17 18:54:32 +02:00
Niko1221 f95fb3e074 Fix agent chat on Linux and Mac: keep the Harness's own dsh-* packages when pruning other-OS binaries, verify imports at staging, plain start-failure message 2026-10-17 18:54:11 +02:00
Niko1221 6aa502d938 strata update: a feed whose only HIP engine is for another AMD card family says so in one plain line (This build of Strata is for AMD RX 7000 cards ... Ask for the RX 9000 build) 2026-10-17 18:25:27 +02:00
Niko1221 67bd2a1609 hip_archs_present: an explicit kfd folder is always read (the Windows adapter lookup only replaces the default Linux one) 2026-10-17 18:21:43 +02:00
Niko1221 fe6fa9a274 Merge commit 'db5be2aa554cb248beb70c9595b9719254373772' into f-split-int 2026-10-17 18:01:54 +02:00
Niko1221 db5be2aa55 vision-e2e: load by the installed model name 2026-10-17 14:50:28 +02:00
Niko1221 50768496f5 vision-e2e: find the installed model by id 2026-10-17 14:48:34 +02:00
Niko1221 ab5adeb79f docs: pictures in strata launch 2026-10-17 14:41:45 +02:00
Niko1221 ac5f0d19a3 desktop: vision-e2e.mjs, the picture-reading end to end test through the installed app layout 2026-10-17 14:41:19 +02:00
Niko1221 9a22b78887 setup: the Mac control path says plainly that the Mac engine has no picture reader yet 2026-10-17 14:40:09 +02:00
Niko1221 7c5e846121 Merge branch 'f-split-int' into f-split-vision 2026-10-17 14:40:04 +02:00
Niko1221 417cbab4ff packaging: the engine package carries strata-vision when the build has it 2026-10-17 14:40:04 +02:00
Niko1221 788e0b731b Merge branch 'f-split-agent8' into f-split-vision 2026-10-17 14:38:52 +02:00
Niko1221 debe8ddabb launch: a missing picture reader file is fetched again instead of starting an engine that cannot read pictures 2026-10-17 14:31:03 +02:00
Niko1221 7ecfe4a6a6 A tester build for another AMD card family: one plain line (This build is for AMD RX 9000 cards ... Ask for the RX 7000 build) in the engine status and the start error 2026-10-17 14:16:44 +02:00
Niko1221 4db866ae00 Engine packages: the folders named in BUILD.json lib_dirs go on the library path with the package folder (the Windows HIP package keeps the ROCm DLLs in rocm/bin), for serve, the binary smoke test and tuning 2026-10-17 14:16:02 +02:00
Niko1221 8c676a7ccd Windows AMD: the engine is picked from the display adapters when no GPU tool answers (an AMD card, not an integrated Radeon), the app lists AMD cards on Windows and no longer refuses the hip backend there 2026-10-17 14:10:08 +02:00
Niko1221 8b3a96779e Merge f-145-pp-volta into f-146-volta (0.1.46): the community Volta/Pascal perf-PR group (35 commits)
Merge resolution as in f-145-perfprs-withvolta: iq_kernels.cu (IQ3_XXS / IQ3_S staged sign-mask tables over the EXPDEC grid staging), prefill/kernels.cu and prefill/prefill.cpp keep both sides; CMakeLists keeps both test blocks.
2026-10-17 14:00:21 +02:00
Niko1221 c61364fc89 qsa prompt attention: STRATA_QSA_WARP=1 only forces the Turing kernels on cards that have the Turing MMA (sm_75+)
The switch made every card answer as sm_75, so a Tesla P100 (sm_60) launched the m16n8k8 prompt-attention kernels, whose sm_60 code is a __trap(): the engine died on the first long prompt with "prefill copy_i32: unspecified launch failure". A card below sm_75 now ignores the switch and runs the kernel it runs without it (the same tokens).
2026-10-17 13:49:20 +02:00
Niko1221 e29112390f launch: turning pictures on for a model that is already installed says it downloads only the picture reader, not the whole model 2026-10-17 13:48:17 +02:00
Niko1221 8d65058c9f launch: Pictures on goes through setup's Control mode (setup keeps the choice when the engine component carries strata-vision); the screen warns only when the installed engine lacks the reader 2026-10-17 13:46:42 +02:00
Niko1221 e819ba0bd0 launch: the Folder screen says that model files already stored are moved into a new folder 2026-10-17 13:10:14 +02:00
Niko1221 0b055d8316 Merge commit 'ee0f32805' into f-split-int 2026-10-17 12:04:03 +02:00
Niko1221 ee0f328053 test: the bundle keeps one real python and launchers for the other names 2026-10-17 11:58:14 +02:00
Niko1221 d4abe4b745 strata launch on a Mac: a Terminal that carries strata_control/mac_fit.py installs the model through setup.py's Control mode instead of adopting prepared files 2026-10-17 11:55:03 +02:00
Niko1221 78fc321cd3 launch: the Pictures screen says plainly that this Terminal cannot set up the picture reader yet 2026-10-17 11:52:23 +02:00
Niko1221 254ee2b9cd Control: the NVIDIA driver version also from /proc/driver/nvidia/version (Fedora has no nvidia-smi without its cuda package); an integrated AMD GPU beside a usable NVIDIA card is not offered 2026-10-17 11:27:51 +02:00
Niko1221 a93264877c metal: the opt-in HIP/Turing entry points (fused_gr_persist_init, fused_gr_read_persist, f16_range_read, gemm_f16_range_read) answer 'nothing to do' instead of stopping the engine; e2e script waits for the bundled engine 2026-10-17 11:26:20 +02:00
Niko1221 e4b885e9fa repack-appimage.sh is executable 2026-10-17 11:19:53 +02:00
Niko1221 f2fe9a9097 metal: stubs for the 0.1.45 entry points Metal does not implement yet (stubs_0145.cpp) and the Gemm::bf16 / bf16_converts_x signatures, so the engine links from the 0.1.45 tree 2026-10-17 11:18:31 +02:00
Niko1221 d2a38c9e5c Linux tester AppImage: repack-appimage.sh puts the staged engine/Node/Python/Harness back byte for byte (linuxdeploy rewrites their RPATH, which broke the bundled engine's manifest check); control status says the bundled engine is being set up; WebKit DMA-BUF renderer off on Linux; e2e script takes --backend and finds the engine by its components path 2026-10-17 11:13:50 +02:00
Niko1221 47e2343b1b launch: Pictures on means the CPU picture reader on an AMD card, and a plain warning when pictures stay off 2026-10-17 11:10:47 +02:00
Niko1221 eeeeddd3f6 metal: the CUDA-name compat layer declares the seven calls the 0.1.45 sources added (compute capability, stream priority, cudaMemcpyPeer, DisablePeerAccess), so the Metal engine builds from the 0.1.45 tree again 2026-10-17 10:54:20 +02:00
Niko1221 63ac0838bc docs/MAC.md: the app and strata launch on a Mac 2026-10-17 10:47:04 +02:00
Niko1221 ccb33b7c7c Mac install plan: no card, KV or RAM-mode options; the context follows unified memory; a Mac too small for the model is refused in plain words 2026-10-17 10:45:17 +02:00
Niko1221 78914f55be Mac app: the model list, fit words and recommendation judge a Mac by unified memory (strata_control/mac_fit.py, one definition shared with metal/setup_mac.py); setup.py in Control mode installs a Mac model with no source build and no setup_mac.py and writes component:strata-engine with the Metal flags 2026-10-17 10:42:27 +02:00
Niko1221 6c4dbe3362 launch: Pascal/Volta cards are admitted when they are the only NVIDIA cards, as setup does (V100 showed 'no usable graphics card' and no Cards step) 2026-10-17 10:41:48 +02:00
Niko1221 8df9e0afd0 Linux app: one real python in the bundle (the other names become launchers), GStreamer bundled in the AppImage, a driver too old for every engine gets one plain line naming the version to install 2026-10-17 10:33:22 +02:00
Niko1221 395ed1ac54 bundled engine: CUDA 13 and CUDA 12 builds of one version install side by side, only the ones the NVIDIA driver can run; fix the duplicate 'ready' parameter that made native-control.mjs a syntax error 2026-10-17 10:30:37 +02:00
Niko1221 13ded2d662 Merge branch 'f-split-tester' into f-split-macapp 2026-10-17 10:22:38 +02:00
Niko1221 60c4d8de1e Merge branch 'f-split-apps' into f-split-macapp 2026-10-17 10:20:35 +02:00
Niko1221 e288308b46 Installer can carry the engine: the host installs a bundled engine package (manifest hashes checked, never over a newer one, no network) before the first model list 2026-10-17 10:14:19 +02:00
Niko1221 3618b48030 stage-harness: drop musl and other-CPU prebuilds on Linux (linuxdeploy stops on koffi's musl build) 2026-10-17 09:47:07 +02:00
Niko1221 59cee65da3 serve tests: the refused-prompt test runs in strict mode (fit_max_tokens off) so a huge max_tokens is refused, not shortened 2026-10-17 07:00:49 +02:00
Niko1221 9a58a4b73a Merge f-145-int (477ac13cb, the 0.1.45 release tree) into f-split-145 2026-10-17 06:33:36 +02:00
Niko1221 f202d08c5f Merge commit '872bcd197' into f-split-int 2026-10-17 06:16:06 +02:00
Niko1221 872bcd1974 launch: see an Intel Arc card (from the kernel's device list) and recommend a context that fits its memory 2026-10-17 05:52:15 +02:00
Niko1221 6ad5596b9d tests: the Intel Arc flags follow the kernel driver in the launch settings 2026-10-17 05:12:09 +02:00
Niko1221 982f43dee5 Regenerate launch previews after the model-list change 2026-10-17 05:11:26 +02:00
Niko1221 87fbbdaaa9 Merge branch 'f-split-agent8' into f-split-int 2026-10-17 05:06:46 +02:00
Niko1221 e3bf0ad04d Time-skip: final 'Both versions are ready' line, no stale warning after B finished 2026-10-17 05:02:30 +02:00
Niko1221 b8c1602715 launch: Intel Arc recipe for prepared files (i915: --ple-io ram; xe: --stream-experts; VRAM reserve) 2026-10-17 04:35:15 +02:00
Niko1221 08297223e4 launch: on a Mac, use prepared model files with the Metal defaults (the Terminal has no Mac model installer) 2026-10-17 04:34:33 +02:00
Niko1221 64fff9e149 launch: model list shows the app's RAM+VRAM figure and the SSD part; strata monitor gets the Connect view (7); Mac env names 2026-10-17 04:31:34 +02:00
Niko1221 b799ef5f6d launch tests: top-folder install case 2026-10-17 04:06:23 +02:00
Niko1221 8cc02778e4 launch: firewall line, overview GPU name and legacy width cap, install.sh accepts an archive with a top folder, tool-call check budget, plain wording for adopted files 2026-10-17 04:05:47 +02:00
Niko1221 87efc9ace5 launch tests: macOS fixes (platform names, python 3.9 tar, no setsid/timeout/cp -l), feed platform macos-arm64, p2006 engine path 2026-10-17 04:01:23 +02:00
Niko1221 c1533d6d5f update: an engine without --version that lists its options passes the test start (with a plain note) 2026-10-17 04:00:17 +02:00
Niko1221 a693660a15 launch: fix the download question from the start screen 2026-10-17 03:41:25 +02:00
Niko1221 436fabc466 launch: ask before downloading a model that is not here; tuning table labels 2026-10-17 03:39:30 +02:00
Niko1221 7c2f878051 launch: a Mac's setup gets only the options it knows 2026-10-17 03:36:59 +02:00
Niko1221 a2bab5c8c2 launch: no choices and no terminal starts nothing 2026-10-17 03:36:37 +02:00
Niko1221 dc1aa5030a launch: docs (TERMINAL_LAUNCH.md), installer tests, plain GGUF issue words 2026-10-17 03:36:03 +02:00
Niko1221 463d6780a9 launch: write the run settings from prepared files when the installer cannot set up the card (Intel Arc) 2026-10-17 03:21:39 +02:00
Niko1221 9330989efa launch: say why the engine could not be installed 2026-10-17 03:16:46 +02:00
Niko1221 69f9cc7b9e launch: environment settings also apply to the engine's test start 2026-10-17 03:12:10 +02:00
Niko1221 b3ff0c2ebf launch: environment names keep their case in --set (IGC_..., ZES_...) 2026-10-17 03:07:25 +02:00
Niko1221 4fbd0b3fcb connect check: responses answer text excludes reasoning items 2026-10-17 03:02:20 +02:00
Niko1221 d247802094 launch: install.ps1 child scope and stderr note, tuning result line, Windows watch hint 2026-10-17 02:57:51 +02:00
Niko1221 1fe683e951 Picture input for a server on this network only when its own model list says it reads pictures (a server without a vision encoder refused whole requests) 2026-10-17 02:53:59 +02:00
Niko1221 eaa3a58172 launch: monitor tests, quit hint, check result note 2026-10-17 02:50:03 +02:00
Niko1221 7ce4b6df37 Time-skip step limit 15 minutes 2026-10-17 02:41:28 +02:00
Niko1221 f548cfd843 Time-skip: a step that runs longer than 20 minutes is stopped and left as it is with a plain note; one shared step runner for A and B 2026-10-17 02:41:20 +02:00
Niko1221 6a29939806 launch: connect --check, tester stage scripts (phaseA-tests/split/launch-tests are outside the repo) 2026-10-17 02:33:31 +02:00
Niko1221 ec06637d48 launch: installer placeholder fix, quieter plain run, flags next to --preset do not rewrite it 2026-10-17 02:26:28 +02:00
Niko1221 4e6a1b46be launch: text previews of every screen, advanced table widths 2026-10-17 02:15:12 +02:00
Niko1221 efa38a99ca monitor: the approved Overview as view 1 (state bar, 8 tiles with sparklines, bars, recent requests) from /metrics; b opens the browser 2026-10-17 02:14:32 +02:00
Niko1221 4e788fbf3e connect: the 8 tested harness configs (limits in plain words, context-aware), Connect view shows them 2026-10-17 02:11:28 +02:00
Niko1221 da4a3a4765 Merge commit 'e0a932a32' into f-split-launch 2026-10-17 02:07:56 +02:00
Niko1221 4c9423e194 launch: one-line installers (install.sh, install.ps1), local test feed builder, package_terminal installer step 2026-10-17 02:07:47 +02:00
Niko1221 4fac421d14 Time-skip sample copy only when the host cannot be reached at all; packaged test follows the Read pictures install 2026-10-17 01:55:37 +02:00
Niko1221 7b1405e104 Time-skip: view-in-browser steps never hang or fail a version (no-window rule, skip with a plain note); steps carry the unattended note 2026-10-17 01:48:50 +02:00
Niko1221 97e4dcaa1f Merge branch 'f-split-ui6' into f-split-int 2026-10-17 01:47:30 +02:00
Niko1221 c473d97109 Merge branch 'f-split-agent8' into f-split-int 2026-10-17 01:47:28 +02:00
Niko1221 4a786e03d6 Time-skip: a step that does not finish is tried once more in a fresh session; B says why it stopped 2026-10-17 01:42:17 +02:00
Niko1221 e0a932a32d Merge branch 'f-split-harness' into f-split-int 2026-10-17 01:06:34 +02:00
Niko1221 f31b6efd68 serve: shorten max_tokens to the room left instead of answering 400 (all three dialects)
Claude Code asks for max_tokens 32000 on every request and the Harness for 16384; on a 32K context the
prompt fits but prompt + max_tokens does not, and the server answered 400 "request exceeds the context
window ... requests are never truncated" before the first token.

Service.prepare() (the one place chat/completions, messages and responses all pass through) now clamps
max_tokens to ctx - CTX_SLACK - prompt when the prompt fits, logs one line, and the answer_room thinking
budget (3/4 of max_tokens) is taken from the clamped value. A prompt that alone does not fit is still the
plain 400 ("no room to answer"). --no-fit-max-tokens / "fit_max_tokens": false brings back the strict
llama.cpp 400; --fit-max-tokens stays accepted.

Tests: serve/test_server.py MaxTokens (shortened by default in the three dialects, exact fit and one over,
prompt-too-long 400, thinking budget follows, strict mode), serve/test_context_overflow.py (default
service), serve/test_monitor.py (error record uses the strict mode).
2026-10-17 00:55:18 +02:00
Niko1221 f11768e873 Time-skip: each step runs in its own agent session (the folder carries the work), so a small window never fills up across a long plan 2026-10-17 00:41:02 +02:00
Niko1221 af34f0a4b7 Reply room 30 percent of the window and an eighth headroom: a 32K window no longer compacts every few steps 2026-10-17 00:04:36 +02:00
Niko1221 64034f1674 launch: terminal archive carries the installer files (shared with the app's Control staging), legacy-console glyph and box fallback, install from a bundle runs in control mode 2026-10-16 23:43:01 +02:00
Niko1221 8251089c1f launch: pilot tests for every screen, run screen state fix, no auto focus 2026-10-16 23:40:55 +02:00
Niko1221 af4a7e5638 Merge branch 'f-split-fp4' into f-split-int 2026-10-16 23:38:52 +02:00
Niko1221 5fbd539628 launch: TUI screens, runner events, monitor Connect view, harness check, tests for flags and rules 2026-10-16 23:32:37 +02:00
Niko1221 e61e90e555 launch: settings, catalog (setup.py tables via strata_mcp, fitOn port), presets store, access, runner, flags, connect, harness drafts 2026-10-16 23:25:34 +02:00
Niko1221 17ba2d71ad A model server on this network keeps half its window for the reply (long Time-skip steps overflowed the window); Time-skip says why A failed 2026-10-16 23:18:34 +02:00
Niko1221 76707b230b Round 8: neutral Open result button, pictures uploaded to /api/chat/upload and sent as imageIds, picture-reading files note on the Models tab 2026-10-16 23:00:11 +02:00
Niko1221 0a6eced5cc Merge branch 'f-split-agent8' into f-split-ui6 2026-10-16 22:50:27 +02:00
Niko1221 490fda2657 Round 8: project folder chip above the message box, Time-skip as two agents (plan, lever with its why, A and B progress, Keep A / Keep B), working view reachable while a run has no files yet 2026-10-16 22:50:05 +02:00
Niko1221 30a367e5f4 Image limits in the validation test follow the new 4 pictures / 6 MB 2026-10-16 22:49:22 +02:00
Niko1221 c3e9480592 catalog: one plain line each for NVFP4 (best quality of the NVFP4 versions) and NVFP4-QAD (NVIDIA's FP4, trained for FP4, slightly faster on short prompts) 2026-10-16 22:44:58 +02:00
Niko1221 477ac13cbe verify: the side-projection q8_1 interleave copies need two or more columns (STRATA_SIDE_MIXER=2 on a 1-column window threw on Q2_0 packs: the queries' branch and the GDN gate branch) 2026-10-16 22:39:03 +02:00
Niko1221 c8265fce0e tests: setup's gfx1103 / gfx1150 switch sets do not copy STRATA_TOPK_REG_MIN_BLOCKS (the gfx1151 line from #2016, measured on gfx1151 only) 2026-10-16 22:39:03 +02:00
Niko1221 e6ad6c81d7 fix(HIP compat): cudaMemcpyPeer and cudaDeviceDisablePeerAccess for the #2666 peer-copy probe (peer_experts.cpp did not compile on HIP in f-145-int) 2026-10-16 22:39:02 +02:00
Niko1221 f260925adc fix(sm75 merge): stray brace after hf() in prefill/kernels.cu (#2093 #1821 resolution) 2026-10-16 22:39:02 +02:00
Niko1221 d615708a7b Time-skip v2: plan, lever, agent A to the end, agent B forked at the lever with an improvement plan, A vs B, keep one 2026-10-16 22:37:12 +02:00
Niko1221 4d7fede4ed Merge branch 'f-split-agent8' into f-split-ui6 2026-10-16 22:36:21 +02:00
Niko1221 9e2d698c9e Round 8: Models navbar family dropdown, deliverable cards with Open menu, Edited summary, grouped activity lines, text in order, drop zone, image generation hidden 2026-10-16 22:35:54 +02:00
Niko1221 7af6b6c152 Vision end to end: vision files are downloaded when asked, pictures through the agent and the plain path, uploads by multipart/data address/raw bytes, dropped files named in the next message 2026-10-16 22:32:11 +02:00
Niko1221 917d2fc345 Deleting a model: a file that is already gone counts as removed; the host verifies the files are really gone 2026-10-16 22:26:44 +02:00
Niko1221 0ac98f80ee Merge branch 'f-split-agent8' into f-split-ui6 2026-10-16 22:26:01 +02:00
Niko1221 8509870849 Round 8: compact segmented Computer use, version chevron and panel gap, download details only when useful, immediate download progress, Models screens no longer follow every monitor tick 2026-10-16 22:26:01 +02:00
Niko1221 aa936f59f0 Deliverable events, grouped activity lines, file preview/raw/attach routes 2026-10-16 22:25:51 +02:00
Niko1221 ef3902477e Per-chat project folder (agent cwd and sandbox), opening files by path in the browser, app or folder 2026-10-16 22:23:38 +02:00
Niko1221 ddfed500a9 A turn always ends: watchdog for a silent model or a tool that never returns; launches return at once 2026-10-16 22:21:57 +02:00
Niko1221 ade3e6adb5 nvfp4_local_pack ple: accept a folder with only the shards that hold the table 2026-10-16 22:02:03 +02:00
Niko1221 4736639a7e merge 5e7a965ac into f-split-int 2026-10-16 21:57:35 +02:00
Niko1221 dbbe79aae8 merge bdb04dd8d into f-split-int 2026-10-16 21:57:34 +02:00
Niko1221 bdb04dd8d6 batch-mtp on automatically for one big card (24 GB+, several slots, MTP on); off on small cards and layer splits; Expert mode MTP per request overrides 2026-10-16 21:57:18 +02:00
Niko1221 48b87d42e1 docs: NVIDIA FP4-Local in NVFP4.md 2026-10-16 21:10:27 +02:00
Niko1221 e7f7dad3e0 FP4-Local engine floor 0.1.45 (the split release's engine has the NVFP4 n-gram table) 2026-10-16 21:04:27 +02:00
Niko1221 0da73d9159 FP4-Local prepare: separate step marks (the pack's own files were overwritten), scale tensors in the checkpoint-direct pack; docs section 2026-10-16 20:06:19 +02:00
Niko1221 5e7a965ac1 Queued-message test: ask Stop again when it answers still stopping 2026-10-16 20:06:04 +02:00
Niko1221 19d16c7dbe ui-flows: Save logs is inside the View logs pop-up now 2026-10-16 19:54:56 +02:00
Niko1221 d465e37bc1 setup: NVFP4-QAD, NVIDIA's FP4-Local checkpoint downloaded pinned and prepared on this PC (nvfp4_local_pack prepare), with its own draft layer 2026-10-16 19:40:17 +02:00
Niko1221 c163678f7d merge f-split-agent6 0b871e2d7 (compaction, /research, response settings, browser and skills checks, time-skip file order, step summaries) into f-split-int 2026-10-16 19:38:02 +02:00
Niko1221 0b871e2d79 Time-skip: a big file that did not change is not listed as changed 2026-10-16 19:34:49 +02:00
Niko1221 4cca90e757 nvfp4_local_pack pack: the experts straight from the checkpoint through iq_pack (no 74 GB GGUF in between) 2026-10-16 19:32:05 +02:00
Niko1221 87d70909a8 Time-skip: the made page first in the compare list, housekeeping files (browser profile, caches) last and unread, never the whole result 2026-10-16 19:29:57 +02:00
Niko1221 7f9e719c8a nvfp4_local_pack: slim (the dense-only GGUF), convert --dense-only 2026-10-16 19:27:08 +02:00
Niko1221 f37abb1365 Tests follow the new reply room; the queued-message test waits for the goal to be running 2026-10-16 19:25:32 +02:00
Niko1221 2b54cf4fbd Model servers on this network get the thinking vocabulary of a local model (the Reasoning menu did nothing for them); effort and sampling drivers 2026-10-16 19:23:03 +02:00
Niko1221 b6b546ae13 Response settings reach agent turns (thinking budget, Expert sampling) through a local forwarding endpoint; step summaries from what the step changed; plan title heading 2026-10-16 19:09:54 +02:00
Niko1221 d42319b6c9 ple_reader_test: NVFP4 shares the I8 type with FP8; the metadata picks it 2026-10-16 19:05:11 +02:00
Niko1221 c937aeb1df /research gathers first (notes hidden), nudges once when no page was opened, then writes the answer in the fixed shape; /research in the slash menu 2026-10-16 19:04:25 +02:00
Niko1221 9c6255cfce NVIDIA Qwen3.8-Flash-Next-FP4-Local: converter (CSF scale decode, MXFP8 to Q8_0/BF16, MTP, NVFP4 PLE table) and the NVFP4 PLE row format
tools/nvfp4_local_pack.py convert|ple|mtp|verify reads NVIDIA's ModelOpt checkpoint and writes what the existing NVFP4 path takes:
the CSF block scales are decoded to plain E4M3 (experts then repack bit for bit), MXFP8 weights are dequantized exactly and take the
Q8_0/BF16 policy of nvfp4_convert, the n-gram table becomes a one-tensor GGUF of 90-byte rows, the MTP head the folder mtp_pack reads.
ngram.cpp reads that table as a new PLE format (I8 bytes, strata.ple.format = nvfp4x16).
2026-10-16 19:02:17 +02:00
Niko1221 998e9df959 /research: web search and page reading with a structured answer and sources; http driver commands for plan, mode switch and long chats 2026-10-16 18:52:45 +02:00
Niko1221 c6a86ce168 Local models compact automatically: reply room is half the window and compaction headroom fits the window 2026-10-16 18:52:45 +02:00
Niko1221 743a6b3c61 merge f-split-ui5 e9a2ca44a (round-7 UI, time-skip fixes, mode switch mid-goal, /plan, queued messages, uninstaller icon; includes agent6 eecb29593) into f-split-int 2026-10-16 18:49:39 +02:00
Niko1221 e9a2ca44a7 UI round 7: plan card, Plan sidebar, goal step and queued messages built on agent6's backend contract 2026-10-16 18:49:13 +02:00
Niko1221 89d0965408 Merge branch 'f-split-agent6' into f-split-ui5 2026-10-16 18:43:25 +02:00
Niko1221 8da1d3f8c1 Installer: the uninstaller uses the Strata icon, and the shortcuts name it explicitly 2026-10-16 18:43:25 +02:00
Niko1221 eecb29593c Plan mode (/plan) with approve/edit/reject/start-as-goal routes, goal steps with per-step events, queued messages during a goal 2026-10-16 18:42:14 +02:00
Niko1221 83c8f0b120 UI round 7: /plan card, Plan sidebar tab with step states, queued messages with cancel, floating round arrow, plain Privacy without a box 2026-10-16 18:37:30 +02:00
Niko1221 c751c23965 Merge branch 'f-split-agent6' into f-split-ui5
# Conflicts:
#	apps/host/src/components/TimeSkipPanel.tsx
2026-10-16 18:17:44 +02:00
Niko1221 2e1fcd15d9 Time-skip: run the page's own script in the A/B preview, working view in the sidebar, retry the file list; mode change applies to the next tool call mid-run 2026-10-16 18:16:56 +02:00
Niko1221 e849aef2ae UI round 7: live chat numbers with tween, fixed status-line columns, Subagents counts on title row, plain Privacy text, round latest arrow, time-skip never empty 2026-10-16 17:59:07 +02:00
Niko1221 50f973c9f9 merge f-split-agent5 60bfe0693 (agent-children stability: agent-bridge race, request-flow test wait) into f-split-int 2026-10-16 16:06:10 +02:00
Niko1221 60bfe0693a stability: helper rows always show working before completed (the held row is sent first), agent-children tests order frames by arrival not by the clock and use generous bounds, request-flow waits for a slow host start 2026-10-16 16:05:42 +02:00
Niko1221 9550318e34 merge f-split-ui4 2812c939f (Use your own GGUF: the refusal reaches the page, a second preparation no longer fails) into f-split-int 2026-10-16 15:20:24 +02:00
Niko1221 2812c939fe local-gguf-e2e.mjs: the Use your own GGUF flow through the installed layout on a box (preview, refusal, prepare, load, answer, unload) 2026-10-16 15:19:41 +02:00
Niko1221 87b90a7882 Preparing an adopted GGUF folder a second time no longer fails on the data folder
An adopted model reports the parent of the person's own GGUF folder as its data folder; the next preparation sent that as the folder for the packs and Control refused it ('a new data folder must be named Strata'). A preparation from a GGUF folder now uses Strata's own download folder. Found by the real-box check on P2003; test fails without the fix.
2026-10-16 15:17:11 +02:00
Niko1221 fa25086125 Use your own GGUF: an unsupported or incomplete GGUF folder is shown in words, not as a generic failure
Control answers a refused folder with exit 5 and the plan (ok:false, issues); the host threw that away and the page said only 'The files could not be checked'. The plan with its reasons now reaches the page, which says 'Strata can't run these files yet' and keeps the checker's codes under Details. Found by the real-box check (a Gemma vocab GGUF on P2003).
2026-10-16 14:53:25 +02:00
Niko1221 7c4b3d743e merge f-split-ui4 a7cd2b38e into f-split-int 2026-10-16 14:33:50 +02:00
Niko1221 a7cd2b38e6 Tune strings that already live in the Tune catalog are not duplicated in the main one 2026-10-16 14:33:24 +02:00
Niko1221 9cede7db08 ui-flows: Models one-view check, outcome card and an empty task navbar 2026-10-16 14:30:31 +02:00
Niko1221 35c72bcd10 Models tab in one view with the family switch, one result card for every tune outcome, the tuned values in Expert mode, glow instead of a spinning ring
- Models: the family switch (Qwen3.8-Flash-Next, Qwen3.6-35B-A3B) is back in the navbar without the word Model; version, context and computer use, the tuning line and Load/Tune sit in one card beside This computer, so the tab fits the first view; image generation moves into Expert mode
- Tune: one result card for faster, already the fastest, couldn't measure (busy PC, with plain copy and Try again), failed and stopped, driven by the host's code, reason_code and values; the same Tune dialog frame for all of them
- Tune running: the step icon glows and a small dot beeps instead of a ring spinning around a square tile; reduced motion respected
- Expert mode: Context memory and Low-RAM mode show the active tuning's values, the engine settings list what the tuning sets, and loading does not re-prepare the model because of them
- the tuning line (Using your tuning / Switch off) is a calm line with an icon
- strings in 8 languages; tests in tune-outcome.test.mjs
2026-10-16 14:30:31 +02:00
Niko1221 99cc9829a0 Other local models: the back link says Back; the running version's group can be collapsed 2026-10-16 14:30:29 +02:00
Niko1221 837f992bd9 Owner round 6: short model names, one logs button, plain Monitor sentence, Privacy notes, Tasks cleanup
- Settings > Storage: downloaded optimized models show family + quantization with the full name as a tooltip, size aligned, Delete at the end
- Engine > Diagnostics: one View logs button; Save logs lives in the logs view
- About > Privacy: six short lines across the box instead of one long paragraph
- Monitor: the request summary is a plain sentence ('In 18 requests your average output was 47.4 tok/s and prompt reading 1,332 tok/s, reusing 91K tokens.'); the screenshot leaves out Model state and Conversation cache
- Tasks: Bypass permissions replaces 'Allowed to make changes' and is the default for a new task; an opened task has an empty left navbar and no Model section, its name in the page, and a readable Markdown last answer; no deleted-tasks list, restore or suggestions; + icon opens the editor modal (Draft it with AI inside it); removing says 'Task removed'
- strings in 8 languages; tests in ui4-fixes.test.mjs
2026-10-16 14:30:29 +02:00
Niko1221 19ea30f6ab merge f-split-agent4 4b5cc08d1 (skills end to end, New skill button, agent skills folder, goal-status race fix) into f-split-int 2026-10-16 14:26:57 +02:00
Niko1221 4b5cc08d1a a goal-step status is never swallowed by the 300 ms gap (fixes the flaky /goal test); skills strings re-added to the merged catalog 2026-10-16 14:24:21 +02:00
Niko1221 ca34357b08 skills: plain card words for loading a skill (8 languages), Skills page render script 2026-10-16 14:21:33 +02:00
Niko1221 4818b1f4a1 Skills: write one in the Skills page (New skill), saved as a folder in the app's data; the whole list is written out for the agent, which scans it (customSkillDirs) and watches it, so a saved skill reaches the next message without a restart; tests 2026-10-16 14:21:32 +02:00
Niko1221 0c165b48bf merge f-split-s5 dab63e9c3 (tuning presets on Models, effective values, outcome codes, apply_saved_tuning) into f-split-int
native-control.mjs: this tree's publicState/localPreview kept, s5's tuneEffective/tuneOff added to the exported API. NativeModels.tsx: the loaded-model Unload/Ready actions kept, s5's NativeTuneBadge added above them. NativeTune.tsx: ui2's result view kept; s5's busy_pc sentence and the steady-aware answer notice added.
2026-10-16 14:19:31 +02:00
Niko1221 dab63e9c38 tuning feedback: structured outcome (faster / already_fastest / busy_pc / failed / stopped + values), tuned values readable (presets effective, settings carry them), which tuning is active and off/on, applied at every start (restart and unload/load tests) 2026-10-16 14:16:07 +02:00
Niko1221 a79e653036 merge f-split-agent3 df52a2854 (API connection output limit) into f-split-int 2026-10-16 14:12:08 +02:00
Niko1221 f9bfe8bc12 merge f-split-agent3 c99896ae6 (status, goal and time-skip events and backend, time-skip keepalive and stop, atomic.py) into f-split-int
Conflicts: locales (agent3's 52 keys renumbered s2937-s2988 in all 8 files), timeskip.mjs (ui3's image() kept, agent3's stop()/whenEnded kept, imports unioned), store.ts (ui3's comparison-open rule plus agent3's markTurn), Chat.tsx (ui3's working card and chat card kept, agent3's timeSkipTab dropped), time-skip.ts (duplicate activity field removed).
2026-10-16 14:12:08 +02:00
Niko1221 df52a28548 API models: answer room is the connection's own maximum output when the provider lists one, else a generous 32K bounded by the window minus the prompt (was a quarter of the window, at most 8,192, which cut thinking models off with no answer) 2026-10-16 14:09:37 +02:00
Niko1221 334dde73a2 merge f-split-ui3 088f8685a (live status line, reasoning panel, goal bar, tool rows, time-skip comparison) into f-split-int 2026-10-16 14:09:13 +02:00
Niko1221 c99896ae66 the real-browser test is opt-in (STRATA_TEST_BROWSER=1); HTTP driver goal bar command 2026-10-16 14:08:27 +02:00
Niko1221 04cca3bca8 status sentences carry their catalog template and values and exist in 8 languages; goal tools have plain words; per-request thinking count 2026-10-16 14:05:21 +02:00
Niko1221 20b377d30d Time-skip and tune job records are replaced with a wait for a reader (Windows 'Access is denied' when the host reads the file at that moment: it ended jobs at random) 2026-10-16 13:29:12 +02:00
Niko1221 088f8685a7 Time-skip in chat: working view with Stop, comparison sidebar only when there is an update, A versus B that adapts to the kind of file
- while a real run works, an animated A/B card shows what the assistant is doing (host activity), elapsed time and Stop (TimeSkipWorking); the sidebar no longer opens at the start and opens only when the finished run changed files; a run that changed nothing says so in the chat and has no tab
- the finished A versus B (TimeSkipCompare) keeps one window size whatever was made: text and Markdown rendered with changed words marked; HTML as two sandboxed previews (no permissions, no scripts, no outside requests) with a View code toggle; pictures side by side, click to enlarge; code and other files as a folded line diff; several files through a compact file switcher
- host: changed pictures are listed as pictures and served from the run's own folders by GET /api/timeskip/file (images only, inside the run, size-capped)
- strings in 8 languages; tests for the file kinds, word and line diffs, the HTML policy, the activity words and the picture route
2026-10-16 13:20:20 +02:00
Niko1221 3e6c354504 status test no longer depends on timing; resume following a Time-skip after a reload keeps the Stop button 2026-10-16 13:19:19 +02:00
Niko1221 b085d54b94 status tests (plain chat, real Harness file write), HTTP driver for real-model checks 2026-10-16 13:16:59 +02:00
Niko1221 ee8b23bcc7 Time-skip: streamed revision answer with a beat (a slow turn no longer ends in 'could not be reached'), Stop ends the job tree and deletes the copies, activity in the status route and SSE, sidebar only when there is an update, Stop button while it runs 2026-10-16 13:15:12 +02:00
Niko1221 7c4092dabe Chat feels alive: live status line, one-line reasoning, goal bar
- status line under the newest message: pulse, a rotating playful word only while it thinks, the real action once a tool runs, elapsed time and tokens; only the newest status frame is ever shown and it is cleared on idle, done, error and stream end (src/lib/live-status.ts)
- reasoning is one quiet line (what it is thinking about, then 'Thought for 38s'); opened it follows the newest line unless the person scrolls up
- goal bar above the input: small, icon-only pause/resume, edit, remove, from the goal frame, the SSE goal event and /api/goal (src/lib/goal.ts); resume sends goalAction:'resume' with no user bubble
- all strings in 8 languages; tests in tests/live-chat.test.mjs
2026-10-16 13:14:01 +02:00
Niko1221 965f22e3db live status frames for agent, goal and plain chat turns (thinking tokens, composing a file call, running tool, waiting for approval, helpers, goal step), goal state frames and goal bar backend (state, pause, edit, clear via the Goal Remotes; resume through the chat stream) 2026-10-16 13:11:40 +02:00
Niko1221 44a12d78be Tool rows in chat: compact lines, no dividers 2026-10-16 13:05:20 +02:00
Niko1221 aaf37247da integration after ui2: a second-try failure is shown once (details keep the first stop), error Details box is height-capped so the small tuning pop-up never scrolls, ui-flows expects the real tuner's Tune model button 2026-10-16 11:55:14 +02:00
Niko1221 c5755ed837 merge f-split-ui2 baeb4533b (approved UI designs, context fix, workspace default with rename, autotune progress) into f-split-int
Locales: both sides added keys from s2676, so ui2's 88 keys are renumbered s2849-s2936 in all 8 files. events.mjs: ui2's verbsFor target plus agent2's ask card. index.mjs: the node:path import takes both sides' names (relative, sep). The workspace default is ui2's (Documents/Strata/Workspace).
2026-10-16 11:27:06 +02:00
Niko1221 baeb4533be Approval cards in plain words, Tasks open view, This computer card, version list, and the Tune steps with a live speed
- approval cards: one plain question per tool call (server/agent/ask.mjs maps commands and files to sentences; src/lib/ask.ts holds the translated templates), a compact calm card, the exact command under Details
- Tasks: the opened task shows schedule, what it may do, last result, instructions and last answer; back link before the title like the chat's navbar; pause/resume and delete icons, Edit, Run now; pause and play icons added to the sprite
- Models: 'This computer' shows graphics memory and RAM bars with what the chosen version takes, and one plain line about the SSD (src/lib/native-fit.ts, from the catalog's memory_gb and the hardware report; the n-gram table is not counted)
- Models: one-line version list: Recommended (the fixed four), Swift, GSQ-RCO, NVFP4, Gyro, Unsloth groups, Use your own GGUF; RAM+VRAM in bold, download size muted, 'Uses SSD N GB' only where it applies
- Tune: five steps with their own animated icons and states, a live speed readout and sparkline, a result card with the gain and what it was saved as; reduced motion respected; the tuner now reports each measurement's speed
- catalog: memory_gb and mem_total per version (tools/strata_mcp.py)
- tests: ask, native-fit, tune-phase, autotune speeds, memory figures; headless checks (ui-flows ui2/ui2d, scripts/approval-flow.mjs), light and dark
2026-10-16 11:24:09 +02:00
Niko1221 ea2d565e60 merge f-split-agent2 c7b65c7f7 (/loop, /goal, /time-skip, keyless search, system-browser tool, Settings Tools, Plan refuses connected tools) into f-split-int 2026-10-16 11:08:59 +02:00
Niko1221 c7b65c7f7b tools flow script restores the settings it changes 2026-10-16 11:05:57 +02:00
Niko1221 d146b0da29 plain browser card sentences (waiting, scripts, tabs, back, close), no empty braces on cards; tools flow script for the real model 2026-10-16 11:01:55 +02:00
Niko1221 df6262f270 Merge perf-PR group sm75 (f-145-pp-sm75) 2026-10-16 10:59:08 +02:00
Niko1221 9f24934663 tuner: a run that could not measure any candidate is inconclusive, not default confirmed 2026-10-16 10:58:33 +02:00
Niko1221 21c579727c Merge perf-PR group rdna2 (f-145-pp-rdna2) 2026-10-16 10:54:10 +02:00
Niko1221 b277412eb1 Merge perf-PR group rdna34 (f-145-pp-rdna34) 2026-10-16 10:53:17 +02:00
Niko1221 cbc6e3d8f8 Merge perf-PR group cache (f-145-pp-cache) 2026-10-16 10:53:00 +02:00
Niko1221 70f8268dab Time-skip refuses the app's own folder and a folder too big to copy, in plain words in all 8 languages 2026-10-16 10:52:01 +02:00
Niko1221 2f3a5d9013 Merge perf-PR group serve (f-145-pp-serve) 2026-10-16 10:51:54 +02:00
Niko1221 da6d8d9205 Merge perf-PR group batch-1441 (f-145-pp-batch-1441) 2026-10-16 10:51:15 +02:00
Niko1221 e8b1edbcb7 Merge perf-PR group batch (f-145-pp-batch) 2026-10-16 10:49:15 +02:00
Niko1221 0848864a03 cards for the browser and the person's own tools say what happened in plain words (8 languages) 2026-10-16 10:48:49 +02:00
Niko1221 8075af7c2a calmer Time-skip wording (one note, no repeated reassurance), plain Manual reason for the person's command tools, agent README for Settings Tools 2026-10-16 10:47:46 +02:00
Niko1221 5a6f4ed5fb Merge perf-PR group cuda86 (f-145-pp-cuda86) 2026-10-16 10:46:42 +02:00
Niko1221 533f6b713f browser tool (Playwright MCP on the person's Chrome or Edge, separate empty profile; Manual asks before opening, clicking, typing, not before looking; Plan refuses), staged into the installer; Tools screen strings in all 8 languages; Time-skip result view without the sample success check; /loop words translated 2026-10-16 10:45:44 +02:00
Niko1221 926b0bbe6d Merge perf-PR group sycl (f-145-pp-sycl) 2026-10-16 10:43:56 +02:00
Niko1221 62b7d251ae Merge perf-PR group cpu (f-145-pp-cpu) 2026-10-16 10:43:55 +02:00
Niko1221 c20c4aed11 tuner: say so when the default itself is not repeatable (RTX 3060 resident mode, A750); exact knobs are still auto-picked there 2026-10-16 10:36:37 +02:00
Niko1221 24e0157b53 Settings Tools behind the screen: tool groups on/off reach the agent (restart on change), web search with no key (DuckDuckGo, polite, cached) or the person's own Exa/Perplexity/DeepSeek key in the environment only, the person's own tools (instructions, commands, tool servers) through the app's tool server, Plan refuses connected tools, keyless API connections work in the agent; browser finder 2026-10-16 10:35:36 +02:00
Niko1221 f662759046 Workspace rename strings in all 8 languages (after rebase onto df13760cf) 2026-10-16 10:23:59 +02:00
Niko1221 0d4e0dde8c Settings Tools tab (screen only, mock-up state), Time-skip trial runs on the chat's API model too 2026-10-16 10:22:09 +02:00
Niko1221 db9c322bce slash commands: /goal and /loop in the menu with plain descriptions, /loop repeats a message while the chat is open, /time-skip runs a real job on a copy of the chat folder (assistant revises, files listed, update kept or original restored on request, rollback copy); apply of a reviewed attempt in Strata Control 2026-10-16 10:22:09 +02:00
Niko1221 a655bf22b2 Context from cached prompts for agent chats, and a default workspace that is never the app folder
- agent context: the Harness reports a cached prompt as cacheReadTokens, apart from inputTokens; the context now counts input + cache read + cache write + reply of the last request (a long agent chat showed 0-3%; checked against a real Swift IQ2_XS 64K run: 8,152 tokens, matching the engine's own 8,042-token prompt + 110 reply)
- first workspace: Documents\Strata\Workspace (display name Strata) unless STRATA_WORKSPACE names one, never the app folder; a saved first workspace that is the app folder moves there and its chats follow through an alias
- workspaces can be renamed (display name only, path kept): PATCH /api/workspaces/:id, pencil in the sidebar, 8 languages
- tests: workspaces.test.mjs, cache-aware context in functional-fixes.test.mjs
2026-10-16 10:22:08 +02:00
Niko1221 39f3670d41 docs: STRATA_KV_HOST_REGISTER and our measurements for STRATA_EXCHANGE_ROTATE (#2042) and STRATA_PCIE_BALANCE (#1548)
#1498 pins the KV host copy with cudaHostRegister under WSL; the switch forces either order. Measurements: 5 interleaved pairs on an RTX 3060, an RTX A4000 and a Tesla P100.
2026-10-16 10:18:08 +02:00
Niko1221 df13760cf4 files made by a run as administrator no longer lock the user out: explicit user-SID entry and owner on create (private_mkdir, restrict_to_current_user), process default owner/DACL set to the user, self-repair of Strata's own per-user folders (APPDATA, LOCALAPPDATA, ~/.strata) on read, at host start and by 'strata repair-access', plain message when it cannot; tests; tune_session loopback allowed in the no-acquisition audit 2026-10-16 10:10:43 +02:00
Niko1221 e1dca4bff5 merge f-split-gyro b15b209a0 (Step B: Gyro trellis types through MMQ in the pinned ggml via an overlay) into f-split-int 2026-10-16 07:08:30 +02:00
Niko1221 95e4f27f17 tuner knobs from TUNING-SPEC draft 2: kv-resident 65536 at long contexts and the prefill chunk as exact candidates (prompt metric), pin-by-need/host-core/kv-grow last, no workers two below, single best knob 2026-10-16 06:42:20 +02:00
Niko1221 1ea66f60af owner decisions: goal, skill, workflow and fork tools back in the agent (short plain prompt kept); agent and chat default reasoning effort Medium, with a test per level (Off/Low/Medium/High/unset) on the chat_template_kwargs the server receives; Tune dialog switch reads 'Allow settings that change answers very slightly (same quality)' in all 8 languages, off by default, with a test 2026-10-16 06:30:44 +02:00
Niko1221 394de92915 Functional-test fixes: agent tok/s and context, plain tool rows, Thinking Off proof, profile used, no download wording for installed models, plain chat, smaller agent prompt
- agent path: live stats frames and a final tok/s from the stream timing; the context is the last request (one source)
- tool rows say what happened in words (Wrote a file, Ran a command, Couldn't write the file) with the file or command; approval cards show it too
- a launch that returned without error reports 'Started.' instead of 'nothing to show'
- agent system prompt replaced by a short plain one; goal, skill, workflow and fork tools left out (first message about 7.5k -> about 4.3k tokens)
- Models: the running model reports the profile it was started with and the page shows it; Reasoning popover names the loaded native model
- installed model with other settings: 'Apply settings and load', no size, no download wording
- plain chat: agent:false or tools:false answers directly and never runs a tool
- tests: stats/context/verbs/launch/prompt size, server-side enable_thinking=false with the agent's request shape, plain chat over HTTP, ui-flows profile scenario
2026-10-16 04:20:20 +02:00
Niko1221 b15b209a03 Step B: Gyro's trellis MMQ prompt kernels on an upstream ggml, so the packaged engine reads Gyro prompts as fast as rc2
cmake/strata_ggml_tq_overlay.cmake writes, at configure time, a copy of the pinned ggml-cuda headers (and ggml.h, quantize.cu) into
the build folder with TQ2_T / TQK6 / TQK7 routed the way IQ2_XS is: same Q8_0_16 tile layout, dot kernels and tile configs (the
fork's own configs for them are copies of IQ2_XS's), three template instances, the tile loader and tq.cuh from third_party/ggml-tq.
Only the strata_mmq target sees the copy; every other type is compiled from the untouched files. Anchors are checked: another
llama.cpp turns the overlay off with a message (FP16 prompt path). moe_mmq.cu knows the trellis block geometry itself.

The tile loader and the trellis tables are agentionai/llama.cpp's (MIT, ggml authors; Laurent Zuijdwijk), ported from
agentionai/llama.cpp 93a24e5 ggml/src/ggml-cuda/{tq.cuh,mmq-load-tiles.cuh,tq-lut-i8.cuh}.

Checked: Gyro-S and Gyro-M exact_pp 4096/20000 residual dumps IDENTICAL to rc2 (R9700 gfx1201); Q2_0 and IQ3_XXS IDENTICAL to f-split
on gfx1201 and sm_86; ctest ggml_tq_overlay.
2026-10-16 03:51:27 +02:00
Niko1221 7ff67fe772 docs: the #2340 / #2207 Q6_K decode kernels and their switches (STRATA_MMVQ_LARGE_GRF, STRATA_Q6_PLANAR) 2026-10-16 03:46:10 +02:00
Niko1221 76cdc271c0 merge f-split-s4b 0398e09af (README docs on top of 0edf5cb9f) into f-split-int 2026-10-16 03:22:26 +02:00
Niko1221 15937b7e80 tuner measures the engine a load starts: the app's request-slot choice (and fit_max_tokens) moves into buildConfig, which both the load and the tune use; test that a tune's saved config equals a load's for the same model and profile, on a 12 GB and a 48 GB card 2026-10-16 03:20:28 +02:00
Niko1221 a161098036 merge f-split-s4b 0edf5cb9f (Harness in the app chat, Manual approvals, request slots) into f-split-int
index.mjs: createNativeControl gets control, appUpdater and hardwareProbe; the agent bridge gets componentsDirectory() and the bundled Harness folder.
native-control.mjs: this tree's file kept, s4b's parallel choice, fit_max_tokens, instance parallel/mcpServers re-applied after buildConfig; its own retry around control.start is folded into load-host's: ONE second try in all, which carries the memory change and/or the drop of several-at-once, with load-host's plain sentence (FALLBACK_NOTE only when no memory change applies). Test added.
locales: both sides had added keys s2571.., so s4b's seven task keys are renumbered s2615-s2621 in all 8 files.
2026-10-16 03:17:17 +02:00
Niko1221 0398e09af5 agent README: thinking and output budget, tasks 2026-10-16 03:12:53 +02:00
Niko1221 004e22c1de merge f-split-s5 (real tuner and presets) into f-split-int
native-control.mjs: this tree's file kept (S6 updates, load retry, remove), s5's buildConfig split out of loadEngine, auto-tuner, tuneConfig/tuneStatus/autotune, autotune and recheck operations and presets re-applied.
NativeModels.tsx: s5's Tune model button (the real tuner) and NativeTune dialog; the old short-benchmark Tune button stays out. native.ts: job progress field.
2026-10-16 03:11:50 +02:00
Niko1221 9691ea2763 sycl: the planar Q6_K head kernel gets the large-GRF launch at 2-8 columns on Arc A-series (#2340 + #2207)
The planar kernel of #2340 is the wide kernel's arithmetic, but it was launched without the 256-GRF mode that #2207 gives the GGUF-layout
wide kernel, so on an Alchemist card it spilled: mmvq_bench 2560 x 248320 on an A750, 4 columns, 1.78 ms (GGUF layout, large GRF) against
15.4 ms planar, 8 columns 4.8 against 48 ms. Same rule as #2207 (STRATA_MMVQ_LARGE_GRF, default on Alchemist only), same arithmetic.
2026-10-16 03:06:25 +02:00
Niko1221 a4d20310f1 Gyro needs engine 0.1.45: it ships in the split release's engine (GYRO_ENGINE, docs, test) 2026-10-16 03:05:27 +02:00
Niko1221 0b22b8c736 tuner: fix the RAM-floor rule syntax 2026-10-16 02:48:56 +02:00
Niko1221 a3dd4b702d tuner: a candidate is refused for low free RAM only if it is worse than the base (a RAM-bound base can sit below the floor itself) 2026-10-16 02:48:30 +02:00
Niko1221 a1447b678e Merge f-145-fix3: #2622 PLE gather, #2267 config args, #2644 split share, #2637 note, #2666 P2P probe, PR #2425 opt-in 2026-10-16 02:41:38 +02:00
Niko1221 5fb8fa95b3 load retry: the same stop twice is reported once (the second try stays in the job's details); fixes the 1280x700 tuning-error pop-up that scrolled in ui-flows 2026-10-16 02:33:25 +02:00
Niko1221 bcbf2caf3f merge f-split-gyro (Gyro-S/M trellis port and catalog entries) into f-split-int 2026-10-16 02:22:37 +02:00
Niko1221 31d36db678 docs: SECOND_GPU.md - the peer tier tests peer access at start, keep headroom on both cards (#2666) 2026-10-16 02:18:30 +02:00
Niko1221 3315072467 peer experts: peer access is used only if a test copy each way arrives exact (#2666)
On 2x RTX A4000 behind a host bridge cudaDeviceCanAccessPeer says 1 and cudaDeviceEnablePeerAccess succeeds, and
cudaMemcpyPeerAsync then returns success having moved nothing (1,048,576 of 1,048,576 floats wrong in both directions
in a stand-alone test; right without peer access enabled, the driver stages it). The peer prompt path trusted it: the
peer's input rows were zeros and its results never reached the primary, the first prompt's MoE output was garbage and
the logits NaN (--peer-device 1, --peer-reserve-mib 1500..2100; STRATA_PF_PEER_COMPACT is not the cause).
PeerExperts::open now copies 64 KiB each way once after enabling access; a wrong result turns peer access off again and
takes the host route (the one STRATA_PEER_NO_P2P=1 asks for), and says so. A test that cannot run keeps the old trust.

Reported by @CaptRauti on #2666 (WDDM, 2x RTX 3060: the no-P2P route is the one they ran).
2026-10-16 01:52:55 +02:00
Niko1221 0edf5cb9fc tasks: the host's task event reaches the page (toast); real-clock task and mode flows for a real model 2026-10-16 01:49:30 +02:00
Niko1221 14acfc3b3c desktop test: the stable-origin test no longer assumes port 47831 is free (an installed Strata holds it on this PC) 2026-10-16 01:42:15 +02:00
Niko1221 529a7019ae install job: the finished job keeps the merged progress (engine part and parts); test that the packaged install job fetches the engine through Control and that a failed engine keeps the model 2026-10-16 01:36:22 +02:00
Niko1221 fd0d7fd875 tuner: the quick budget reserves only the extra base rounds and the final base 2026-10-16 01:36:19 +02:00
Niko1221 0bf5fe1ebe tests: the packaged-mode fixture allows Control's installer tool process (fr runs strata_mcp.py there); still no other process 2026-10-16 01:33:54 +02:00
Niko1221 07fe758ac9 merge f-145-int 194c26881 (model-card sampling defaults, web sampling, answer room, thinking budget) into f-split-int
serve/server.py: the model_arch line joins the existing engine == strata block next to requested_slots.
2026-10-16 01:31:17 +02:00
Niko1221 90a5d082bc merge f-split-load (pin_budget reserves on Windows) into f-split-int 2026-10-16 01:30:58 +02:00
Niko1221 fa0d99e5b7 merge f-split-load-host (one safer second try when a load fails) into f-split-int 2026-10-16 01:30:58 +02:00
Niko1221 8820fa433d merge f-split-fr (owner app UI fixes, rounds 1-5) into f-split-int
native-control.mjs: fr's file kept, S6's engine step, component updates, plain update messages and app updater re-applied on it;
native.ts: fr's types plus capabilities.componentUpdates; SettingsUpdates: fr's per-row update buttons, Update engine enabled without a model when component updates exist.
2026-10-16 01:30:48 +02:00
Niko1221 781690f397 tuner: per-request points get a control and a fresh-engine 5-round confirmation (the first real run showed sending any setting reads ~10% faster) 2026-10-16 01:25:47 +02:00
Niko1221 20e0bf2e9d direct chat: Thinking budget in the Reasoning effort menu (reasoning_budget_tokens); Off reaches the server as none 2026-10-16 01:25:09 +02:00
Niko1221 3571ceb1e4 do not track generated Tauri schemas 2026-10-16 01:22:20 +02:00
Niko1221 a4a0da0d5d app updater: the installer starts outside the app's Job Object (found by the end-to-end run), plain message when no update key exists 2026-10-16 01:22:14 +02:00
Niko1221 22265c6ac7 tuner: fix the killed-engine pattern 2026-10-16 01:16:07 +02:00
Niko1221 e62e6bf2f5 tuner: a killed engine (no message) is an out-of-memory 2026-10-16 01:15:37 +02:00
Niko1221 84ca0b6c21 tuner: five decodes per run, early drop only after a 5% loss, noisy default-confirmed says so 2026-10-16 01:11:57 +02:00
Niko1221 daa45b1e37 thinking: the Strata server reads chat_template_kwargs, not a top-level enable_thinking (Off had no effect); plain output for read-only PowerShell noise 2026-10-16 01:11:17 +02:00
Niko1221 ff62ce3fbe app: Tune strings without the duplicate Check again 2026-10-16 01:11:15 +02:00
Niko1221 7d43470f31 app: Tune dialog tidy, short Check again label; headless UI fixture for it 2026-10-16 01:10:56 +02:00
Niko1221 14168890c9 app catalog: a test for the Gyro sizes in the Qwen3.8 group 2026-10-16 01:08:57 +02:00
Niko1221 10ede85dd1 tests: skip the real-install step test when the disk is too small for setup's own check 2026-10-16 01:05:23 +02:00
Niko1221 5f054d2a2c tests: resident-mode detection 2026-10-16 01:04:17 +02:00
Niko1221 aab3810b54 tuner: the GPU cache's 'resident experts' wording is not the low-RAM resident mode 2026-10-16 01:03:41 +02:00
Niko1221 bd773bbd77 app: Tune model dialog (real tuner, saved tunings, undo) reusing the Auto-tune look; its strings in separate catalogs 2026-10-16 01:02:17 +02:00
Niko1221 35cfd187cf host: autotune and recheck operations, saved-tuning routes (Strata Control's tuner and presets, run on the config a load would use) 2026-10-16 00:58:17 +02:00
Niko1221 eaf8745697 docs/GYRO.md: setup is the install path, our measurements, credit and licenses; docs/MODELS.md lists Gyro-S and Gyro-M; tools/gyro_setup.sh (rc2's script) is replaced by setup 2026-10-16 00:56:04 +02:00
Niko1221 c0616f06be tuner: AMD architecture from the kernel for the never list 2026-10-16 00:54:20 +02:00
Niko1221 e6b3af5889 tuner: waiting for the box lock does not count against the time budget; session tests 2026-10-16 00:53:30 +02:00
Niko1221 8e2460d18c owner feedback round: unified local-GGUF page and Manage models, no Tune model, Engine settings reduced with working Save logs, update buttons only when available, GPU data in Monitor with no model, thinking dots stop, Latest spacing, calmer native status reads, local GGUF adoption via --gguf-dir, no Contrast option or request-log button 2026-10-16 00:53:05 +02:00
Niko1221 33120935a3 tuner: a queued kernel flock for the box lock, so other streams' waiters are served in order 2026-10-16 00:52:06 +02:00
Niko1221 bb33349149 tuner: memory and quiet checks run inside the box lock right before each start 2026-10-16 00:49:59 +02:00
Niko1221 f23eda8c95 auto-tuner: single-knob candidates from the measured spec, interleaved rounds with noise and drift guards, memory guards, saved presets (one store for CLI and app), re-check, strata tune auto / strata presets 2026-10-16 00:48:42 +02:00
Niko1221 194c26881a serve: Anthropic thinking.budget_tokens is a thinking budget too; docs for the answer room and the budget; test 2026-10-16 00:44:38 +02:00
Niko1221 d0e990ba78 serve: max_tokens counts the thinking - without a budget, thinking is closed early enough to leave a quarter of the cap for the answer (cap 8192 cut the thinking off with no answer); config answer_room:false opts out; tests 2026-10-16 00:38:04 +02:00
Niko1221 326e391188 merge f-split into f-split-s6 2026-10-16 00:33:37 +02:00
Niko1221 158a42d8b6 tasks run as the agent in their own session under a per-task mode, with a plain result and a notification; task editor and toast strings in all eight languages 2026-10-16 00:33:05 +02:00
Niko1221 be1a871349 agent: output budget for a local model is the window (was 8192), the Reasoning menu reaches the request, fit_max_tokens for app-loaded models, plain outcome for silent commands 2026-10-16 00:30:13 +02:00
Niko1221 5e33c4e85d Gyro: the MCP recommendation asks 4 GB of room beside the memory figure; tests for the Gyro catalog 2026-10-16 00:16:01 +02:00
Niko1221 b74d6ad3d3 stage-harness: flatten pnpm's links into one real folder per package (the installer copies files) 2026-10-16 00:15:08 +02:00
Niko1221 375f76fef0 setup: a tip for Gyro installs on the ready-made engine (the fork's prompt kernels need a source build) 2026-10-16 00:14:27 +02:00
Niko1221 02bbf51fb7 Gyro: the Hadamard row kernels go up in slices of 65535 rows (gridDim.y limit): the FP16 prompt path rotates T*K rows, which a 16K-token chunk takes past it (invalid configuration argument on a build without the fork's MMQ) 2026-10-15 23:46:50 +02:00
Niko1221 e863486567 web: the page sends no sampling until the user changes it, so the server's model-card defaults apply (stored 0.6 counts as not set); test (#2685) 2026-10-15 23:34:15 +02:00
Niko1221 945971648a desktop: ship the staged Harness as resources/harness (built by stage-harness.mjs) 2026-10-15 23:28:08 +02:00
Niko1221 365c60e7a4 Metal: stubs for the Gyro Hadamard entry points (the engine does not run trellis experts; native_expert_supported refuses them at load) 2026-10-15 23:02:00 +02:00
Niko1221 34204c4a31 Gyro: the CPU trellis rows build for arm64 (scalar kernel; the AVX units are x86-only) and the Metal build lists hadamard.cpp and tq_cpu.cpp 2026-10-15 22:47:19 +02:00
Niko1221 805a308d2e pin budget: CUDA-on-Windows default only (the HIP build keeps its pinning), docs/DETAILS.md 2026-10-15 22:45:55 +02:00
Niko1221 a8091f4d2e serve: a request with no sampling gets the model card's (thinking 1.0/0.95/20; not thinking 0.7/0.8/20 + presence 1.5; Qwen3.6 too); greedy only when asked (#2685) 2026-10-15 22:35:55 +02:00
Niko1221 ff14364de0 packaging: recipe and table row for the gfx1103 (Radeon 780M) Linux HIP engine package
Built on P2006 from the f-split tree and run through the Terminal archive: resolve (kfd gfx1103), tune, serve and client, TUI, Ctrl+C/SIGHUP, saved preset, memory back to baseline.
2026-10-15 22:35:46 +02:00
Niko1221 4e67f1f045 host: one automatic safer second try when the engine dies while loading (smaller pin / RAM budget), said in plain words and not saved; Leave 20% also sets the engine's own RAM headroom 2026-10-15 22:31:15 +02:00
Niko1221 e1d11fe727 Gyro: native_fmt knows the trellis types' geometry itself, so a ggml built from upstream (no such types; its type table is indexed by the id) is never asked about them 2026-10-15 22:28:55 +02:00
Niko1221 4ff5cd4050 pin budget: test knobs (simulated ceiling, 'driver took everything'), report lines, smaller default reserve, pin_budget_test 2026-10-15 22:27:30 +02:00
Niko1221 4f74de9cdb Keep pinned-memory headroom while the expert arena pins; retry the engine's required pinned buffers once after handing it back (Windows load failure: verify mapped staging allocation failed) 2026-10-15 22:13:37 +02:00
Niko1221 45f6111fc8 docs: MULTI_GPU.md - an unequal pair under layer-split auto (#2644, measured on z440 with the speed-scale seam), STRATA_SPLIT_UNIFORM (#2418) 2026-10-15 22:03:34 +02:00
Niko1221 ffbc606682 delete a downloaded native model: strata_remove (own files and run config, shared files kept), Delete in Models and Storage, real disk use per model 2026-10-15 21:48:51 +02:00
Niko1221 eca3aef3a2 Gyro: the CPU trellis kernels run on every CPU (scalar without AVX2: upstream ggml has no trellis vec_dot to fall back to); the SYCL build lists the new CPU sources 2026-10-15 21:47:32 +02:00
Niko1221 ba64156fd4 Catalog: Gyro-S and Gyro-M (agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) in setup, the Terminal, the app and the MCP server
Both files are one GGUF with the n-gram table inside: 58.5 GB and 92.0 GB downloads for 27.6 GB and 35 GB of VRAM + RAM together.
Neither is a default; the MCP recommendation names them only where the 2-3-bit sizes and the Coder do not fit. Pinned to the Hub
revision 1ada08422e40 (sizes and SHA-256 from its LFS pointers). Model license: Qwen Community License 1.0.
2026-10-15 21:44:02 +02:00
Niko1221 ec53cae11f request slots: fit to the expert cache the card would have (measured), hardware from the host probe, safe default with no read 2026-10-15 21:43:30 +02:00
Niko1221 697cbc6376 stage-harness: peers, safe pruning; profile: office converter off 2026-10-15 21:41:06 +02:00
Laurent 26da97bde0 docs/GYRO.md: clone the rc2 branch (Strata v0.1.40.3 + Gyro)
Ported from agentionai/Strata rc2 9ff02c9 (rc2 = 9ff02c9), author Laurent.
2026-10-15 21:31:35 +02:00
Laurent Zuijdwijk 701f31964b docs/GYRO.md: the CPU trellis kernels and their switches
Ported from agentionai/Strata rc2 6bfb1e4 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:35 +02:00
Laurent Zuijdwijk 5ebea67b79 native_expert_parity: check the CPU trellis kernels in the synthetic modes
For tq2_t / tqk6 / tqk7 experts (--synthetic, --synthetic-hadamard), the
gate and down rows on every ISA the CPU has against the scalar kernel bit
for bit, and the scalar kernel against ggml-cpu's vec_dot on the same
Q8_0 activations. The CPU path of check (b) already runs through the new
kernels via native_gu_rows / native_down_rows.

Ported from agentionai/Strata rc2 7d0ecd0 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:35 +02:00
Laurent Zuijdwijk d0a02fe426 CPU trellis kernels: AVX2 / AVX-512 decode-and-dot for TQ2_T / TQK6 / TQK7
Expert-cache misses of the Gyro files run on the CPU workers. Until now those
rows went through ggml-cpu's vec_dot one token at a time. Strata now has its
own kernels for the three trellis types (tq_cpu.hpp), chosen at run time like
the i-quant ones:

- the 32768 x 4 int16 state table (one 8-byte lookup per four weights, built
  from the codebook in third_party/ggml/ggml-common-tq.h, now also visible to
  host code), scalar state extraction, four table rows per YMM by broadcast
  loads + VPBLENDD, VPMADDWD (AVX2) or VPDPWSSD (AVX-512 VNNI);
- the weights of a block are decoded once for all the tokens routed to the
  expert (a verify window), then each token is a few instructions;
- one fixed arithmetic for every ISA: exact int32 sums per Q8_0 sub-block,
  four float FMA chains, a fixed final sum. AVX2, AVX-512 and the scalar
  reference give the same bits, and a token gives the same bits alone or in a
  group of any size (no group-size rule needed, cf. #152).

native_gu_rows / native_down_rows take them for types 144-146 on AVX2 and
AVX-512 CPUs. STRATA_NO_TQ_KERNELS=1 restores ggml-cpu's vec_dot,
STRATA_TQ_ISA=scalar|avx2|avx512 caps the ISA. A CPU without AVX2 keeps
ggml-cpu.

tq_cpu_parity (CPU only, ctest tq_cpu_parity and tq_cpu_parity_avx2):
AVX2 / AVX-512 vs the scalar kernel bit for bit for groups of 1-8 tokens,
vs ggml-cpu's vec_dot (rel ~1e-7, float order) and vs to_float in double
(rel ~1e-5, the int16 table), plain and Hadamard-rotated, for Gyro-S
(tqk6/tqk7) and Gyro-M (tq2_t) experts through the engine entry points.
`tq_cpu_parity --bench` is the micro-benchmark (clock-calibrated cycles).

Zen 5, one thread, weights streamed from DRAM, core cycles per 128-weight
block (one token / per token in a group of three):
  ggml-cpu vec_dot, AVX2 build:     108-117 / 108-116
  ggml-cpu vec_dot, AVX-512 build:   64-66  /  61-62
  Strata AVX2:                       47-49  /  22-23
  Strata AVX-512:                    47-49  /  22
  (Q2_0's AVX2 kernel: ~20; the repacked AVX-512 Q2_0 kernel: ~7)

Ported from agentionai/Strata rc2 7c85f1d (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:35 +02:00
Laurent 25a1f6e046 APR on 0.1.40: the sub-16 gate/up rows assume 80 dots per row; trellis types (qk 128) take the full-warp kernel
v0.1.40's row_dot_80_sub16 covers (n_embd / qk) * ipb == 80, true for the 256-value i-quants but not for
TQ2_T / TQK6 / TQK7 (160 dots at n_embd 2560): half of every gate/up row was dropped (native_expert_parity
gpu rel 0.87, garbage output). Restores gpu rel 2.13e-2 / decode-once bitwise.

Ported from agentionai/Strata rc2 c4ba288 (rc2 = 9ff02c9), author Laurent.
2026-10-15 21:31:26 +02:00
Laurent Zuijdwijk 84507d7179 hadamard_rows: say what is wrong with a bad block before exiting
hadamard_rows / hadamard_rows_f16 still exit on an invalid block, like
check() does on a failed launch, but now through one helper whose message
names the entry point, the block, the row width and the requirement (a power
of two in [2, 1024] dividing the width), and where the mismatch comes from.

Ported from agentionai/Strata rc2 48ff8b9 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:26 +02:00
Laurent Zuijdwijk 86faf035b6 prefill: say why the FP16 path rotates H in place in fp16
Not changed to an fp32 rotation: hadamard_rows_f16 already does its
arithmetic in fp32 and rounds once on the store, so the only extra error is
the fp16 rounding of the SwiGLU output before the rotation, while an fp32
copy would cost a T*K*640 float buffer or a fused kernel on the non-MMQ
fallback. The comment states this, the precision cost and the parity bound
(relative L1 < 2e-3 against the float rotation); the gate/up input refers to it.

Ported from agentionai/Strata rc2 50a1972 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:26 +02:00
Laurent Zuijdwijk 4557f68053 iq_pack: refuse to mark a native_experts.txt other than v3/v4 as v5
The v5 rewrite for Hadamard-folded experts was a re.sub on "v[34]": on any
other header it changed nothing and still wrote hadamard.txt, an
inconsistent pack the engine refuses at load. native_experts_v5() now
rewrites v3 and v4 only and otherwise returns an error naming the version
found and the supported ones; main() prints it and exits 1 before any file
of the pack is written. Tests: both rewrites, the refusal for v2/v5/v6/v10
and a missing header, and a full pack run that must leave no
native_experts.txt or hadamard.txt behind.

Ported from agentionai/Strata rc2 235787c (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:26 +02:00
Laurent Zuijdwijk 8fd8e6da3c prefill: resolve the APR sign vectors once per device, not per layer and chunk
run_impl called hadamard_device_signs for every folded layer of every chunk,
which takes a process-wide mutex and may cudaMalloc. Prefill::init now
resolves the per-layer gate/up and down sign pointers on its device into
Impl::had_sx / had_sh, as peer_had_signs does for the peer and the layer
split's helper (P.had_sh), and run_impl only reads them. carve no longer
warms the device table on every relayout (init's lookup does that now, once);
the comments on peer_had_signs and the new members say what is resolved when.

Ported from agentionai/Strata rc2 3d191fa (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:26 +02:00
Laurent Zuijdwijk dd32704d4d prefill: look up the APR sign vectors at the layout's widths, not 640 / N
run_impl asked hadamard_device_signs for the down projection's signs at a
literal 640 (and gate/up's at N), while carve and the peer path resolve them
at lay.fmt[l].n_ff / n_embd. The widths are part of the device copy's key, so
the two only shared an entry because the widths happen to match. All lookups
now use the layout's widths.

The other 640s next to the rotations are the row width of the prompt path's
expert buffers and products (swiglu, quantize, GEMM), fixed for this model
like N; Prefill::init now refuses a native layout whose experts are not
N x 640 instead of assuming it. No other sign or rotation width in the tree
is hardcoded (verify, peer and remote experts use the geometry).

Ported from agentionai/Strata rc2 ff463c8 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk 390da5d905 Trellis numerics: say which codebook each path reads and what is bitwise
Decode reads the int8 codebook tq_lut_i8 (one global scale, 0.85% relative
RMS); the FP16 prompt path reads the fp16 codebook and the MMQ path is the
GGML fork's tile loader. The comment over tq_load / tq_apply now says that
only the decode-once and per-entry kernels are bitwise equal to each other,
and that native_expert_parity bounds the decode path against ggml's float
dequantizer at relative L1 error < 3e-2 (the prompt dequantizers at < 1e-6).
The same goes into a short "Numerics" section of docs/GYRO.md and the parity
test's header.

Ported from agentionai/Strata rc2 457047e (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk 99581ed5e3 verify: carve the rotated expert input only for Hadamard-folded packs
nat_xr_ (T x N floats) was taken from the verify arena for every pack, though
only a layer whose gate/up is Hadamard-folded reads it. It is now carved only
when some layer of the native layout has had_gu, as the prompt path does with
its rotated copy; the one reader is under that layer's f.had_gu, so it never
sees the nullptr.

Ported from agentionai/Strata rc2 687b7e6 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk 294c95ebbc APR packs: drop the STRATA_APR opt-in; warn only on unvalidated setups
The GPU path for Hadamard-folded experts is now validated end to end on
CUDA sm_86, sm_89 (also a two-GPU layer split and peer decode) and sm_120,
and on HIP gfx1151, so `strata generate` no longer exits without
STRATA_APR=1 (the variable is ignored if set). It logs a warning instead for
the two setups that have not been run end to end yet: a HIP GPU other than
gfx1151, and the P2P prompt peer (Prefill::set_peer).

docs/GYRO.md and tools/gyro_setup.sh no longer set STRATA_APR=1; the setup
script's AMD note now says what was validated on gfx1151.

Ported from agentionai/Strata rc2 0fabd20 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk 7709a0e2c8 docs/GYRO.md: read as part of Strata (no fork clone step); render-group note for HIP containers
Ported from agentionai/Strata rc2 f9a40f8 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk cce9b16a72 docs/GYRO.md: RTX 30-series setup line, RTX 4090 validation and the two-GPU notes
Ported from agentionai/Strata rc2 98125c0 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent 926171c779 docs/GYRO.md: 2026-10-03 results (5090 #2, Gyro-M, 3090, AMD smoke); fix Gyro-S size
Gyro-S is 58.5 GB (54.5 GiB). Gyro-M and AMD have now run end to end;
long-prompt prefill is measured. The measured section now has the int8
decode numbers (rc1 includes them and --prefill auto), the server
protocol, Gyro-M and the 24 GB RTX 3090; the cache-size rows stay,
labelled as the 2026-10-02 build.

Ported from agentionai/Strata rc2 c494121 (rc2 = 9ff02c9), author Laurent.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk f5b7628a7f APR port: Split<144..146>::load takes upstream's STAGE_GRID form
Upstream 0.1.39 calls the decode-once loaders as
S::template load<STAGE_GRID>(row, kbx, iqs, s_grid) (the IQ types can stage
their grid table in shared memory). The trellis loader has no grid to stage;
it takes the template parameter and the table pointer and ignores them.
Found by the first CUDA build of rc1 (CUDA 12.4, sm_89).

Ported from agentionai/Strata rc2 07e9afd (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:20 +02:00
Laurent Zuijdwijk 28651a4079 APR port: multi-GPU prompt offload (peer, layer split helper) for Hadamard-folded packs
set_peer and set_stage_helper no longer decline an APR pack. Their MMQ
share now takes the same rotations as the primary's MMQ path:

- gate/up input: the primary sends its rotated copy (m.Xr = H (s * x), made
  once for its own MMQ quantize) instead of `mixed` to the peer / helper
  card, over P2P or the mapped host route. One rotation serves both cards,
  bit for bit the same input; `mixed` still feeds the shared expert.
- down input: on the peer / helper card, hadamard_rows on H between
  mmq::swiglu and mmq::quantize, in the compact (per group) and the
  layer-sized buffers, with that card's own copy of the layer's sign
  vector (PeerPrefill::had_sh, made at setup on the peer's device).
- The fused int8 prompt kernels never take a folded layer (fused_nat), and
  bind_stage_helper already refuses layers on the fused kernels, so the
  helper's folded layers are always on MMQ with the separate
  swiglu -> rotate -> quantize passes.
- The trellis types need nothing more there: the peer gathers the blobs
  with mmq::gather_native and runs the same MMQ instances (STRATA_MMQ_TQ).

Not run on a GPU yet. Needs a 2-GPU box: Gyro-S with --peer-device (and
separately a layer split with STRATA_PREFILL_HELP=1), the first token's
logits and a greedy generation against the single-GPU run.

Ported from agentionai/Strata rc2 c64e1ec (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:19 +02:00
Laurent Zuijdwijk 72f21aca93 docs/GYRO.md: clone the rc1 branch
Ported from agentionai/Strata rc2 87fc901 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:04 +02:00
Laurent b4e477c014 APR trellis decode: int8 codebook + dp4a in tq_load / tq_apply
Port of the agention llama.cpp CUDA change (perf/tq-cuda-decode 0005): the 2048-point trellis
codebook as int8 pairs with one global scale (max |point| 3.984 -> 127; relative RMS error 0.85%
of the codebook std). A trellis step's 4 weights are two 2-byte entries = one int, so a decoded
lane (TqLane) is 4 ints instead of 16 floats (5 registers instead of 17) and tq_apply is 4 dp4a
instead of 16 int8->float conversions + 16 FMAs. load/apply stay split: decode-once and per-entry
kernels remain bitwise equal. The dequantizers (prompt path, embedding) keep the fp16 codebook.

native_expert_parity --synthetic tqk6/tqk7 tq2_t/tq2_t and --synthetic-hadamard: 0 failures (RTX
3090, CUDA 12.4); GPU rel vs reference 1.18e-2 -> 2.13e-2 (tqk6/tqk7), 2.20e-2 (tq2_t), within the
3e-2 bound; gpu decode-once vs per-entry: bitwise equal.

Ported from agentionai/Strata rc2 87bad56 (rc2 = 9ff02c9), author Laurent.
2026-10-15 21:31:04 +02:00
Laurent 81aef750d2 gyro_setup.sh: --prefill auto, like setup.py writes for the GSQ-RCO models
The Gyro config used --prefill 512, so the prompt path borrowed only 364 cache slots. Measured on
an RTX 3090 (24 GB, cache auto = 15,756 slots, ctx 131k), same prompts: 4K-token prompt 767 ->
1,325 tok/s, 16K 817 -> 1,870 tok/s, on par with ISTA IQ2_XS under setup.py's config (1,401 /
2,058); decode unchanged within noise.

Ported from agentionai/Strata rc2 a4d70dd (rc2 = 9ff02c9), author Laurent.
2026-10-15 21:31:04 +02:00
Laurent Zuijdwijk 75d9b260da gyro_setup.sh: export STRATA_GGUF_PY for every step (mtp_pack.py needs it too)
Ported from agentionai/Strata rc2 0d0f4b0 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:04 +02:00
Laurent Zuijdwijk 2fce1e1e61 gyro_setup.sh: point the HIP build at ROCm (ROCM_PATH or /opt/rocm)
Ported from agentionai/Strata rc2 73c3106 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk e05565e223 gyro_setup.sh: set up the venv first and build with its cmake/ninja (distro cmake can be < 3.24)
Ported from agentionai/Strata rc2 9a89f63 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk 21aa723a2a gyro_setup.sh: skip the download when the model file is already there
Ported from agentionai/Strata rc2 ffad2f0 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk ffbb367b8a docs/GYRO.md: clone from the agentionai fork
Ported from agentionai/Strata rc2 0802c90 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk 5f48f323da docs/GYRO.md: drop the agent-client note
Ported from agentionai/Strata rc2 e466545 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk 0ca3752ea6 Gyro: setup guide and script for the published agentionai Gyro-S/Gyro-M files
docs/GYRO.md: build against the agentionai llama.cpp fork, pack the published GGUF unchanged,
Strata's MTP runtime, server config, RTX 5090 measurements (full and 16/8 GB-sized expert caches)
and known limits. tools/gyro_setup.sh does every step on Linux (CUDA or HIP).

Ported from agentionai/Strata rc2 759675e (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk f4abf43154 APR port: count the prompt path's rotated-input buffer in moe_set_bytes
carve() takes m.Xr (T x 2560 floats, the Hadamard-rotated copy of `mixed` for
the MMQ path) from the MoE set of the shared region, but moe_set_bytes, which
sizes that region (and bytes_needed / the borrowed slots), did not count it.
With an APR pack and MMQ on, the MoE set is the region's largest, so its last
take overflowed and init failed with "device buffers for a chunk of N tokens
do not fit" at every chunk and cache size, borrowed or own buffers alike
(reported on an RTX 5090 with the published Gyro-S). Workaround on an older
build: STRATA_PREFILL_MMQ=0 (the FP16 prompt path takes no Xr).

Ported from agentionai/Strata rc2 09042dd (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:31:03 +02:00
Laurent Zuijdwijk 62bb21a2d7 APR port: run the published Gyro files unchanged (Q6_K embedding, Q8_0 PLE table)
- iq_kernels.cu: dq_q6_k (llama.cpp's dequantize_block_q6_K, two halves per
  thread) in dq_dispatch; embed_type_supported and iq_row_bytes take Q6_K
  (embedding only) - Gyro-S's token_embd.
- ngram.cpp: PleTable reads Q8_0 rows (170 B; PLE_ROW_BYTES_MAX raised), mapped
  and direct SSD I/O - Gyro-M's per_layer_token_embd.
- native_expert_parity: --gguf-embd (random token_embd rows via iq_embed_rows
  vs ggml to_float), --gguf-ple (random PLE rows via PleTable, both I/O modes),
  --load-check (the engine's start-up checks for a pack + GGUF, no GPU load).

gfx1151: Gyro-S token_embd (Q6_K) and Gyro-M token_embd (Q8_0): 300 rows,
0 values differ; Gyro-M PLE (Q8_0) and Gyro-S PLE (IQ4_NL): 320 rows, 0
differ, mmap and direct. --load-check on packs of both published files:
0 failures. (Measured before the rebase onto 0.1.39.)

Ported from agentionai/Strata rc2 cc287a6 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:30:56 +02:00
Laurent Zuijdwijk f7f21bc823 APR port: decode-once verify kernels and MMQ prompt path for TQ types
- iq_kernels.cu: the TQ lane dot split into tq_load (16 codebook values) and
  tq_apply; vec_dot_tq_q8_1 = apply(load), Split<144..146> + kSplit, so
  verify windows decode each weight lane once for all entries (bitwise
  equal to the per-entry kernels).
- MMQ: with STRATA_GGML_DIR = the agention fork, the fork's
  mmq-instance-{tq2_t,tqk6,tqk7} are built and moe_mmq takes the types
  (STRATA_MMQ_TQ). prefill.cpp rotates a copy of `mixed` (Xr) before
  mmq::quantize and H between swiglu and quantize; device sign vectors are
  made at prefill setup.
- hip_prefill_mmq_parity: TQ gate/up (1280x2560) and down (2560x640, odd
  blocks per row) passes; MMQ_PARITY_REPEAT; the output sentinel is now set
  with hipMemsetAsync on the test stream (a null-stream hipMemset could land
  after the MMQ kernel and make every output look unwritten).

- Rebased onto upstream 0.1.39: the fused int8 prompt kernels (STRATA_PF_FUSED)
  never take a folded layer. The multi-GPU prompt paths (set_peer,
  set_stage_helper) decline a Hadamard-folded pack here; a later commit
  gives their MMQ share the rotations.

gfx1151 (ROCm 7.2.1, STRATA_PREFILL_MMQ=ON): MMQ parity passed 3 runs x 10
repeats (rel_l2 1.4e-3..3.2e-3); native_expert_parity --synthetic and
--synthetic-hadamard tqk6/tqk7 tq2_t/tq2_t: 0 failures, decode-once bitwise
equal. CUDA 12.8 sm_120 full build compiled (incl. TQ MMQ instances) before the
rebase onto 0.1.39.

Ported from agentionai/Strata rc2 dc19786 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:30:50 +02:00
Laurent Zuijdwijk bda227817e APR port: TQ row bytes on the GPU (iq_row_bytes returned 0 for 144-146)
native_expert_layout takes gu_row / d_row / up_off / down_off from
iq_row_bytes, which had no cases for the trellis types and returned 0:
every gate, up and down row read row 0 of the blob, and up/down read the
gate matrix. The dequantizers (flat superblock indexing) were unaffected,
which is why they matched ggml bit for bit while the expert matmul did not.
native_expert_supported now also refuses a type without row bytes.

gfx1151 (ROCm 7.2.1): native_expert_parity --synthetic and
--synthetic-hadamard tqk6/tqk7 tq2_t/tq2_t: 0 failures (gpu rel = cpu rel,
cpu-gpu 5e-5..1e-3).

Ported from agentionai/Strata rc2 dae1f66 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:29:42 +02:00
Laurent Zuijdwijk fbf045eb6a APR port, stage 4: trellis decode and Hadamard rotation on the GPU paths
- third_party/ggml/ggml-common-tq.h: the fork's TQ2_T/TQK6/TQK7 block
  structs and 2048-point codebook (device table).
- iq_kernels.cu: Fmt<144..146> (the fork's tq.cuh lane decode against q8_1,
  4 q8_1 blocks per 128-block), dq_tq in dq_dispatch (prompt FP16 path,
  embedding), is_iq / row bytes; hadamard_rows / hadamard_rows_f16
  (shared-memory FWHT in the CPU's butterfly order) and
  hadamard_device_signs; native_expert_grouped rotates h when
  NativeExpertLayout::had_block is set.
- verify.cpp (local GPU), remote_experts.cpp (helper GPUs), prefill.cpp
  (FP16 fallback; folded layers never take MMQ) rotate x / h into their
  own buffers; the shared expert and the CPU doorbell copy keep x.
- native_expert_parity --synthetic-hadamard; ctests for tqk6/tqk7 and
  tq2_t when ggml is the agention fork.
- generate.cpp: APR packs run only with STRATA_APR=1 until validated end
  to end on a GPU.
- hip_backend.cmake: gfx1151 as an unvalidated architecture.

Rebased onto upstream 0.1.39: native_expert_grouped takes the separate
SwiGLU / rotate / q8_1 passes when the down weights are folded (the one-pass
swiglu_q8_1 kernel has no h to rotate); the new decode peer GPU
(peer_experts.cpp) rotates its private input copy and passes the down signs,
as remote_experts.cpp does.

Ported from agentionai/Strata rc2 6c7abde (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:29:38 +02:00
Laurent Zuijdwijk f47b0385f3 APR port, stage 3: Hadamard-folded experts on the CPU path
- hadamard.{hpp,cpp}: hadamard.txt parser, process-wide spec, in-place
  signs + normalized FWHT per block.
- expert_layout_load reads native_experts.txt v5 + hadamard.txt and sets
  NativeFmt::{had_block, had_gu, had_d, gu_signs, d_signs}; inconsistent
  packs are refused.
- native_quant_act / native_quant_h rotate before quantizing, which covers
  every CPU consumer (expert_source's doorbell copy, the pool's h).
- generate.cpp refuses Hadamard packs until the GPU paths rotate too.
- native_hadamard_test: Strata's CPU expert vs the fork's ggml graph
  (ggml_mul(signs) + llama_mul_mat_hadamard + ggml_mul_mat): bit-identical
  on real Gyro-S (TQK6/TQK7) and Gyro-M (TQ2_T) experts, also through
  ExpertPool::run_split_multi_native; synthetic Q8_0 reproduces the
  unrotated float expert to 1.2%.

Built with -DSTRATA_GGML_DIR=<agention llama.cpp> (ggml-cpu supplies the
trellis vec_dot kernels).

Ported from agentionai/Strata rc2 1587897 (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:28:58 +02:00
Laurent Zuijdwijk 89febede8b APR port, stages 1-2: map, trellis type tables, prism.hadamard packing
- Type tables: PTQ1_0/TQ2_T/TQK6/TQK7 (143-146, 128-value blocks) in the C++
  and Python GGUF readers.
- iq_pack.py: validates prism.hadamard.* and writes hadamard.txt (block,
  sign vectors, folded roles per layer); native_experts.txt becomes v5 for
  such models so an engine that cannot rotate refuses the pack. Anything
  it cannot apply (dense folded weights, inverse tables, gate without up,
  Q2_0 experts, missing sign widths) is refused. 3 new unit tests.
- embd_bf16_pack.py --gguf: a GGUF's token_embd dequantized to BF16, for
  embedding types without a GPU dequantizer (Gyro-S: Q6_K).

Ported from agentionai/Strata rc2 d864b6a (rc2 = 9ff02c9), author Laurent Zuijdwijk.
2026-10-15 21:28:47 +02:00
Niko1221 31556cd1a0 helpers wait for their turn visibly; batch-mtp policy off; helper cap max(slots,4) 2026-10-15 21:25:37 +02:00
Niko1221 322e3fdfb3 Terminal: a failed runtime start names the engine log; doctor warns when the render group is missing
- runtime start/restart/verify that fail print the run config's engine log path and its last lines (the line was only 'server_exited')
- doctor: on Linux with a SYCL or HIP run config, a /dev/dri/renderD* (and /dev/kfd) the user cannot open is GPU_DEVICE_ACCESS (warning) naming the render group;
  seen on P1006 where the engine only said 'No device of requested type available'
2026-10-15 21:14:37 +02:00
Niko1221 8401c1cdb5 stage-harness: simpler prune of files 2026-10-15 21:14:08 +02:00
Niko1221 486555d4f1 scripts: agent-flow-s4b (helpers, one-card edit, web call, Stop) for a real model 2026-10-15 21:13:23 +02:00
Niko1221 3152013186 agent README: tools, helper agents, /goal, plain words, Manual gate 2026-10-15 21:12:33 +02:00
Niko1221 1e73f53160 host: tool servers (MCP) set up for the model are connected to the agent, secrets only through the environment 2026-10-15 21:09:32 +02:00
Niko1221 b9c6c65482 engine component: a HIP package is not chosen for a card it has no code for
P2006 (Radeon 780M, gfx1103) resolved the gfx1200/gfx1201 package as usable: HIP detection looked at the PCI vendor only.
The Linux kfd topology (gfx_target_version) names each card's gfx target; a package whose archs list none of them is skipped with the reason.
Unknown targets (Windows, no kfd) are not a refusal, as with CUDA.
2026-10-15 21:08:57 +02:00
Niko1221 37cab2c76f tests: /goal over HTTP, plain unavailable notice 2026-10-15 21:07:26 +02:00
Niko1221 14d0ec7ac8 tests: request slots default, sizing, retry with one 2026-10-15 21:05:55 +02:00
Niko1221 5a00c8fb64 host: agent tools, helper agents, one approval card in Manual, /goal, plain agent states, request-slot sizing
- strata-gate plugin: Manual asks before a change (no blocked attempt first); children of a Manual chat reach the same card
- preset.mjs: the Harness standard preset minus scheduler, search without a key, unanswerable question tool, plan mode
- helper agents projected into the existing agent rows; Stop stops the helpers; the run ends when they have reported
- /goal through the Harness goal Remotes; goal rounds stream into the chat
- native-parallel.mjs: request slots by default in the installed app (fit policy, one-line switch), batch-mtp, one-slot retry
- plain wording for agent failures and tool errors, original text kept as details
- stage-harness.mjs: pruned production Harness for the installer (not run yet)
2026-10-15 21:05:15 +02:00
mes b884451697 Layer split auto: STRATA_SPLIT_UNIFORM=1 restores the uniform miss cost
0.1.41 scaled each stage's miss term by its probed link (0391e478);
0.1.44 scales from a 4x gap against the fastest link (9d41e18b, #2007).
On an x16+x4 dual-3090 rig (26.9 vs 6.4 GB/s, 4.2x) either rule moves
auto off the balanced placement (K=27 -> K=19/K=31): decode -4..-8%,
prefill +15..+19% at 10K / -6..-9% at 90K vs 0.1.40.x at matched
calibration. Opt-in restore next to STRATA_SPLIT_LINK_ABS, default off.
2026-10-15 20:33:15 +02:00
Niko1221 1696b5c959 installer: header text, branding line, no details pane 2026-10-15 20:29:43 +02:00
Niko1221 0db5d17102 chat speed is decode speed with reasoning, context shown is the running model's, Models shows Unload while loaded, VRAM reserve floor, default text size 14, installer branding, icon 2026-10-15 20:25:35 +02:00
Niko1221 85c7bfda29 Terminal archive: calibrate refuses in words, models import works, tuner runs a packaged engine and boolean flags
Found qualifying the Terminal archive on real boxes:
- engine calibrate crashed with ModuleNotFoundError (tools.calibrate / setup are not in the archive): refuse with exit 5 naming STRATA_INSTALL_DIR and tune run --mode api; --plan reports tool_available
- models import --plan failed with "fingerprint catalog is unavailable": data/gguf_fingerprints.json is now in the archive
- tune plan/run --engine <package exe> could not load the package's libraries: engine_env puts the package folder first; --engine component:strata-engine accepted
- a boolean candidate (--kv-grow) was passed as "--kv-grow 0" ("unknown argument: 0"), and the protocol harness turned "0" into a bare flag (on): flag_argv
- an unmeasured candidate's reason showed the engine usage dump; the reason line comes first now
2026-10-15 20:13:16 +02:00
Niko1221 42ffb27951 stable window origin (fixed port with fallback), run configs repaired when Control moves, app installs to Programs\Strata and migrates an old install out of the data folder 2026-10-15 20:10:06 +02:00
Niko1221 bfe0798660 install progress step as a sentence; Model files label; ui-flows details check 2026-10-15 19:51:52 +02:00
Niko1221 5df75b1d4d first-run feedback 2: plain words for failures with Details, parts-based download progress, reading size reaches all conversation text, tuning pop-up fits its error, headless UI flows, installer mock-ups
ErrorNotice and toasts map known failures (server_exited, engine missing, network, integrity, Harness missing, Control) to a sentence and a next step with the original text under Details. The install progress sums a list of parts (model files, engine) into one count with a time left. The Read pictures block is hidden when the catalog has no pictures. Control reports the last server log lines of a failed start. Reading size scales headings, code, tables and reasoning. All eight locales.
2026-10-15 19:42:08 +02:00
Niko1221 844f91c606 app: every updater message in plain words, technical reasons in details 2026-10-15 19:23:38 +02:00
Niko1221 36c6ff200e host: the model install job also gets the engine for this PC (engine-step), one combined progress total, retry message on failure 2026-10-15 19:21:21 +02:00
Niko1221 7b2082dbb0 first-run feedback: combined download progress with an ETA, catalog without pictures for app installs, RAM budget for the 20% profile, human engine-start errors, installed model preferred in Models, system theme default, NSIS ignores a remembered temp install folder 2026-10-15 19:10:26 +02:00
Niko1221 4a1ef86b0a layer split auto: STRATA_SPLIT_SPEED_SCALE scales each stage's speed, a test seam for an unequal pair on equal cards (#2644) 2026-10-15 18:50:06 +02:00
Niko1221 8a9d724e6e host: look for the Harness in the user's component folder too, not only when STRATA_COMPONENTS_DIR is set 2026-10-15 17:53:22 +02:00
Niko1221 1b6ed0898d desktop: first-run-ui.mjs, headless Chrome on the packaged host's onboarding with the real catalog 2026-10-15 17:50:11 +02:00
Niko1221 0c37d0e285 update --check: do not print a note twice 2026-10-15 17:49:49 +02:00
Niko1221 2bd0cdc4f9 installer: home() instead of a constant (callers can still move ROOT); close cancels a running install; tests for the app's install mode 2026-10-15 17:46:28 +02:00
Niko1221 31340f7279 docs: STRATA_FILE_CACHE_OLD_RULE (#1752) 2026-10-15 17:45:17 +02:00
Niko1221andKevinX8 1d6fd9f97a file tier: the old room rule on the gfx906 backend (#1752), STRATA_FILE_CACHE_OLD_RULE=1/0 on any Linux build
On an MI50 ROCm queues are suspended while the kernel reclaims around the HMM-tracked arena; the cached
reads of the new rule churn the page cache.  The gfx906 backend keeps the old room >= read_bytes rule;
other HIP cards keep the current rule (the PR applied it to every HIP build).

Co-authored-by: KevinX8 <[email protected]>
2026-10-15 17:45:01 +02:00
Niko1221 d949d600f8 updates: shell bridge tests, host test that a start never reaches the update step, install hint 2026-10-15 17:40:14 +02:00
Niko1221 5a8985620c start path: a signed package is verified with the keys this build trusts; engine-missing notices name Update engine and strata update 2026-10-15 17:38:20 +02:00
Niko1221 10a326a90c packaging: UPDATES.md, feed_tool url prefix and private key file, tests 2026-10-15 17:37:03 +02:00
Niko1221 5322caafe0 Merge branch 'f-split' into f-split-fr 2026-10-15 17:36:37 +02:00
Niko1221 11bf85d8a8 host: the packaged app gets the model catalog, the GPUs and the install job from Control's bundled installer
native-control runs Control's tools/strata_mcp.py on Control's Python (installerEnvironment: STRATA_NO_ACQUIRE stays for the
process; only the install job the user starts may fetch model files), in a writable per-user install folder. The catalog,
hardware (nvidia-smi) and recommendation come from it, the install plan and job go through it, and a failed catalog is an
empty one. Calibration stays off in the packaged app. packaged-control tests adjusted and extended.
2026-10-15 17:36:32 +02:00
Niko1221 ec42c93947 installer: setup.py and strata_mcp.py run under Control (STRATA_CONTROL_INSTALL) for the desktop app
In the app's mode the install downloads the model the user chose, prepares it with the bundled tools, installs no Python
package, fetches no engine and compiles nothing, and writes a run config naming component:strata-engine into a per-user
folder (STRATA_INSTALL_DIR). Only bundled tools run (run() refuses pip, git, cmake and anything else). The engine
component is looked up if present; a model can be downloaded before an engine is installed.
stage-control.py adds setup.py, the preparation tools (static import closure), ref/load.py and data/ to the staged Control.
2026-10-15 17:36:32 +02:00
Niko1221 5778a9ef17 update: Control bundles may carry relative links inside themselves (packer and unpacker); tests 2026-10-15 17:35:22 +02:00
Niko1221 4868f3234d updater: require the one-token probe (force the load of a lazy server), keep the test start's engine log, report what was installed when a step fails 2026-10-15 17:33:47 +02:00
Niko1221 d3c4bba4ca host: Settings update check, Update engine/Strata and rollback through Control's explicit update step; shell bridge client; updated Control is found 2026-10-15 17:28:33 +02:00
Niko1221 8ad2a62730 Merge f-split-life: engine always ends on close (app and Terminal), single app instance, Bypass is full access 2026-10-15 17:28:03 +02:00
Niko1221 e3de5044b2 procs: init adopting an orphan does not make init its starter (found on Linux: the orphan check found nothing) 2026-10-15 17:27:15 +02:00
Niko1221 13cd63567e tests: first-run update test ignores the machine's own installed models 2026-10-15 17:19:40 +02:00
Niko1221 1aa306d7ea update: the last check's result is shown as a one-line notice by components engine (no network) 2026-10-15 17:16:33 +02:00
Niko1221 912d7c1673 terminal: strata update, strata components update and rollback 2026-10-15 17:15:35 +02:00
Niko1221 bc9058a8d4 host UI: the first screen no longer crashes on an empty or failed model catalog
GuidedSetup read catalog.recommendation.family without checking that a catalog has a
recommendation; the packaged host's catalog had none and the app showed 'couldn't show this
screen'. GuidedSetup and NativeModels now tolerate no catalog, no recommendation, no families
and families without sizes. A test renders both components with each of those shapes.
2026-10-15 17:14:01 +02:00
Niko1221 c8698f6257 updates: signed feed, Ed25519 verifier, component updater with resume, staged test-start, switch and rollback 2026-10-15 17:13:41 +02:00
Niko1221 5ec49d7adf test_procs: the orphan's stdio is not the test's pipe 2026-10-15 17:10:14 +02:00
Niko1221 95654fe073 matrix: drain the pty a TUI paints to, give it a size, and tolerate an already-reaped child 2026-10-15 17:09:15 +02:00
Niko1221 b386055d52 prefill_fused_iq_test: the Q2_0 / Q2_0 pair is skipped where the device does not set it up (it is the Turing opt-in's pair only) 2026-10-15 17:08:13 +02:00
Niko1221 921668da66 installed-flow: a killed shell takes the host and its engine with it 2026-10-15 16:49:47 +02:00
Niko1221 7d0b3e7090 tests: a real ssh disconnect ends the server and its engine on a Linux box 2026-10-15 16:44:12 +02:00
Niko1221 b11004671f app matrix: a Terminal-started engine survives the app's close 2026-10-15 16:43:07 +02:00
Niko1221 1d26952f72 serve: the engine's clean-stop deadlines are named, and a test shows an engine that ignores STOP, QUIT and SIGTERM is killed 2026-10-15 16:39:14 +02:00
Niko1221 871a242721 matrix: the TUI is quit with q (typed into its console or pty) and killed; the service it watches is untouched 2026-10-15 16:38:42 +02:00
Niko1221 e09733aa9d tuning: the engine and worker a tuning sweep starts end with the sweep, however it ends 2026-10-15 16:35:49 +02:00
Niko1221 d0e9cb3e37 docs: CLOSING.md, what ends the engine on each OS and how leftovers are found 2026-10-15 16:35:15 +02:00
Niko1221 896c66180e serve: a hangup that was ignored at start (nohup) stays ignored; the matrix proves it and resets inherited signal dispositions 2026-10-15 16:32:10 +02:00
Niko1221 14038ee30c serve: an explicit --pipeline-windows 2 on two stages says when the draft layer is not on the first card (#2637)
The default takes the pipelined decode on two stages only with the draft layer on the first card (mtp_first); asked
for by hand it was taken without a word, and without mtp_first the draft chain shares the last card with its
window (2x R9700: the pipeline lost to the serial loop before 0.1.44). Recommend, never force: it stays on as
asked and a note says what it costs, why mtp_first is not on (no --mtp / --spec 2+ / draft_vocab.bin, or
STRATA_MTP_STAGE0=1 to ask for it) and how to turn it off. No default output changes.

Reported by @dinesh-chander on #2637 (closed by the reporter).
2026-10-15 16:31:46 +02:00
Niko1221 3b848837fa layer split auto: a two-GPU rig whose first pick has a tiny stage tries each card's share of the room before the three-layer rule (#2644)
On RTX 4060 Ti 16 GB + RTX 3060 12 GB (48 layers, IQ3_S) the first pick was K=2 and the #1616 retry
(every stage at least 3 layers) landed on its own edge, K=3: the 3060 ran 3 layers, 12 GB idle, 452 tok/s of
prefill against 802 at 28 layers (the prompt flows through the stages as a pipeline). For two GPUs the retry
now first asks for a placement in which each stage owns at least floor(layers x its room / all rooms) layers
(split_rules::share_floors, at least 3, trimmed to fit); the three-layer rule stays the second try and the
first pick the last, and the log says which one was taken. Nothing changes where the first pick has no
one- or two-layer stage (equal cards), and three or more GPUs keep the #1616 rule alone.

Reported by @Tworks25 on #2644, with the sweep of layer_split 12..32 (PR #2643). split_rules_test gets the
reporter's rooms (14.2 GiB and 5.50 GiB) and the identical-card and degenerate cases.
2026-10-15 16:31:45 +02:00
Niko1221 f31f24c8dc matrix: STRATA_MATRIX_LOGS keeps the server and engine logs of each case 2026-10-15 16:29:47 +02:00
Niko1221 6b83f43dc9 matrix: the fake engine claims the family the OS supports (metal on macOS) 2026-10-15 16:24:34 +02:00
Niko1221 ec73f63f28 matrix: sample GPU memory while the engine runs, so 'freed' is measured against a loaded value 2026-10-15 16:21:05 +02:00
Niko1221 22fea7b767 desktop: close-path matrix for the installed app, self-test proves the shell is in the job, installed-flow checks a killed host takes its engine 2026-10-15 16:20:45 +02:00
Niko1221 e437bc472d matrix: graceful means the clean-stop line, the engine's marks only for the fake 2026-10-15 16:06:50 +02:00
Niko1221 6aa4663bf2 matrix: a real engine's log goes to the rig 2026-10-15 16:06:02 +02:00
Niko1221 651e64ff32 matrix: real-engine mode takes a component folder and waits long enough for a model load 2026-10-15 16:05:32 +02:00
Niko1221 322790aff0 app: one instance, and closing always ends the engines this app started
Tauri single-instance plugin: a second launch focuses the open window and starts nothing. On Windows the shell puts itself in a
kill-on-close Job Object, so killing it, a crash, logoff or shutdown ends the host, Control, the owner host, the server and the
engine. The host follows its parent (STRATA_EXIT_WITH_PARENT) and takes SIGHUP and SIGBREAK down the clean shutdown; every engine
it starts is tied to the host with runtime start --die-with-pid (drain 15 s, then ended); it runs runtime cleanup once at start.
The shell waits 60 s for the host's clean stop (the host's own deadline is 45 s) before ending it.
2026-10-15 16:04:43 +02:00
Niko1221 347d255eaa runtime start: check the engine package before any process exists; a changed file is refused with exit 5 and its name 2026-10-15 15:58:32 +02:00
Niko1221 48e6d64c4f serve, control: a server's engine ends with it however it ends, leftovers are found by PID, start time and command line, runtime stop/cleanup recover a lost owner
lifeline.popen ties every engine, image encoder, MCP server and sd-cli to the server that started them (kill-on-close job on Windows,
PR_SET_PDEATHSIG from one anchor thread on Linux, a pipe watcher elsewhere). Engines and encoders get their own process group so
Ctrl+C, a closing terminal and a console Break reach the server only, which stops the engine the clean way (STOP, QUIT, wait 20 s,
terminate 5 s, kill). SIGHUP, SIGBREAK and the Windows console close/logoff/shutdown events take Ctrl+C's path; the handlers are
installed before the model loads and the browser launch no longer holds the main thread (#1872). The owner guard escalates SIGTERM
to SIGKILL after a grace period. strata_control/procs.py records every launch and finds leftovers; the next engine start, runtime
start, runtime stop and the new runtime cleanup end them after re-checking identity. runtime start --die-with-pid for app-owned
instances. tests/terminal/lifecycle_matrix.py is the close-path matrix.
2026-10-15 15:58:04 +02:00
Niko1221 865b184a0f packaging: what each target's engine package holds 2026-10-15 15:36:59 +02:00
Niko1221 c5388c78cd tests: collect_libs arch filter, ELF check, system/driver split 2026-10-15 15:35:05 +02:00
Niko1221 1bd9746f53 collect_libs: name the libraries a dlopen'd library cannot find 2026-10-15 15:31:11 +02:00
Niko1221 f116778bd5 #1825 qsa_select: the thread-per-head prompt scorer is on for the Turing series (STRATA_SM75_PERF=1) or STRATA_SCORES_THREAD=1, not a default elsewhere
Tesla P100 (sm_60) prompt read, interleaved 5 pairs each, thread vs warp scorer: 30000 tokens 339.29 vs 335.45 tok/s (1.011) and 338.98 vs 330.20 (1.027), 60000 tokens 338.88 vs 330.15 (1.026), 5/5 pairs faster each, below the +3% bar for a new default. Same bits either way. docs line; qsa_scores_parity runs with the switch on.
2026-10-15 15:25:10 +02:00
Niko1221 7e469d797c tests: a schema-2 package made by package_engine.py starts a stub engine, and a flipped library byte refuses the start 2026-10-15 15:25:09 +02:00
Niko1221 b6443f3ae0 moe_fused_iq: #2145's Q2_0 gate/up fused coverage is the Turing opt-in's only (sm_80+ with STRATA_PF_FUSED=1 is bit-identical to base again)
The z440 A4000 Q2_0 gate with STRATA_PF_FUSED=1 on both arms found the residual dump different from base: gu_covered() took Q2_0 on every CUDA card and d.ok on sm_80+ now required the Q2_0 gate/up setup.
2026-10-15 15:21:02 +02:00
Niko1221 83426781d9 Revert #2223 (STRATA_NO_PCIE_GROUP=1): no gain measured on our boxes
Interleaved A/B, 6 pairs, --pcie-frac 0 vs --pcie-frac 0 + the switch, decode ALL: RTX 3060 IQ3_XXS 53.36 vs 53.16 tok/s (0.996, 3/6 pairs faster); RTX A4000 Q2_0 61.72 vs 62.17 (1.007, 4/6). The author's own +2.2% is below our +3% bar and does not reproduce on Ampere. Declined; the PR author's commit stays in the history only as this revert's parent.
2026-10-15 15:20:52 +02:00
Niko1221 dc1bc972ba metal: find the kernel sources, ggml-common.h and default tables beside the executable first, so an engine package can move away from the source tree 2026-10-15 15:18:53 +02:00
Niko1221 21da8442f5 collect_libs: dlopen extras are named by their soname, ELF files only 2026-10-15 15:17:00 +02:00
Niko1221 ac4770f2b1 host: Bypass mode is the Harness's full access with no prompts (danger-full-access + never) 2026-10-15 15:15:03 +02:00
Niko1221 d7895e1690 collect_libs: one copy per real library file 2026-10-15 15:13:43 +02:00
Niko1221 633a87577f collect_libs: --data-arch keeps only the wanted gfx kernels 2026-10-15 15:11:44 +02:00
Niko1221 2d784fa6ff packaging: collect_libs.py gathers an engine's shared libraries (ldd closure, dlopen extras, kernel data) for Linux packages 2026-10-15 15:07:14 +02:00
Niko1221 3e8b5161be engine component: optional variant (cuda13/cuda12 of one version), equal versions prefer the newer-driver build 2026-10-15 15:05:00 +02:00
Niko1221 8101a8903b serve: an engine option written as one entry of the config's args is split, a JSON slip is said with its place (#2267)
"--ple-io mmap" as ONE element of "args" (or "--ple-io=mmap") reached the engine as a single
argument, which it stops on ("unknown argument: --ple-io mmap"); the usual form is two entries.
The server now splits such an element into the flag and its value and says so at startup
(never one in a flag's value place). A config with a JSON mistake (a missing comma between two
args) gave a traceback; it now names the file, line and column. The slow-PLE-probe warning says
how the two entries are written in a config. --ple-io mmap, ram and direct themselves parse
correctly in the engine (checked on the 0.1.45 build).

Reported by lama7 on #2267.
2026-10-15 15:04:33 +02:00
Niko1221 480cfc5f3b engine component manifest schema 2: archs, min_driver, build_id, signature hook; start hashes every file; packager takes nested files 2026-10-15 15:04:15 +02:00
Niko1221 36381540f9 ple: gather_batch reuses prefetched keys that were reordered or duplicated (#2622)
PleTable::gather_batch matched the Direct-mode prefetch slots by position, so rows
prefetched as A..F and gathered as D,E,F,A,B,C (or A gathered as A,A,A) were read a
second time. Each requested token now looks its key up among the prefetch slots and
decodes from the matching slot into its own output position; duplicates share a slot.
All outstanding tickets are drained on a failed collect (the first error is kept)
instead of dropping the rest. The decoded bytes are unchanged.

Reported by @hotschmoe, who also supplied the fixture and the patch; the patch is
used as proposed. ple_reader_test --selftest gets the reorder / duplicate / subset /
missing / over-the-window cases, compared against the mapped reader.
2026-10-15 14:59:12 +02:00
Niko1221 4c968e6199 gdn chunk / Volta recurrence: name the kernels in every device pass (wmma body guarded inside the kernel), not behind __CUDA_ARCH__ at the call sites: the dea9b6de5 guard left the host stubs undefined in a 60;70;86 build (link error) 2026-10-15 14:53:40 +02:00
Niko1221 be885026f9 Merge f-split-s3: Control bundled in the installer, local engine components, no runtime acquisition 2026-10-15 14:52:46 +02:00
Niko1221 dc87d45eb3 app: chat cards drop Demo wording, finished tools say Ran unless approved, and the Mode choice travels with the turn
Approval label is Needs your approval on every card; example-card results lose the Demo prefix. A finished agent tool shows Approved only when the person approved it. Other Demo labels (simulated-model toasts, estimates, preview test, time-skip, context panel, tasks, serving) become truthful Example/Illustrative/Simulated wording in every locale.
2026-10-15 14:49:07 +02:00
Niko1221 2496301c60 host: chat modes (Manual, Plan, Bypass) as Harness permission presets, applied per turn
The page sends the chat's mode with each turn. The bridge switches the session to our preset with the Harness's own /permission command, verifies it through the permissions projection, and fails the turn closed if it cannot. Manual is read-only with ask (a change needs a one-shot approval), Plan is read-only with never (refused, no card), Bypass is workspace-write with never. Tests run the three modes against the real Harness with the scripted worker.
2026-10-15 14:49:07 +02:00
Niko1221 24d28c3f41 packaging: document the Control component, the engine component folder and the no-acquire line 2026-10-15 14:46:06 +02:00
Niko1221 6aa3b99816 tests: the old-engine hint check accepts update.sh as well as UPDATE.bat (found running the suite on Linux) 2026-10-15 14:45:49 +02:00
Niko1221 e1ee39169d sm_60: IQ3 table staging is opt-in (STRATA_IQ_STAGE_TABLES=1), not the default on cc 6.0: no end-to-end gain on a Tesla P100 (#1424) 2026-10-15 14:33:21 +02:00
Niko1221 9546c3af26 fetch-node.sh: mktemp with an X template (GNU mktemp on Linux refused -t); installed-flow.mjs gets a real-engine mode 2026-10-15 14:33:00 +02:00
Niko1221 64d8488c67 serve: a component engine's folder goes first on the library path; with_component_engine helper + test 2026-10-15 14:30:02 +02:00
Niko1221 aecee58bdd host: Control runs without STRATA_API_KEY (an empty one is refused by the server); installed-flow.mjs drives the installed app with a stub engine 2026-10-15 14:28:32 +02:00
Niko1221 4736323d86 desktop: ship the Control component (bundled CPython + locked wheels) in the installer, set STRATA_NO_ACQUIRE and STRATA_CONTROL_DIR for the host, clean host stop through Control, self-test checks for Control and the no-fetch engine notice 2026-10-15 14:22:38 +02:00
Niko1221 2622762f89 host: Strata Control bridge (control-runtime, packaged native-control via owner runtime), acquisition.mjs gates the llama.cpp and cloudflared fetches, /api/control routes, acquire-spy tests 2026-10-15 14:18:13 +02:00
Niko1221 9ddebc1b11 control: run configs can name "component:strata-engine"; the server, doctor and calibration resolve it from the component folder; targets.py keeps the start path off the fetcher; start-through-owner-runtime tests with a fake engine package 2026-10-15 14:13:08 +02:00
Niko1221 dc859b26ff control: resolve the engine from a local component folder (no download path), strata components engine, package_engine.py; the component fetcher refuses under STRATA_NO_ACQUIRE 2026-10-15 14:05:36 +02:00
Niko1221 a7bb051154 docs: agent bridge overview 2026-10-15 13:55:47 +02:00
Niko1221 a51d311fb9 tests: pure agent bridge units (profile, environment scrub, event projection) 2026-10-15 13:55:30 +02:00
Niko1221 cfd45e0c4d host: answer the agent's own questions as unavailable so a turn never waits for a missing answerer 2026-10-15 13:54:08 +02:00
Niko1221 3df612f80a host: stop a leftover Harness from a dead host at the next start (PID record, command-line check) 2026-10-15 13:52:17 +02:00
Niko1221 2425a18705 scripts: agent-flow waits for visible output before Stop and compares tool cards across reload 2026-10-15 13:51:45 +02:00
Niko1221 178bde4ce6 scripts: agent-flow checks per-chat model and two chats at once 2026-10-15 13:49:09 +02:00
Niko1221 537333045a scripts: agent-flow drives the app through approve, Stop and reopen with screenshots 2026-10-15 13:46:03 +02:00
Niko1221 7f81445c02 ui: wire chat to agent frames, real approvals, per-chat Stop and per-chat model choice (no visual change) 2026-10-15 13:42:19 +02:00
Niko1221 a24e78f6f1 host: Stop during agent start cancels the start instead of leaking a turn 2026-10-15 13:42:18 +02:00
Niko1221 fad1ce5568 host: route agent chats through the Harness (stream, approval, stop, status) with a visible fallback 2026-10-15 13:36:58 +02:00
Niko1221 e4da566272 host: internal accessors for the agent bridge (connections with keys, ready instances) 2026-10-15 13:36:58 +02:00
Niko1221 597039b6cb host: agent bridge (sessions, approvals, Stop, resume) with scripted-worker tests 2026-10-15 13:26:08 +02:00
Niko1221 3e373187ea host: project Harness session events into the chat frames 2026-10-15 13:26:08 +02:00
Niko1221 bf63faab2d host: Harness runtime launcher, Gateway client and Strata profile generator 2026-10-15 13:26:08 +02:00
Niko1221 9627ee00ce Merge f-split-desk (Tauri Windows build: node.exe entry path, self-test layout, pinned tauri CLI) 2026-10-15 12:48:43 +02:00
Niko1221 b73ea23b68 a started server and a managed runtime's server are told not to acquire (STRATA_NO_ACQUIRE=1)
setup.start passes it to the server it spawns (subprocess.call env, or execve for Docker); the runtime host sets it in
the owned server's environment. The test checks the start's child environment.
2026-10-15 12:33:40 +02:00
Niko1221 236cca71bb host: no /usr/bin/tar, /usr/bin/env or HOME/.local/state on Windows
- engine-install.mjs extracts with tarBinary(): System32\tar.exe on Windows (bsdtar reads .zip and .tar.*), /usr/bin/tar
  elsewhere.
- util.mjs which() asks System32\where.exe on Windows (first match), /usr/bin/env which elsewhere.
- the tuning and Time-skip state default to %LOCALAPPDATA%\Strata\tuning and \time-skip on Windows
  (apps/host/server/platform-paths.mjs; POSIX keeps $XDG_STATE_HOME/strata). The Python side resolves the same folders
  (strata_control/paths.py, used by the runtime directory and the tune and timeskip commands), so the host and the
  terminal agree.
- tests: apps/host/tests/platform-paths.test.mjs (both platforms from either host) and tests/terminal/test_state_paths.py.
2026-10-15 12:31:49 +02:00
Niko1221 3e5e0c73b2 terminal: an existing install is found through settings.json and used in place
The terminal archive is a new folder with no run configs. models.resolve_install() now looks at the install folders
setup.py remembered in settings.json (installs, next to data_dir) when the archive's own folder has none, read-only,
and reports how it found them (explicit, here, remembered, default). The run configs name their engine and model
files by absolute path, so the old install and its data folder keep serving: nothing is copied, moved or created.

tests/terminal/test_migration.py builds today's layout in a temp dir (strata-<model>.json, engine/, run scripts,
.venv, a separate Strata-data with GGUF, packs and MTP, settings.json) and checks the listing, inspect, a profile and
serve --plan, a plain start and the explicit setup path leave every file, size, mtime and hash as they were.
2026-10-15 12:29:44 +02:00
Niko1221 2e324bbbe8 package_terminal: --wheelhouse builds with no internet; lock_wheels --download fetches exactly the locked wheels
A box with no DNS (p620) can build the Linux archive from a folder of wheels fetched and hash-checked elsewhere:
pip still runs --require-hashes against the lock, with --no-index --find-links.
2026-10-15 12:27:36 +02:00
Niko1221 d5be478937 runtime: the managed owner runtime on Windows (loopback TCP + token, job object, byte-range leases)
strata runtime start/status/stop/verify/restart now work on Windows. The owner transport (loopback-only TCP, 256-bit
token, state file in a private directory) was already there; the supervisor around it was fcntl/getuid/killpg only.

- strata_control/winproc.py: LockFileEx byte-range leases (shared and exclusive, on a byte beyond the data so the file
  stays readable) and process identity (OpenProcess + GetProcessTimes). Liveness never uses os.kill(pid, 0): on
  Windows that is TerminateProcess.
- runtime.py: state directory %LOCALAPPDATA%\Strata\runtime with a protected one-entry ACL (existing ones must be
  unshared); owner-only checks through private_files.is_unshared; the state file's address must be a loopback port;
  the owner host is detached with CREATE_NEW_PROCESS_GROUP|CREATE_NO_WINDOW; the server runs in a kill-on-close job
  object (serve/winjob.py, terminate() added), so the whole tree ends with the host however the host ends; a graceful
  stop closes the server's stdin pipe (--owner-fd 0, serve/owner_guard.py turns the EOF into the Ctrl+C drain path);
  the deadline kill ends the job; the ownership marker records the child's creation time, so a PID that was reused
  by another process is not mistaken for the recorded child. A PID from a state file is only compared, never signalled.
- A connect failure to a lost owner is a typed ProfileError (exit 3) instead of a raw OSError.
- `strata serve` and calibration take the activity lease on Windows too.
- tests/terminal/test_runtime.py runs on Windows (32 tests were skipped): a .cmd stands in for the fake engine, the
  signal tests have Windows variants; new tests: a second client attaches from another process, wrong/stale tokens
  cannot stop the owner, a recorded PID of an unrelated live process is never touched, a reused PID with another
  creation time is not the recorded child, the liveness check does not end its target, the directory ACL, shared and
  exclusive leases.
2026-10-15 12:25:58 +02:00
Niko1221 52d58ba32e desktop: pin the Tauri CLI, fix the packaged Windows start
The shell now builds and tests on Windows. Three fixes found by running the installed NSIS build:
Tauri's resource_dir() is a verbatim path (\?\C:\...) that node.exe cannot take as its entry script, so the host died at once and the window hung; the path is made plain.
The self-test only knew the macOS Resources layout; it now finds resources next to the exe (Windows) and under lib/Strata (Linux).
A host that exits early is reported at once instead of after the 60 s health wait.
@tauri-apps/cli 2.12.3 is a pinned devDependency with a lockfile (npm ci), and the npm scripts call it instead of cargo tauri.
2026-10-15 12:14:33 +02:00
Niko1221 66bc889072 terminal archive: bundled Python for Windows x64 and Linux x86_64, hash-locked wheels, launchers, smoke test
package_terminal.py builds the terminal archive for windows-x86_64 (.zip), linux-x86_64 and darwin-arm64 (.tar.gz)
from the pinned python-build-standalone CPython 3.12.15 (SHA-256 per target, from the release's SHA256SUMS) and a
per-target lock, packaging/locks/runtime-<target>-cp312.txt, installed with pip --require-hashes --no-deps
--only-binary=:all: at BUILD time only (pip is the interpreter's own, no upgrade; the result must equal the lock).
packaging/lock_wheels.py makes a lock. Console-script launchers that name the build path are removed; a VERSION
file and the read-only gguf_reader tool join the payload.

- bin/strata.cmd and bin/strata.ps1 (Windows), bin/strata (POSIX): the bundled runtime first, PYTHONNOUSERSITE=1,
  no PYTHONHOME/PYTHONPATH; a path with spaces works.
- START-HERE.bat, SETUP.bat, UPDATE.bat, setup.sh, update.sh use a python/ folder beside them when present (no
  Python lookup or install).
- strata --version (one line: version and interpreter), strata serve --engine mock, a typed exit-5 message when the
  server's packages are missing (strata.py preflight).
- packaging/smoke_terminal.py extracts an archive in a clean env (space in the path, PATH without Python, dead proxy)
  and runs --version, doctor, profile list, install, serve --engine mock (health, models, a chat answer).
- tests/terminal/test_packaging_pins.py: digests, lock shape and agreement with requirements.txt, launcher order.
2026-10-15 12:06:29 +02:00
Niko1221 2e3085a555 start and serve never acquire: one guard, a typed error, a test that fails on egress
setup.py mixed acquisition (pip, downloads, engine fetch and compile) with configuration and start. A start of an
installed model used to check the engine version against the release and download a newer one, and moved model
files into the data folder.

- strata_control/acquisition.py: forbidden() / require() / AcquisitionRefused (exit 5, code acquisition_refused,
  names SETUP / UPDATE / `strata install`). STRATA_NO_ACQUIRE=1 forbids a whole process.
- setup.py: start() runs inside forbidden(); download, pip_install, run, every urlopen (now _urlopen), the
  engine fetch/compile and CUDA-library installs call require() first. A plain start only says when the engine is
  older than expected (engine_update_note) and no longer touches the network; UPDATE / setup keep
  update_installed_engine. A plain start reads the data folder from settings.json and creates, moves and copies
  nothing (data_folder_in_place); setup, check, calibrate and update still gather files as before.
- strata install [--update] [setup options]: the explicit installer entry; a folder without setup.py says so.
- tests/terminal/test_no_acquisition.py: the start/serve import graph reaches no installer code, no serving module
  runs pip/git/cmake/npm or opens a URL (listed exceptions are request-time or loopback), and a real start with the
  network and every subprocess mocked makes no request and runs no installer.
2026-10-15 11:52:21 +02:00
Niko1221 9587a7a1e0 bench test: kill the lease holder's whole process tree on Windows (a venv python.exe is a launcher) 2026-10-15 11:40:42 +02:00
Niko1221 874b8850e8 host tests: the reliability and tuning-route tests use the hardware seam and a Python stub engine 2026-10-15 11:25:29 +02:00
Niko1221 135430b076 host: STRATA_TEST_HARDWARE stands in for the probes in tests; the two lifecycle tests no longer depend on the memory of the PC running them 2026-10-15 11:24:25 +02:00
Niko1221 4d988cb460 host tests: native-onboarding matches serve/server.py with either path separator 2026-10-15 11:23:23 +02:00
Niko1221 44d8858747 host tests: fileURLToPath replaces the regex that stripped the slash before a drive letter (replaces 439339ab1; the pathname form also left %20 in paths with spaces) 2026-10-15 11:22:56 +02:00
Niko1221 03d9f35ed5 components test: the execute-bit assertion is POSIX-only (NTFS has none); the setuid check stays everywhere 2026-10-15 11:21:28 +02:00
Niko1221 c24c426c06 connections test: nest deeper than any Python's JSON recursion limit (3.13 parses 1100 levels) 2026-10-15 11:21:22 +02:00
Niko1221 341228c718 calibration tests: use setup.settings_path() (APPDATA on Windows) instead of an XDG-shaped path 2026-10-15 11:21:01 +02:00
Niko1221 a0bd7b25ea control: Windows leases lock a byte past the marker so another process can still read the holder (benchmark and calibration worker leases; msvcrt locks are mandatory) 2026-10-15 11:20:43 +02:00
Niko1221 18bb5b793b bench tests: no real GPU in the telemetry fixture (its first reading raced the second run), Windows ACL check for the job receipt, lease marker size on Windows; private_files.is_unshared 2026-10-15 11:19:23 +02:00
Niko1221 b0e3b5add6 tuning: peak resident memory of the engine's process tree on Windows (Toolhelp + GetProcessMemoryInfo; ps does not exist there) 2026-10-15 11:16:30 +02:00
Niko1221 f7e12d23ce tuning tests: a Python stub engine (READY/PP/T/DONE) started as [sys.executable, stub.py] replaces the .sh stubs on every OS
strata_control.tuning.engine_command runs a .py engine through the current interpreter (the --help
inventory and the measurement harness both use it).
2026-10-15 11:16:01 +02:00
Niko1221 62413d835c time-skip: a check whose program does not exist is unknown on cmd.exe too (9009, or exit 1 with no such program on PATH); a timed-out check stops the whole process tree; tests no longer need coreutils 2026-10-15 11:14:20 +02:00
Niko1221 1ee7c7337d host: one Python resolver for every spawn (env, repo venv, py -3, python.exe on Windows; python3 elsewhere)
Tuning, Time-skip, the revise command, the native MCP client, setup and
calibration jobs and the native server all go through server/python-resolver.mjs;
the Microsoft Store alias is skipped because it must start.
2026-10-15 11:13:11 +02:00
Niko1221 01f309efef host tests: the native-control fixture fakes .venv/Scripts/python.exe on Windows 2026-10-15 11:13:11 +02:00
Niko1221 decb3f3f8d host: folder picker and skill-add work on Windows (PowerShell FolderBrowserDialog) and Linux (zenity, kdialog); a typed path replaces the dialog where none exists 2026-10-15 11:11:49 +02:00
Niko1221 8a11d125db desktop: Windows node.exe in the Tauri shell, hidden host console, and fetch-node.ps1 (win-x64, same SHA-256 pin as fetch-node.sh)
node_binary() looked for 'node' only. Not compiled here: no Rust toolchain on this PC.
2026-10-15 11:10:00 +02:00
Niko1221 f549c31d1d host: the API-key file is owner-only on Windows (a protected ACL for the current user), and the test asserts the effective ACL 2026-10-15 11:08:58 +02:00
Niko1221 5f051b83d0 control: owner-only files on NTFS (a one-entry protected ACL via ctypes) and the owner state-file check uses it on Windows
POSIX keeps 0600/0700. The address-stability test is skipped on Windows (the
placeholder address is constant by design; the listener picks the port).
2026-10-15 11:08:26 +02:00
Niko1221 515e045faf packaging: monitor rollback works on Windows (os.rename cannot replace the mkdtemp directory) 2026-10-15 11:05:28 +02:00
Niko1221 d442b0307f packaging: the monitor bundle's LICENSE check accepts CRLF; LICENSE is eol=lf in .gitattributes
A Windows checkout (autocrlf) has CRLF in LICENSE, and the bundle verify
required the file to start with b'MIT License\n'. Normalize line endings
before the check, pin the file to LF, and test a CRLF member.
2026-10-15 11:05:24 +02:00
Niko1221 439339ab14 apps/host tests: Windows file URLs in the esbuild entry points
14 test files took the path of `new URL(rel, import.meta.url).pathname`, which on
Windows is "/C:/..." and cannot be resolved (esbuild "Could not resolve"). Strip the
leading slash before the drive letter. No behavior change on macOS/Linux. Went from 14
whole-file failures to 6 real test failures (listed in the S0 notes). Easy to drop if the
owner would rather have a different fix.

Follow-up to the owner's rebuild branch strata/rebuild-2026-10 (small, test-only).
2026-10-15 10:58:10 +02:00
Niko1221 5389eb9db8 kv_hybrid_parity: the tensor-core prompt kernel is sm_70+ (base launch gate), so a refusal below sm_70 passes 2026-10-15 10:47:40 +02:00
Niko1221 31a3d5553e repo: restore the executable bit on the terminal and desktop scripts
The files were copied through Windows, which does not carry the mode; the modes are
as in the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:10 +02:00
Niko1221 e424cc940d docs/work: Phase A implementation status, qualification kit and evidence text
IMPLEMENTATION_STATUS.md, ACCEPTANCE-EVIDENCE.md, QUALIFICATION-KIT.md and the
evidence logs (text only, about 140 KB). The 170 MB of screenshots under
apps/host/docs/evidence and checkpoints are not in git; they are in
StrataSplit\evidence-archive. The Electron mentions in IMPLEMENTATION_STATUS.md are
history (Electron was replaced by Tauri); the superseded top-level desktop/ folder is
not carried over.

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 cef6027dcb packaging: terminal distribution, portable monitor bundle and the acceptance matrix
packaging/package_terminal.py builds the self-contained terminal archive with its own
Python runtime, monitor_bundle.py the portable monitor, acceptance_matrix.py maps the 67
acceptance rows to evidence, README.md documents the target matrix. The built archive
(release/artifacts/*.tar.gz, 42 MB) is not in git.

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 48fc04a425 tuning: job contract, history, apply/rollback and the real-API measurement harness
strata_control/tuning.py (the job contract and the HyperCore-Tune decision), tune_engine
(measures through the real API and records memory), tune_history, tune_apply (explicit,
reversible apply and rollback of a recommendation) and tune_api, with tests/tuning
against a stub engine. Review-only: nothing here starts a sweep from the UI, and the
sweep has the known gaps listed in the S0 notes (e.g. --kv-grow 0 / --expert-cache 0
command lines).

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 486120787d timeskip: strata_control/time_skip.py and tests/time_skip
The Time-skip state machine: checkpoint, revise (a bounded Harness turn), check and
decide, with a durable job record, cancellation, and the "inconclusive means ask the
person" rule. The host routes are in apps/host; the CLI entry points are in strata_cli.

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 090e45adf1 integrations/harness: the Strata profile and the agent bridge
A Harness profile for Strata, a bridge that drives a Harness agent session against a
Strata server (_bridge.py, bridge_e2e.py, bridge_tests.py), the tool-execution /
approval / cancel end-to-end test (tool_flow_e2e.py) and the bounded revision turn the
Time-skip job uses (revise_once.py). Tested only against a scripted mock worker; the
Harness is not wired into the app's chat yet.

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 48ae834b58 apps/desktop: Tauri 2 shell for the Strata desktop app
The Tauri shell (src-tauri Rust, 400-odd lines) that starts the Node App Host and
opens the owner's UI, with the macOS entitlements, icons and generated schemas, and
the build scripts in apps/desktop/scripts that package the app around a private Node
sidecar (digest-verified fetch). The Node binary, Rust target/ and DMGs are not in git;
src-tauri/resources/node/ keeps only its .gitignore placeholder.

Windows and Linux packaging is not working yet (the sidecar lookup expects "node" and the
scripts are bash-only); see the S0 notes.

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 0b3299a019 apps/host: the owner's UI and the Node App Host
The UI source from the 14 October handoff (src/, public/, index.html, vite and
tsconfig, tests, audit-fixes, design docs) and the App Host (server/*.mjs): model
import and monitor routes, API-provider catalog and streaming (OpenAI chat, OpenAI Responses,
the Messages API), serving, tunnel, tasks, plus the host routes for Time-skip
(server/timeskip.mjs) and the tuning plan/reports. src/lib/time-skip.ts renders a real
job's own record.

The UI design is unchanged. The rebuild routed #/tuning to a new TunePanel; that is
reverted: src/screens/ModelRun.tsx and src/main.tsx are the handoff's again (#/tuning
goes where the handoff sends it), and src/components/TunePanel.tsx and
src/styles/tuning.css, which only that route used, are not carried over. Left out of
git on purpose: dist/ (build output), node_modules, and docs/evidence + docs/checkpoints
(170 MB of screenshots, kept in StrataSplit\evidence-archive).

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:24:09 +02:00
Niko1221 a242537838 setup: split build helpers (cmake, ninja) out of the runtime requirements
requirements.txt keeps the runtime packages only; requirements-build.txt pins
cmake and ninja. setup.cmake_build installs only the helper that is missing, right
before a local engine or image-encoder build, so the ready-made engine path (and
serving) never runs pip for build tools. ram_gb() reads hw.memsize on macOS and no
longer crashes without /proc/meminfo. docs/INSTALL.md notes the split; two tests in
tools/test_setup_pins.py cover it.

From the owner's rebuild branch strata/rebuild-2026-10. setup.py merged onto 0.1.45
without conflicts (0.1.45's MIN_ENGINE and pins are unchanged).
2026-10-15 10:23:23 +02:00
Niko1221 c968d37a48 serve: owned-server contract for the terminal (ready file, owner guard, launch identity, queue deadline, stage telemetry)
New serve/ready.py, serve/owner_guard.py and serve/launch_identity.py; server.py
gains the queue wait deadline (QueueWaitExpired/QueueCancelled), the opt-in
STRATA_BATCH_TELEMETRY plumbing (BSTAT/PSTAT parsed off the engine's token stream,
dropped from request output), the engine stage-plan reader, a main(argv, ...)
entry the terminal can call, and per-replica selection hooks; telemetry.py reports
GPU names; the tests for all of it.

server.py merged by hand onto 0.1.45: one conflict, in StrataEngine._take_control, where
0.1.45's STRATA_ADMIT_SHORT_FIRST ordering and the rebuild's queue deadline both edit the
acquire step; both are kept (ordering check first, then the deadline-bounded acquire).
Every line 0.1.45 added to server.py (the #2518 pick_slot fix, the havok multi-user
switches, the Qwen3.6 parallel/vision work) is still present.

From the owner's rebuild branch strata/rebuild-2026-10.
2026-10-15 10:23:23 +02:00
Niko1221 79d8ea066a engine: opt-in batch telemetry (STRATA_BATCH_TELEMETRY=1 prints BSTAT/PSTAT lines)
include/strata/core/batch_telemetry.hpp (BatchWindowSampler, StageInFlightSampler),
its unit test, and the hooks in the generate.cpp batch loops and the --batch-groups
pipelined loop. Off unless STRATA_BATCH_TELEMETRY is exactly "1": no new output, no
new state beyond two disabled samplers, so the default output is unchanged.
Host-side C++ only (no CUDA/HIP/SYCL code), so it builds on every backend.

From the owner's rebuild branch strata/rebuild-2026-10. generate.cpp and
CMakeLists.txt merged onto 0.1.45 without conflicts.
2026-10-15 10:23:23 +02:00
Niko1221 64a91dcdc0 terminal: strata_cli, strata_control, bin/strata and the terminal tests
The command-line and TUI client with profiles, doctor, monitor, serve, calibrate,
bench, the runtime owner, component registry and offline policy, plus tests/terminal,
the component/offline tests, the smoke tools, the terminal CI workflows and the
terminal docs. Tuning and Time-skip modules of strata_control follow in their own
commits (strata_cli/cli.py imports them lazily).

From the owner's rebuild branch strata/rebuild-2026-10 (1e4171e2, based on 296b59d2).
README.md and tools/calibrate.py applied as small 3-way patches.
2026-10-15 10:23:23 +02:00
Niko1221 85b75dcbea repo: .gitignore entries for the terminal/desktop split
Keep node_modules, Tauri target/, DMGs, the Node sidecar, release archives, the
built App Host dist/ and the Mac evidence screenshots out of git. The two
.terminal-*venv entries are from the owner's rebuild branch strata/rebuild-2026-10;
the rest are new for the clean integration branch.
2026-10-15 10:19:47 +02:00
Niko1221 2016b7fe01 sycl: the migrated Gemm::bf16 / gr_norm* definitions take #2091's new parameters (the shared headers changed; SYCL never uses the FP16 image)
The SYCL build of the sm_75 series failed with 'out-of-line definition of bf16 does not match any declaration'
(sycl/src/prefill/gemm.dp.cpp); the four gr_* writers would have failed at link. Gemm::bf16_converts_x() answers false.
2026-10-15 09:52:43 +02:00
Niko1221 456ec79c0a docs/NVIDIA_V100.md: STRATA_GDN_REC_WARP (#1098) 2026-10-15 09:49:14 +02:00
Niko1221 4ef3855745 gdn_chunk_parity: skip (77) on a card whose shared-memory opt-in cannot hold the kkt stage (Pascal), instead of failing 2026-10-15 09:48:05 +02:00
Niko1221 dea9b6de5d gdn chunk / Volta recurrence: name the sm_70-only kernels only in the host pass and the sm_70 device pass (a 60;70 or 70;86 build failed in the other device passes) 2026-10-15 08:55:08 +02:00
Niko1221 bcef7b7818 moe_fused_iq: #2145's host_astages() also for HIP (the shared launch() calls it; the HIP build of #2145 as submitted did not compile) 2026-10-15 08:53:39 +02:00
Niko1221andc3c 285e90aee2 qsa_prompt_attn: Q4_0 KV prompt attention on Turing, opt-in STRATA_PROMPT_ATTN_Q4_TC=1 (#1581, the Q4 attention part only)
Ported from c3c's #1581 (the native-IQ and VRAM-loan parts are not in this commit).  Default unchanged: Turing keeps returning false for q4_0.

Co-authored-by: c3c <[email protected]>
2026-10-15 08:53:32 +02:00
Niko1221 7f8e6f12f4 docs: the Turing (sm_75) kernel series and its switches (STRATA_SM75_PERF) 2026-10-15 08:53:32 +02:00
Niko1221 b30d09cb98 moe_fused, moe_fused_iq: the Turing opt-in (#2145) reads the compute capability through STRATA_EMULATE_CC like the other kernels 2026-10-15 08:53:32 +02:00
Niko1221 0639deff7d qsa_prompt_attn: #1821's register prefetch and shared K/V buffer are a template flag, on with STRATA_SM75_PERF=1 or STRATA_PA_V1_PREFETCH=1
The v1 kernel keeps its exact previous code (and 37.9 KB of shared memory) unless the flag is set.  launch_i8 picks the Turing
single-stage layout by the (emulated) compute capability, so the kernel and the launch agree under STRATA_EMULATE_CC.
2026-10-15 08:53:32 +02:00
Niko1221 7f1964a6cb sm_75 series: the bitwise Turing kernels (#2069-#2072 #2081 #2093 #2220-#2222) are opt-in with STRATA_SM75_PERF=1
No sm_75 card is available here to confirm the series, so the kernels that were on by default for compute capability 7.5
in the submitted PRs now follow one switch, include/strata/core/sm75_series.hpp.  Each keeps its own variable, which
wins in both directions.  The one-read prompt top-k (#2093) was on for every CUDA card as submitted; it is now Turing only
like the rest.  STRATA_EMULATE_CC=75 counts as a Turing card, so a 75-virtual build on an sm_86 GPU runs the series for tests.
2026-10-15 08:53:32 +02:00
xrip 6e76ae23fd #1825 qsa_select: four lanes per query in the Turing prompt scorer (4x fewer shared loads)
block_scores_thread_kernel gave a lane one (query, head) and the whole
128-value query head; ncu on a CMP 50HX showed it bound by the MIO pipe
(mio_throttle first): one shared-memory key load per 4 FMAs. Lane 0's
xor-shuffle tree splits into four independent quarters - c[j] sums groups j,
j+16, j+8, j+24, j+4, j+20, j+12, j+28 - which only meet at the end
(c0 + c2, c1 + c3, then their sum). Now four lanes share a query, lane j
holds quarter j of all four heads, a key load feeds four heads (16 FMAs),
and two xor shuffles finish the same tree. The same pairs in the same order,
so the same bits.

qsa_scores_parity on both cards: all 141 M scores identical to
block_scores_kernel; per call 1,024 queries at 140K 7.76 -> 6.17 ms (-20 %),
at 200K 10.78 -> 8.54 ms (-21 %), 2,048 queries at 40K 12.60 -> 11.57 ms.
agentbench 34K -> 190K: AGENT GATE PASS on the rerun (the first run diverged
at turn 2 - decode's known run-to-run variation; the rerun of the same binary
matched all 36 turns), incremental read -0.6 % median, about -1 % from 150K
up, decode unchanged.
2026-10-15 08:53:32 +02:00
xrip 33237ff367 #1825 qsa_select: prompt-path block scores with a thread per query head (Turing)
On cards without the TF32 scorer (sm_75) the prompt path scored every
(query, key block) with a whole warp: 20 shuffles and five 512-byte loads per
score. block_scores_thread_kernel gives each lane one (query, head), keeps the
128-float query head in registers, reads each key once from shared memory as
a broadcast, and adds the 32 four-value partials in the tree lane 0 of the
xor-shuffle sum computes. Float addition is commutative, so the same pairs in
the same tree give the same bits; the heads are summed 0..3 from 0.0f as
before; the tail block keeps block_scores_tail_kernel. STRATA_SCORES_THREAD=0
selects the old kernel.

Measured on 2x CMP 50HX (TU102, sm_75, unlocked to RTX 2080 clocks):
- qsa_scores_parity (new): 141 M scores bitwise identical on both cards
  (normal values, wide magnitudes with zeros, small integers with ties);
  per call 4.6-5.2x faster, e.g. 1,024 queries at 200K: 52.4 -> 10.8 ms,
  at 140K: 35.9 -> 7.8 ms. 199 registers, no spills.
- nsys, one +4.4K-token turn at 140K: block_scores_kernel was the top prompt
  kernel at 1,045 ms per card (17 %).
- agentbench, IQ3_XXS, layer split, 262144 context, 34K -> 190K (with the
  tokenizer cache): engine read per turn at 190K 9.25 -> 6.96 s (476 -> 632
  tok/s, +33 %), time to first token 10.49 -> 7.13 s (-32 %); median
  incremental read 576 -> 682 tok/s; decode unchanged (73.3 tok/s).
  AGENT GATE: all 36 turns word-for-word identical to the base run.
2026-10-15 08:53:32 +02:00
xrip 1d5319ae88 #1821 qsa_prompt_attn: int8 K/V prefetched in registers and sharing one buffer (Turing)
The Turing (v1) prompt attention kernel used 37.9 KB of shared memory per
128-thread block, so one block ran per SM (4 warps, 12.5 % of the warps an
SM can hold), and each 32-cell chunk gathered its scattered K and V rows
after the previous chunk had finished: ncu on a CMP 50HX showed
long_scoreboard (global load latency) as the top stall. For int8 K/V
(--kv int8, mode 1):
- the next chunk's K/V rows and scales are loaded into registers while the
  current chunk computes (Turing has no cp.async), then stored where the
  gather stored them;
- K is read only by the scores and V only by p.v, so they share one buffer:
  V is written over K after the scores' barrier (the softmax's barrier
  orders it before p.v). 37.9 -> 29.2 KB: two blocks per SM.
The same bytes in the same places and the same arithmetic, so the same bits.
Other KV modes keep their code. qsa_prompt_attn_parity now prints a checksum
of the new kernel's output, so two builds compare bitwise.

Measured on a CMP 50HX (sm_75), qsa_prompt_attn_parity, 2,048 queries:
int8 21.45 -> 12.91 ms at 40K (-40 %), 21.94 -> 13.31 ms at 140K (-39 %);
all output checksums (int8 and fp16, 4K/40K/140K) identical to the previous
kernel. agentbench 34K -> 190K (IQ3_XXS, layer split, 2x CMP 50HX): AGENT
GATE PASS (36 turns identical), incremental prompt read time -3.5 % (median
per-turn ratio 0.965), decode unchanged.
2026-10-15 08:53:32 +02:00
Rafa e3b3ba1c66 #2223 verify: STRATA_NO_PCIE_GROUP=1 drops the empty PCIe-share stage
With --pcie-frac 0 the pool never plans PCIe experts, yet every layer of a
verify window still runs the flag B wait, the staging copy and the grouped
launch over zero groups: six kernels, ~18 us a layer on an RTX 2080 Ti
(kernel-node nsys). STRATA_NO_PCIE_GROUP=1 records the window graph without
them; a plan with PCIe experts then stops the engine with a message.

Engine A/B on an RTX 2080 Ti (Flash-Next Q2_0, 512 tokens, two runs each, with the open sm_75
decode PRs applied): Spanish chat 87.8 -> 89.8 t/s, code 106.1 -> 108.5,
32K 78.4 -> 80.1, 128K 75.0 -> 76.8, 390K 74.7 -> 75.1.
2026-10-15 08:53:32 +02:00
Rafa 88a1c23f61 #2222 sm_75: two launches fewer per decode layer (bitwise)
Two kernel pairs of a verify window's layer run as one launch on Turing,
every byte as before:

- the plan's host-flag wait folded into its copy (143-amdk's
  wait_copy_i32_from_mapped, gfx12-only until now). STRATA_FUSE_WAIT on
  CUDA: 2 = the plan only (Turing default), 1 = also the CPU rows'
  copy_rows_from_mapped_after, which makes every row block poll the flag
  over PCIe and was -1.5 % on an RTX 2080 Ti, 0 = neither.
- native_quantize_q8_1_il: a projection input's q8_1 blocks and their
  interleaved copy (native_q8_1_interleave) in one kernel; an int of the
  copy is four codes gathered with two shuffles. Used for the GDN qkv and
  out-projection inputs and the QSA input and out-projection. STRATA_QIL=0 / 1
  forces it off / on (Turing default on).

RTX 2080 Ti, Flash-Next Q2_0, 512 tokens, two runs each: the plan fold
+0.6..+1.4 % decode, the fused quantize another +0.5..+1 % (code 108.0 ->
109.1 t/s). The fused quantize writes the same bytes as quantize +
interleave at 2560/6144/10240 inputs and 2-4 columns.
2026-10-15 08:53:32 +02:00
Rafa bd45500839 #2221 sm_75: Q2_0 matvec with one warp per four rows (bitwise)
The exact layout gives each Q2_0 row four warps, a half block per thread and a
cross-warp reduction. At the engine's 2560 inputs (40 blocks a row) 48 of the
128 threads have no chunk and the reduction dominates: the QSA attn_q
projection (2560x12288) at three columns took 80 us (~110 GB/s) on an
RTX 2080 Ti.

q2_0_warp_rows_kernel: lane l runs the chunks of the exact kernel's threads
l, 32+l, 64+l and 96+l in their loop order, adds the four sums as warp 0 adds
the other warps' partials, then the same warp_sum - every output is bit for
bit native_q2_0_mmvq_kernel's (one column) and launch_multi's exact layout
(two to four). 1/2/3/4 columns: 36.6/60/68/88 -> 24.8/31/31/37 us.

Default on sm_75 only; STRATA_Q2_WARP=0 keeps the old kernels, =1 takes this
one on any CUDA card.
2026-10-15 08:53:32 +02:00
Rafa 996b9f84f1 #2220 sm_75: QSA decode attention v3 for INT8 pools (bitwise)
attn_chunk_kernel<1> saturates the SM's shared-memory pipe on Turing: its
time grew linearly with the queries while the SMs were half empty (two
16-byte q reads per head per cell with a two-way bank conflict, and 12
warp_sums = 60 shuffles per cell).

attn_chunk_q8_v3_kernel computes the same operations in the same order:
  - the lane's slice of the 12 query heads is read into registers once;
  - the 12 lane sums are reduced with reduce12_gfx906 (16 shuffles, every
    sum warp_sum's pairs);
  - the probabilities are stored cell-major, so the value loop reads a
    cell's 12 as three 16-byte broadcasts.

RTX 2080 Ti, with the merge: 2051 selected cells, 1-4 queries
51.6/78.5/110/146 -> 34.5/41.0/73.0/84.9 us; 600 cells
45.9/74.6/105/137 -> 28.6/32.8/33.5/33.3 us. qsa_prompt_attn_parity
reports the same errors with STRATA_ATTN_V3=0 and 1.

Default on sm_75 only; STRATA_ATTN_V3=0 keeps the default kernel, =1 takes
this one on any CUDA card.
2026-10-15 08:53:32 +02:00
Rafa 9512b9c84b #2145 prefill: fused int8 experts on Turing (sm_75), opt-in
The fused prompt-expert kernels (moe_fused.cu for the Strata Q2_0 pack,
moe_fused_iq.cu native_kernel for the native packs) on sm_75, with
STRATA_PF_FUSED=1 STRATA_PF_FUSED_SM75=1:

- cp.async -> plain 16-byte copies (the barriers around them stay),
  prefetch.global.L2 kept;
- mma m16n8k32 / m16n8k16 s8 -> four / two m8n8k16 (exact int32 sums,
  the same d as the sm_80 instructions), MAGIC as the C operand;
- two activation stages on Turing (64 KB of shared memory);
- native_kernel takes Q2_0 gate/up (64-value blocks, a block per stage),
  so the canonical Q2_0 pack (Q2_0 / Q2_0 layers) runs fused; on Turing
  only that pair is set up;
- Turing, Q2_0: the two halves of a stage share the weight scale, so a
  row's dot is d (dx_0 dot_0 + dx_1 dot_1) - 11 % fewer instructions.

RTX 2080 Ti, one layer, 512 experts top 10 (prefill_fused_iq_test,
Q2_0 / Q2_0 pair added): 2048 tokens MMQ 12.98 ms -> fused 6.59 ms,
16384 tokens 55.08 -> 37.16 ms; fused vs MMQ rel RMS 2e-5, the same
error against the FP32 reference.  In the engine (canonical Q2_0 pack,
98K prompt, 16K chunks): prefill 58.3 -> 53.3 s; layer 0 expert rows vs
MMQ: median rel 1.6e-7.  Teacher-forced 2048 tokens at P = 16K / 32K /
64K: perplexity vs MMQ +3.4 % / -4.5 % / +0.6 % (FP16 path vs MMQ
+0.7 % / +0.1 % / +0.3 %) - trajectory noise, no systematic shift; the
verify phase (same tokens) 42.36 s MMQ, 42.28 s fused.
2026-10-15 08:53:32 +02:00
Niko1221 036d473a3c gdn: the Volta chain-split recurrence is opt-in (STRATA_GDN_REC_WARP=2), not the sm_70 default: it changes the summation order 2026-10-15 08:40:13 +02:00
Niko1221 c0d02ba96a V100 fusions (#1097) behind STRATA_FUSE_DECODE=1; pf_wmma opt-in and only when every visible card is cc 7+; WMMA path skips NVFP4 layers; docs 2026-10-15 08:39:46 +02:00
Niko1221 5333327bd5 metal: wrap_split_test: the note's newline 2026-10-15 07:02:42 +02:00
Niko1221 3ef8d7a09f metal: wrap_split_test reports run_icb instead of counting it (a bench path that misreads at some offsets of a single buffer too) 2026-10-15 07:02:42 +02:00
Niko1221 a35ac34944 metal: wrap_split_test also runs with --factor <= 1 (one buffer) as a control 2026-10-15 07:02:42 +02:00
Niko1221 38978a6dfd metal: wrap_split_test - a sparse file longer than maxBufferLength, wrapped in chunks, read at every chunk boundary (#2338) 2026-10-15 07:02:42 +02:00
hokar3361 8b3ec44b1d metal: wrap a model bigger than maxBufferLength as overlapping no-copy buffers (64 GB Macs keep IQ3_XXS resident); MoE kernels bound per layer 2026-10-15 07:02:42 +02:00
Niko1221andBlueKingMuch 2b296987b9 conversation cache: a match that reaches a pin=N prefix is taken instead of borrowed (#1164)
A borrow copies the donor's checkpoints up to the match, so a match that reached a pinned shared prefix gave the borrower the pin too. Such a match now takes the conversation as before. The borrow's guard is guard()/unguard()/guard_blocked(), and the refusal before evicting counts the pinned conversations along with the donor.

Rebased by hand onto 0.1.44.1 (evict_oldest() honours the guarded donor).

Co-authored-by: BlueKingMuch <[email protected]>
2026-10-15 07:02:42 +02:00
Niko1221andBlueKingMuch c9e571e057 serve: a new conversation that starts with a checkpoint of a parked one borrows it, the parked conversation stays (#1164)
Sibling subagents with the same system prompt and tools no longer cost each other their history: only the K/V up to the checkpoint and the checkpoint itself are copied into the session (conversation_snapshot_restore_prefix); the parked conversation stays where it is and is pinned while the outgoing one parks. A match on a parked conversation's live state or newest checkpoint is that conversation going on and is taken as before.

Rebased onto 0.1.44.1 by hand: the pinned shared prefix (#879) and evict_oldest() now also skip the borrowed donor, serials follow every erase, and park-gain (#2265) and the pin= sibling rule gate the outgoing park.

Co-authored-by: BlueKingMuch <[email protected]>
2026-10-15 07:02:42 +02:00
Niko1221 04e3481b99 docs: --conversation-cache-park-gain (#2265) 2026-10-15 07:02:42 +02:00
Niko1221 3d99819e38 setup: the engine builds pin -DSTRATA_MMQ_KQUANTS=ON too (465cd61d flipped it with STRATA_Q6K_EXPERTS, and a stale CMakeCache.txt keeps the old OFF the same way; found by qiuzan2001 on #2515) 2026-10-15 06:22:32 +02:00
Niko1221 dd08a7d0ec serve: pick_slot sends a chat's next turn to the slot that holds it, found by its turn checkpoint (#2518)
A client renders the history again for its next turn, without the reply the slot decoded or the assistant header,
so the slot's held stream (prompt + reply) is no start of the new prompt and pick_slot fell back to the least
recently used slot. With two chats in two slots and two next turns arriving together, whichever was read first
could take the other chat's slot, and the engine's slot copy then overwrote the cache the second turn needed
(0 tokens reused, a full prompt read). pick_slot now also counts the held tokens up to the last <|im_start|>
(where the engine takes its turn checkpoint) when the prompt agrees with them, and a pipelined layer split with one
slot per group (--batch 2 on two cards) no longer ignores what its slots hold.
2026-10-15 05:59:33 +02:00
Niko1221 c2215a2af3 docs/DETAILS.md: the elastic encoder at 196K context 2026-10-15 05:59:13 +02:00
Niko1221 5d83130cdb setup: the HIP engine build pins -DSTRATA_Q6K_EXPERTS=ON too (the same stale CMake cache as #2515 for the CUDA build); the CUDA configure line is tested (tools/test_setup_stale_stamp.py) 2026-10-15 05:59:13 +02:00
xrip a7d4b75443 serve: --prompt-cache-spread, checkpoints kept spread back from the conversation's end (opt-in)
Least recent use keeps the newest checkpoints and every turn adds one, so after a few short turns all 6 sit in the
last 1-2K tokens. An agent client that rewrites its history a few thousand tokens back then resumes from the system
prompt: on 2x CMP 50HX with the pi coding agent a 210,528-token prompt that was the same as the session for 204,349
tokens resumed at 9,525 and read 201,003 again (100 s).

With the flag the victim is the checkpoint whose removal leaves the smallest gap for its distance from the newest, so
the kept points thin out geometrically: after 200 turns of 1,000 tokens they sit 0, 1K, 3K, 8K, 43K and 200K before
the end, and an edit 6K back reads 8K again (least recent use: 200K), 42K back 43K. The same number of checkpoints, no
more RAM; never the root, the newest or a pinned one, the tail rule first, the plain policy without a candidate. The
resume is the usual exact-prefix match.

On the box: an 82,593-token conversation grown turn by turn, a turn ~6 steps back rewritten, resumed at 73,610 tokens
in 10.2 s against 40.0 s read in full; with the expert placement fixed (--adapt-swaps 0, --pcie-frac 0.17) the
greedy answer was the same text as the full read. conv_cache_test: the spread cases (a 200-turn simulation against
least recent use, the fallbacks). docs/DETAILS.md: a paragraph after the prompt cache's options.
2026-10-15 05:42:53 +02:00
Krypto-Whitehat d9cb22d287 Conversation cache: --conversation-cache-park-gain N stops a turn that grew a few hundred tokens from rewriting a whole snapshot (RTX 5090 Laptop, IQ3_XXS: 1,418 parks -> 1, 2,251.9 GB -> 3.26 GB host copy at 4.7 % read back; decode -3.4 %, inside the run-to-run spread)
A conversation that only advanced a few hundred tokens since its last park does not need a fresh 3 GB copy: the entry it would supersede still holds this conversation's prefix, and the next turn drops the copy anyway (drop_superseded). Reported in #2264 over 1,418 park events on one long single conversation.

The flag is opt-in and defaults to 0, which parks every turn exactly as before. What this adds is a way to ask for the gate without a rebuild: on the reporter's rig it is 3.26 GB instead of 9.79 GB per benchmark, the superseded class disappears, and the decode median moves -3.4 %, inside that arm's 92.2-124.1 tok/s spread.

parked_prefix_tokens() reports exactly the entries drop_superseded() would remove - it finds them the same way, by the deepest checkpoint the outgoing chain still holds - so the gate cannot name a different entry than the park would drop. conversation_cache_test covers it (7 checks; 4,226 pass).

Fixes #2264
2026-10-15 05:42:53 +02:00
BlueKingMuch 53f72571fb conversation cache: a conversation parked again keeps at most an eighth of each K/V buffer in reserve (was up to 16 MiB per buffer, 0.4-0.5 GB per conversation)
A conversation that was restored and is parked again keeps its unchanged K/V
pages and appends the new ones (92a3d5f). The appended segment reserved room
for later turns, growing geometrically up to 16 MiB - per buffer, and a parked
image has one per K/V layer and part (k, v, their scales, the indexer rows, the
draft layer). With Qwen3.8-Flash-Next (IQ3_S, int8 K/V) a 32K-token main agent
parked again took 1.49 GB instead of 0.97 GB, a 17K-token subagent 1.16 GB
instead of 0.74 GB. The cache counts that capacity against its budget, so with
2816 MiB the reserve decides which conversations are evicted.

The reserve is now at most an eighth of what the buffer held before the append
(at least 64 KiB, and at most 16 MiB as before). Small turns still fill the
reserve instead of adding a transfer each: the 4096-append test keeps its
segment bound, and 256 turns of 1 MiB onto a 16 MiB buffer take 28 segments
(17 before).

The docs said parking falls back to a full capture when the reserve would evict
another conversation; that check went in 6cd29d9. With the reserve bounded the
two captures differ by at most an eighth of the K/V, and the docs now say what
the reserve is.
2026-10-15 05:42:52 +02:00
qni-live b6dac80740 logit_bias: normalize a long list once per request instead of five times
A standing ban of a whole script (65,929 IDs) was normalized five times per request: in _openai, twice in
validate_request and twice in engine_key. About 200 ms of Python per request, whatever the answer length.

A list is now normalized once. The same object again in a request is answered from the last call; an equal list in
a later request is found by its compact JSON text, which tells False from 0 and "1" from 1 (== does not: [[7, False]]
== [[7, 0]]). Only a list that normalized without an error is stored. Lists under 64 entries skip the cache.
No change to what normalize, engine_key and validate_request return or raise.
2026-10-15 05:41:36 +02:00
f13a870d52 serve: tokenise the prompt from the last shared prefix, not from the start (#1852, #567)
PromptEncoder keeps the last prompts with their ids and resume points (one per
conversation-cache slot) and encodes only the text after the longest shared
special-token boundary; the ids equal a full encode. Rebased onto 0.1.44.1
(prepare() keeps the new `cancel` argument; forget_prompt on a context refusal;
count_tokens reads without storing). SECURITY.md notes the in-memory prompts.

Based on gputier's #1852 (8 commits) and blange48's #567.

Co-authored-by: Guillaume PUTIER <[email protected]>
Co-authored-by: benoit lange <[email protected]>
2026-10-15 05:41:24 +02:00
Niko1221 40a6220f24 docs: DEQUANT_WIDE on sm_86 also measured on the RTX A4000 2026-10-15 05:11:05 +02:00
Niko1221 abce866ebe docs: the CUDA opt-in switches ported for 0.1.45 (#2058 #2059 #2075 #1888 #1889 #1418 #1370 #1367 #1262 #1368 #1107), measured on sm_86; STRATA_DEQUANT_WIDE default on sm_86 2026-10-15 04:10:19 +02:00
Niko1221 5f843a926f dequant: the wide kernels (STRATA_DEQUANT_WIDE) are on by default on sm_86 too
Bit-identical (gate: 10/10 + exact_pp 4K/20K IDENTICAL with STRATA_DEQUANT_WIDE=1 on the RTX 3060 and the RTX A4000), and the time
to the first token of a 150 / 300 / 450-token prompt falls 12-20% on the RTX 3060 (5 of 5 pairs each), the prompt read +2.3%.
STRATA_DEQUANT_WIDE=0 turns it off. Arch-gated to the measured sm_86 and sm_61. Follow-up to #2178 (Techannel), the kernels this switches on;
the same stores are what #1525 (sergqwer) proposes for Q8_0 / IQ4_XS.
2026-10-15 02:38:49 +02:00
p3lusa 2783569494 setup: pin -DSTRATA_Q6K_EXPERTS=ON in the CUDA engine build
A build folder configured while the option's default was OFF keeps
STRATA_Q6K_EXPERTS=OFF in its CMakeCache.txt even after the default
flipped to ON (465cd61d): CMake does not rewrite an option already in
the cache, and source_hash() fingerprints the source files but not the
CMake option values, so setup recompiles the engine with the Q6_K
gate/up expert kernels still disabled. A model whose routed experts are
Q4_K/Q6_K (Unsloth UD-Q4_K_XL) then fails to load with:

  strata-q36: blk.34.experts are Q4_K/Q6_K, which the native expert
  kernels do not take (build with -DSTRATA_Q6K_EXPERTS=ON for Q6_K)

Passing -DSTRATA_Q6K_EXPERTS=ON explicitly overrides the stale cache
entry, so the engine always has the kernels its default build promises.
Same pin in the Dockerfile build for consistency. Not hardware-specific:
it is a CMake cache-staleness issue, independent of GPU/platform.
2026-10-15 02:29:55 +02:00
Niko1221 c8cab03503 docs/DETAILS.md: the image encoder on 16 GB cards, measured (#2406) 2026-10-15 01:13:31 +02:00
Niko1221 df44bcb712 Merge f-145-havok (#2364 multi-user series, opt-in) 2026-10-15 01:11:37 +02:00
Niko1221 45752724d9 setup: Flash-Next images 'yes' gets the CPU encoder on an NVIDIA card under 24 GB, --vision gpu kept with a note (#2406: the GPU encoder cost 3-5% of text speed on 16 GB cards, 35-60% on an RTX 4080 SUPER under Windows as reported); goldens re-recorded for the two 16 GB Unsloth configs 2026-10-15 01:11:37 +02:00
Niko1221 0408f0921c Merge f-145-q36speed (Qwen3.6 speed: decode/prompt kernels, bigger prompt chunk by default) 2026-10-14 22:23:29 +02:00
Niko1221 bb90c106b6 docs/QWEN36.md: speed section (kernels, measured R9700 / RTX 3060 numbers, the prompt chunk default and its KL, what was dropped) 2026-10-14 22:04:58 +02:00
Niko1221 7e8485a48b verify: a Q4_0 KV cache takes the batched verify window by default (STRATA_DF_QB_Q4=0 for the old path): the 6-token graph is 1950 nodes, not 2610, ids identical; + kv_window_parity test (#2403: a gfx1030 fault with q4_0 + KV streaming + MTP, not reproduced; cause unconfirmed) 2026-10-14 21:55:16 +02:00
Niko1221 8e2c70a132 metal: STRATA_QSA_SLOTS launchers decline (no Metal kernel); the shared verify.cpp links again 2026-10-14 21:42:05 +02:00
Niko1221 79e5cd1827 docs/BATCHING.md: more than 8 slots on a layer split, the opt-in admission and decode switches, and the 4x R9700 numbers (PR #2364 port) 2026-10-14 21:40:11 +02:00
Niko1221 f755a06d7c #2234 follow-up: the rows attention runs the default kernel's body from its own kernel (the default kernel's signature, and the pre-sm_75 / HIP function-pointer selections, stay as they were: HIP did not build); per-row calls where the per-row path runs a variant (lane-cell, pre-sm_75, gfx906)
batch_rows_parity also checks the one top-k launch over all rows against one call per row.
2026-10-14 21:38:26 +02:00
Niko1221 a08db82dbd Version 0.1.45: project version, Metal string and the setup minimum engine 2026-10-14 21:38:21 +02:00
Niko1221 55bae973fb qwen35: the prompt delta rule takes two barriers per token instead of three
The output sum of token t is done after token t + 1's first barrier (sk / sq double-buffered), off the recurrence's chain. Same arithmetic: the
3000-token prompt's logits and 8 greedy steps are bit-identical to the base engine at chunk 512 and 2048 (R9700: 913 -> 885 ms, 607 -> 573 ms).
2026-10-14 21:38:07 +02:00
Niko1221 4b348ad4b8 docs/QWEN36.md: the P100 row 2026-10-14 21:38:01 +02:00
Niko1221 34db47a8e1 setup: Qwen3.6 --parallel on small cards follows the measured cost (the slots may take 35% of the experts' room; 12 GB at 8K: 4, 16 GB at 32K: 2), and the text says the gain against one-at-a-time with drafts is small 2026-10-14 21:37:45 +02:00
Niko1221 e79156ae2e Merge f-145-q36 (Qwen3.6 start that cannot fit its buffers says so; vision_cmp --dump-ngl) 2026-10-14 21:36:20 +02:00
InB4DevOps fc27296dee serve: skip tool-body rescans until a closing tag arrives 2026-10-14 12:49:34 +02:00
InB4DevOps b145a4a2d6 serve: avoid exhaustive partial-tag suffix checks 2026-10-14 12:49:26 +02:00
Niko1221 c2947a9230 q36 engine: a hybrid start that cannot fit its buffers says so with the numbers
With many slots on a small card (8 sessions on a 12 GB RTX 3060, UD-Q6_K_XL) the sessions left too little VRAM for the hybrid
mode's staging pair, CPU-output buffer and expert slots, and the start died in a bare cudaMalloc failure. place_experts now adds
up what it is about to allocate and exits 1 with "not enough VRAM left for the hybrid experts' buffers and slots (N MiB needed,
M MiB free ...)" and the way out (fewer slots, a shorter --max-context, fewer --expert-slots). Only a start that would have
crashed changes; the placement and every answer are as before.

tools/qwen35/q36_vram_refusal.py: demanding starts (8 slots, long contexts, every expert as a slot) must reach READY or exit 1
with one of the engine's refusals, never a raw allocation failure.
2026-10-14 12:33:29 +02:00
Niko1221 739bc94881 tests: batch_rows_parity, #2234's one-launch kernels against the per-row launches (scores, decode attention, GDN window and commit), bitwise 2026-10-14 12:26:10 +02:00
bsvinay cbc1578a2f layer split overlap: off beside --pipeline-windows (#859)
--pipeline-windows runs its own overlapped decode loop with its own hand-offs, so
STRATA_SPLIT_OVERLAP now switches itself off when it is set (one log line says so).

Rebased onto main 1735d64: the early launch of the next stage keeps #871's graph
variant (the next stage's refresh_ar() runs before its capture and prelaunch, and
not again in its run() when it was prelaunched), resets the plan error word like
run()'s launch, and the chain path checks that word after its sync as well.
2026-10-14 12:26:10 +02:00
bsvinay 090b8e9f13 layer split overlap: the hand-off publish reads the test word only with STRATA_TEST_HANDOFF_DROP set
Without the hook the window graph passes no word, so the publish kernel does no
extra read of mapped memory per window.
2026-10-14 12:26:10 +02:00
bsvinay 2443c9225c layer split overlap: bound the GPU-side hand-off wait (STRATA_SPLIT_WAIT_MS)
Review on #936 (justxiami): the reading stage's wait_flag_ge spun with no bound,
so a lost publish would leave that stream spinning.

- wait_handoff: the reading stage's graph now waits with a bound (default 30 s,
  STRATA_SPLIT_WAIT_MS, 0 = unbounded, GPU clock as in gpu_stamp). On timeout it
  sets a mapped error word and returns, so the graph completes; the host checks
  the word after the window and fails it with an error that names the setting,
  instead of using a stale hand-off.
- The host's per-layer wait (20 s, #267) still normally ends a stalled window
  first; release_gpu_waits already raises the hand-in flag.
- Test hook STRATA_TEST_HANDOFF_DROP=N: the first stage's N-th window withholds
  its flag (a lost publish), to exercise both paths.

Only active with STRATA_SPLIT_OVERLAP=1; the default path is unchanged.
2026-10-14 12:26:10 +02:00
Niko1221andbsvinay 56146c4c49 layer split: overlap the stages' hand-off (STRATA_SPLIT_OVERLAP=1, opt-in)
Each verify window of a layer split runs its stages strictly in turn: stage 1 finishes, the host syncs its stream,
then stages and launches stage 2's graph. Under HIP the sync and the graph launch both sit on the critical path of
every window.

With STRATA_SPLIT_OVERLAP=1 the hand-off carries a ready word (mapped, one per hand-off buffer):
- the writing stage's graph ends with handoff_publish: the payload with volatile stores (RDNA keeps plain stores to
  mapped memory in L2), a system fence per thread, and the last block raises the flag;
- the reading stage's graph starts with wait_flag_ge(flag, 1);
- Verifier::run captures the next stage's window on this thread, then stages and launches it on a helper thread
  (prelaunch) while this thread serves its own layers; it then serves the next stage's layers directly and syncs
  the earlier stage once the chain is done;
- release_gpu_waits also raises the hand-in flag; an early return after the prelaunch releases the next stage.

Batch windows and --split-device 0 keep the serial order. Off by default.

Measured on 2x RX 7900 XTX (ROCm, Linux), Swift 1.5 IQ3_XXS, 512-token greedy answers, median of 8:
decode 53 -> 70.1 tok/s (code prompt), 45 -> 55.9 tok/s (prose prompt); greedy output byte-identical with the
flag at 0 and 1; quiz 47/51 both ways. Not measured on NVIDIA.

Split-overlap launch merged with 0.1.44 HostGraphLaunch (STRATA_VERIFY_ASYNC_LAUNCH): the launcher object is
kept in an optional so an early-launched window has none.

Co-authored-by: bsvinay <[email protected]>
2026-10-14 12:26:10 +02:00
Niko1221andSuperDistance 5d6bf8d414 batch-groups: decode and give way between the pieces of a prompt read (#2050, opt-in STRATA_PIPED_READ_PIECE), on top of #2326's concurrent read
Squash of SuperDistance's three commits of PR #2050 (decode between the pieces of a read on the pipelined groups, the
piece lines counting the rows the slots decoded meanwhile, the stall watchdog staying on for a read in pieces),
applied after #2326 here: the BYIELD give-way is #2326's give_way lambda, the concurrent block runs first and the
piece path serves a read that STRATA_PIPED_CONCURRENT leaves to it (pictures, configurations where it is off).

Co-authored-by: SuperDistance <[email protected]>
2026-10-14 12:26:10 +02:00
SuperDistance 073d079d27 batch-groups: the concurrent read's threads do not share the host thread's core on Linux
On Linux a thread inherits its creator's CPU mask, and the host thread is pinned to the one core the pool leaves it
(aux_cpus.hpp). The read's thread and the doorbell thread were created from it, so they shared that core with the host
thread that pumps the windows, and every window waited there for the doorbell of each layer: 6.1 rows/s during a read
on 4x RTX 5080, 4 stages (blange48 on #2326). Windows places the host with CPU Sets, which new threads do not inherit,
so it did not show there. Both threads now move off the host's core and the pool workers' CPUs (on an SMT machine with
the default pool: the idle sibling threads; without such CPUs, every CPU but the host's), and the threads that already
run on the host's CPU (the prompt path's copy threads, started by the first prompt) move the same way at a read.
--aux-cpus places them all when it is on. STRATA_PIPED_CONCURRENT_UNPIN=0 keeps the inherited CPUs (an A/B).

Measured on Linux (WSL2, Ubuntu 24.04, 2x RTX 5090, 4 streams, a cold 32K prompt; same binary, UNPIN=0 / default): the
slots decoded 25.9 and 23.7 rows/s during the read with the threads kept on the host's CPU (76-81 threads there), and
75.6 and 62.8 with this change; with the n-gram table in RAM (WSL's disk reads it 7x slower than Windows) 110.7 rows/s,
as on Windows (107.0). piped_read_join_bench.py (cold32k, followup, two_joiners; table in RAM, 2-3 runs each): 24-37
rows/s during the 30K+ reads kept on the host's CPU, 95-127 with this change, and 925-1,065 stream tokens during the
cold 32K join against 454-484. On Windows, STRATA_TEST_CONC_ON_HOST_CPU=1 (a test) puts both threads on the host
thread's CPU set first: 18.6 rows/s kept there, 105.4 with this change.
2026-10-14 11:58:16 +02:00
SuperDistance 0d47295f5a batch-groups: --remote-expert-opt without a helper cache does not keep the concurrent read off
The flag works only through a helper cache (--expert-cache-device1..3), and on a plain layer split it is inert
(MULTI_GPU.md, #1447), so the check looks at the helper caches themselves. #2275's rig runs a plain split with the
flag set, and the mode would have stayed off there for a cache it does not have.
2026-10-14 11:58:16 +02:00
SuperDistance 6c5c382b94 docs/BATCHING.md: reading beside the decoding slots, and what it measured
The opt-in's section: how the read and the windows share the GPUs, the doorbell thread, the give-way at a chunk, where
the mode is off. Measured against main (a0116b27) on 2x RTX 5090 in one window (main, this, this, main): the stops,
the streams' rate during the reads with the same answers every run, the joining prompts' TTFT, a question during a
read, the same randomised load on both, the doorbell thread's first wait as an A/B; 16 GB cards on each side; a weaker
host as well in #1906's shape; the default path's check; text equality.

tools/piped_read_join_bench.py, the client that measured it, is #2050's, unchanged (the same file as there).
2026-10-14 11:58:16 +02:00
SuperDistance 34b726e090 batch-groups: the concurrent read allocates nothing on its threads, keeps its checks and the watchdog, and waits on a launch
From a review of the concurrent read:
- With external serving batch_poll still finds a window that will not finish (its graph ended or failed with a layer
  to ring, or no layer rang for 20 s): the served count is read before the stream query and the ring count after it,
  at most one query a millisecond; the last ring's time is an atomic. batch_serve reads a window's row count after its
  served count, so a window launched while the doorbell thread was descheduled is not served with the last one's rows.
- #1996's pinned upload rings (0.1.43.1) grew a slot at its first larger upload, a pinned free and allocation on the
  read's thread: the drafter's records and the prompt path's routing tables reserve their slots before the read
  (PinnedRing::reserve, Prefill::reserve_uploads).
- Off, said once, where serving a doorbell makes a CUDA call (--pcie-mode dma, STRATA_FS_SLOTS, the helper caches) or
  the expert source recycles the buffers it hands out (the GGUF in place, unbuffered, STRATA_IO_PREFETCH); a prompt
  with pictures is read the usual way. The expert sources' read counters are atomics.
  The check is a pure function, concurrent_read_off (batch_read.hpp), with a test in batch_read_test.
- pump's all-idle branch turned the stall watchdog off when the last slot finished mid-read; it stays on while a prompt
  is read until the request ends.
- Between windows the doorbell thread slept 50 us, a whole timer tick on Windows (up to 15.6 ms): it now waits on a
  condition each launch notifies (STRATA_TEST_BELL_SLEEP_US=N, a test, restores the sleep for an A/B). With nothing
  decoding the main thread waits for the read's end or a line instead of sleeping. A pool failure the doorbell thread
  set is reported before pump clears the flag for the next window.
- The read's log line ends with the slots' rate and, per stage, its windows and the host time that set each up. The
  priority stream's cudaGetLastError runs only under the opt-in.
2026-10-14 11:58:15 +02:00
SuperDistance bc90e1acff batch-groups: read a prompt beside the decoding slots on a thread of its own (opt-in, experimental STRATA_PIPED_CONCURRENT)
With a layer split and --batch-groups > 1 a prompt read is one sp.run, so every decoding slot stops for the whole read
(#1906); #2050's pieces bound the stop but drain the pipeline at each piece. With STRATA_PIPED_CONCURRENT=1 the read
runs on a worker thread while the main thread keeps the groups going through the pipeline (pump) and takes BSTOP and
BYIELD meanwhile. The prompt's K/V goes to the main sessions and the slots' to their own; the windows' streams get the
device's highest priority on CUDA (STRATA_PIPED_CONCURRENT_PRIO=0 keeps the default).

- The slots' doorbells are served on a thread of their own that makes no CUDA call (Verifier::batch_serve, now
  batch_poll's per-layer service too, and set_external_serve). Two hangs found while building this, read from the
  stall dumps against the PDB, had #31's shape: the main thread waited for the driver's lock in cudaSetDevice while its
  holder waited for the GPU (a cudaFree in the drafter's prompt pass; cuBLASLt's lazy load of a GEMM module), and a
  window on that GPU waited for its doorbell.
- MtpDrafter::reserve_prefill captures the drafter's prompt graphs and sizes its records before the read.
- Prefill::stop_at_chunk: a BYIELD ends the run at its next chunk, also with no slot decoding; the pipeline is drained
  there and the part read gives way as #656 does. A read nobody waits for is not cut.
- The reader is joined on every way out (an exception too). Off beside STRATA_PREFILL_HELP and STRATA_SPLIT_MTP_BATCH,
  which allocate during a read.
2026-10-14 11:58:15 +02:00
SuperDistance a2de744653 PleTable: a lock around its reader state, so two threads can use one table
The prompt path gathers a chunk's PLE rows on its own thread and the batch windows gather theirs on the main thread,
from the same table (ss.ple.table). Today they never overlap - a prompt run's read-ahead ends with the run, and the
windows run between runs - but a prompt read beside running windows would race on the reader's tickets and the
prefetch slots. Every public method that touches them now holds a recursive mutex (gather holds it across its issue
and collect). Uncontended it is a few nanoseconds per call; nothing else changes.
2026-10-14 11:58:14 +02:00
Niko1221andruibeikaa 125025a770 prefill: --prefill-pipe, chunks sized for a layer split's pipeline (#1441, opt-in; b/a from setup's --calibrate)
Squash of ruibeikaa's six commits of PR #1441 (the first fixed b/a, then runtime-learned b/a, then calibrated b/a, the
segment planned once, --prefill-pipe 1 following STRATA_PREFILL_EQUAL), applied onto 0.1.44.1 with the conflicts in
generate.cpp (Options, BSlot, the lend's chunk with --prefill-head-alias) resolved.

Co-authored-by: ruibeikaa <[email protected]>
2026-10-14 11:52:32 +02:00
Niko1221 81ead832d6 docs: STRATA_CPU_IQ_ALL_EXACT1 (the exact one-token expert rows of #1415 and #1544), what was measured 2026-10-14 11:50:52 +02:00
Niko1221 d30f5d1042 docs (#1749): the wide top-k kernel on HIP, what it does and the switches that restore the old ones (R9700: 22.0 -> 0.93 ms per 256 queries at 524K; a 500K-token prompt +34%) 2026-10-14 11:46:11 +02:00
Niko1221 6612542e17 tools/qwen35: q36_lgcmp.py (two logits dumps: KL, top-1) and the multilingual draft-head prompts 2026-10-14 11:43:29 +02:00
Niko1221 b0f675be76 qwen35: the chunk rule counts what the chunk really costs (measured on the R9700)
A bigger chunk is only taken when the experts, the 0.77 GiB reserve and the chunk's buffers (0.5 MiB per token) still fit without giving the packed
decode copies back: 131072 cells -> 512, 100000 -> 2048, 32768 with --batch 4 -> 1024, --batch 8 -> 512 (hybrid as before).
2026-10-14 11:40:05 +02:00
Niko1221 93f0f48f10 qwen35: the prompt chunk defaults to the largest that fits (4096 .. 1024); the router and the shared gate in one product for decode batches; conv tile fix for batch rows
Prompt chunk (owner decision): with every expert in VRAM the auto rule picks 4096, 2048 or 1024 (the chunk's buffers cost ~0.5 MiB per token,
the sessions' caches 45 KB per cell and session; 1024 at most with --batch N so the other slots' windows are not held for a whole chunk), else 512.
The hybrid mode keeps 2048 and falls back to 1024 / 512 on a card with little room. STRATA_Q36_CHUNK=512 or --prefill 512 gives the old size.
R9700 pp4096: 512 3390, 1024 4495, 2048 5353, 4096 5750 tok/s. Not bit-identical to 512 (other GEMM tiles): see docs/QWEN36.md for the KL.
Decode batches of up to 8 rows take the router and the shared expert's gate in one product (rows of 257), as a single token already did.
k_conv_seq's tile mode switched off the history write for batch rows too (grid y is the group count there): fixed.
2026-10-14 11:24:15 +02:00
Niko1221 2bc0a7cafd gguf_reader: the placeholder note ends in a newline 2026-10-14 11:16:52 +02:00
Niko1221 94d189fca4 gguf_reader: an all-zero attention.compress_ratios list is a converter's placeholder and is treated as absent, with a one-line note (#2396: orcarouter Qwen3.8-Flash-Next-Uncensored GGUFs refused since 0.1.44); real lists are still checked in full 2026-10-14 11:16:26 +02:00
Niko1221 6f4697f662 qwen35: the prompt's gated delta rule loads the next token's inputs while it computes this one
k_gdn_seq_cols started every token with a chain of global loads; they are issued one token ahead now. Same arithmetic: the 3000-token
prompt's logits and the next 8 greedy steps are bit-identical to the previous engine at chunk 512 and 2048 (R9700: prompt 931 -> 902 ms
at chunk 512 together with the tiled conv, 616 -> 591 ms at 2048).
2026-10-14 11:05:46 +02:00
Niko1221 c9e7f6dd5a qwen35: faster decode kernels with the old kernels' bits (STRATA_Q36_VERIFY=1 compares them on real activations)
q36_kernels.cuh: one warp per row for the shared expert's and attention k/v Q8_0 products (grid capped at 1024 warps: 2048 rows as 2048
warps took 21 us, 5 us as 1024), shared gate+up+SwiGLU+q8_1 in one launch, k_moe_out with 1024 threads and the next layer's q8_1 image, k_add_rms
with the q8_1 image, router top-k and expert grouping in one launch. CUDA: the division / exp the --use_fast_math TUs they mirror use are spelled out.
Engine: prompt GDN conv in tiles of 16 tokens side by side (same arithmetic per token), --draft-vocab N (the MTP draft head over the first N rows only,
a draft proposes and the verify window decides: output unchanged), the packed Q8_0 default on Windows too (it was Linux only), STRATA_Q36_OLD_KERNELS=1
switches all of it off.
R9700, UD-Q4_K_XL: decode 82.5 -> 120.6 tok/s (greedy 10 prompts identical), --mtp 2 160 -> 174.
2026-10-14 10:52:17 +02:00
Niko1221 fb51ae6fbd iq_kernels: the V2K gate/up kernels take Qwen3.6-35B-A3B's expert geometry (n_embd 2048, n_ff 512, Q4_K/Q5_K gate+up, Q5_K down)
For that geometry only: the s27 gate/up kernel and the fused SwiGLU/q8_1, then the old down kernel (the new down kernel is slower at n_ff 512).
Flash-Next's shapes (2560/640) take the same paths as before; the path is still STRATA_EXPERT_V2K (default on gfx12).
2026-10-14 10:50:29 +02:00
Niko1221 c298fa559e qsa scorer (#1943): the exact four-lanes-per-query scorer is the default on gfx12; the test runs there too
The same kernel measured on an R9700 (gfx1201): qsa_scores_gfx1100_parity --full compares 223,664,777 scores and 11,895,800 ids bitwise equal
to the warp scorer (35 cases), the scorer stage with 256 queries is 3.6x faster at 131K and 4.0x at 240K, and a fresh prompt reads 8.7% faster at
131K tokens (4 of 4 pairs) and 16.4% faster at 230K (3 of 3), with identical answer tokens and byte-identical exact_pp residual dumps at 70K and 131K
tokens. So on gfx12 it is on for batches of >= 64 queries with >= 16,385 live blocks (65K cells and more) unless STRATA_SCORES_THREAD=0; gfx1100
stays the author's opt-in (STRATA_SCORES_THREAD=1). A one-time stderr line says when it engages by default.
2026-10-14 10:48:36 +02:00
Niko1221 7d3fd4b24f tests (#2147): test_setup_rocm_pins skips an index with no saved version list (the gfx1150 opt-in has none yet) 2026-10-14 10:48:36 +02:00
Niko1221 a61436c4da refill parts (#2078): not in gfx1151's default table (only gfx1151 measured it, and nothing here can gate that arch): opt-in STRATA_REFILL_PARTS=1, documented in section 5 2026-10-14 10:48:36 +02:00
Niko1221 b317c23f24 pcie (#2224): the PcieAuto share on HIP outside gfx12 is opt-in (STRATA_PCIE_FRAC_DEFAULT=auto), not a gfx1100 default: the author's gate was no-regression only and a second gfx1100 report saw it never engage 2026-10-14 10:48:36 +02:00
Niko1221 595686f6a1 hc persist (#2226): not offered on gfx11 and gfx12
The persistent read's barrier timeout uses __builtin_amdgcn_s_memrealtime, which gfx11 and gfx12 do not have (clang: "needs target feature
s-memrealtime"): the PR as ported failed to build for gfx1100, gfx1151 and gfx1200/gfx1201. On those parts the barrier compiles to an immediate
abort, fused_gr_persist_init refuses the read with a start-up line, and the default read is used. An R9700 (gfx1201) measured the read +0.7% / +1.0%
per window slower than the default in the PR thread, so nothing is lost. The silent, permanent barrier abort at run time that the same thread
describes is noted in docs/AMD_HIP.md.
2026-10-14 10:48:35 +02:00
Niko1221 44dbf66af5 Merge f-145-nv (#2347: NVFP4 on AVX2 CPUs: the pack-folder hint, setup note, docs) 2026-10-14 10:02:45 +02:00
Vop3p b2b5000952 hc persist results: ten interleaved pairs on 0.1.43.1 with no other switch (-6.6 % / -4.4 % ms per verify window at 4K / 32K, 10 of 10 pairs) 2026-10-14 09:59:02 +02:00
Vop3p cea3d61b66 tests: hc_persist_parity - the STRATA_HC_PERSIST=1 read against the default multi read at T = 1..8 (bounded 1e-4, finite, two runs bitwise), and hc_persist_default_off: with the variable unset it never runs; a skip where the card cannot hold the 80 blocks and on CUDA 2026-10-14 09:59:02 +02:00
Vop3p 6cbc15a81a STRATA_HC_PERSIST=1 (opt-in, HIP): the verifier's hyper-connection read as one persistent kernel
The verifier reads the hyper-connection twice per layer at 1-8 tokens: three small launches with two global
dependencies, the down projection on 41 blocks. Opt-in STRATA_HC_PERSIST=1 runs it as one kernel of 80 blocks with two
grid barriers on an atomic counter: the down projection split by column slices over all 80 blocks, a distributed
reduction, then the up projection and the same epilogue as gr_up_kernel.

gfx1030 (2x RX 6900 XT, layer split): the read 1.35-2.10x at 1-8 tokens; ms per verify window -4.2 % at 4K and
-2.8 % at 32K (one binary, the variable unset / set, 6 alternating rounds, every round the same way). Another summation order (not bitwise); against a
double-accumulation reference (STRATA_HC_PERSIST=3) as close as the default read.

Residency is decided by running every token count once at start-up with a 50 ms barrier timeout and comparing with
the default read; a timeout keeps the default read for that token count and larger ones, wave64 cards are refused.
One line per card says what was decided. Only the verifier's reads use it (one stream per card); the drafter's reads,
the prompt path, CUDA and SYCL are unchanged (CUDA and SYCL get two empty stubs). With the variable unset both entry
points return in their first line: the output is bitwise the same as main.

Measurements, the simulated smaller cards and a stand-alone timing tool:
bench/results/2026-10-12-hc-persist-gfx1030/.
2026-10-14 09:59:02 +02:00
2jztricks e5d91bf239 hip: add opt-in exact gfx1100 QSA prefill scorer 2026-10-14 09:59:02 +02:00
sa 82922f55e2 topk: compile the wide kernel on HIP too (it was CUDA-only; every deep context on AMD paid the ref radix)
block_topk_wide_kernel is plain shared-memory / atomicAdd code with no sm_90
feature, but its definition and the dispatch branch sat behind
#if !defined(__HIPCC__), so on AMD a context past the register kernel's reach
(about 135K cells) always fell back to the 4-pass ref radix.  The ids are
identical by construction (the same radix rule), which the bench states:
256/256 at 256 queries x 65537 blocks, 1/1 at nq=1.

Measured on a Ryzen AI Max+ 395 (gfx1151, ROCm 10.1), qsa_select_bench:
- nq=1, 131073 blocks (a decode window at 512K context): 2.24 -> 0.77 ms (2.9x)
- nq=256, 65537 blocks (a prompt batch at 262K):        3.16 -> 1.43 ms (2.2x)

End to end this moved the deep-context decode of a 512K-token prompt from
19-22 to 24-26 tok/s and the 512K prefill from 473 to 634 tok/s on that PC
(together with the split scorer of another change).  STRATA_TOPK_STREAM=0
still restores the register kernel.
2026-10-14 09:59:02 +02:00
Niko1221andZac Chen b0e88eb12f pcie: the in-window share refinement (PcieAuto) on HIP too — gfx1100 port of the XeStrata stabilization cluster's finding
Upstream aa7b0cb gates PcieAuto to CUDA single-GPU (#if !defined(STRATA_USE_HIP)); the
machinery itself is engine-generic (pool ms per missed expert + miss counts vs the probed
link rate), and this build's start-up H2D probe is already best-of-4 with a +-0.7% spread,
so the HIP exclusion was unmeasured-only, the same class as the ROUTE_TAIL_SKIP gate.
Same opt-outs as CUDA: layer splits excluded, STRATA_PCIE_FRAC_DEFAULT=old restores the
fixed start-up rule. Provenance: MistVVK/XeStrata commits 05309096/055cf9b0/c71cac34/
5952e8e7 (their measurement-stabilization cluster) demonstrated the failure mode and the
fix class; their exact subsystem is not portable (it is their own engine code).

(cherry picked from commit acb3deb0522a8fc351a4047d2bd11b8ab7a73dca)

Co-authored-by: Zac Chen <[email protected]>
2026-10-14 09:59:02 +02:00
Niko1221andonionbutties 01e1bc5c0e setup: write the gfx1150 speed switches for the opt-in install
The engine turns 18 exact (bit-identical) switches on by default on gfx1151
(src/core/arch_defaults.cpp) but keeps gfx1150 out of its arch defaults
(src/core/arch_defaults_test.cpp asserts arch_default_env("gfx1150").empty()).
A gfx1150 install gets the 17 that apply here - the gfx1151 set less
STRATA_HCD_EXACT, which copies the gfx1151 hipBLASLt table's solutions 1176 / 1177
and so does not fit beside a gfx1150 table - written into the config's "env"
beside STRATA_HIPBLASLT_TUNING.  The Radeon 890M report (docs/AMD_HIP.md;
bench/results/2026-10-07-community-gfx1150) measured the same greedy answers with
all 17 on and with all 17 off, decode +2% and prompts +3-6%.

The set is setup's own (SETUP_ENV): a re-run refreshes it, and it does not linger
after the opt-in is turned off.  A new test (tools/test_setup_strix_halo.py,
Gfx1150Switches) keeps GFX1150_SWITCHES equal to the gfx1151 arch-default table
minus STRATA_HCD_EXACT, keeps the engine's own default path for gfx1150 empty, and
checks the SETUP_ENV wiring.  Setup's suites pass (strix_halo 40, amd 36, choices 47).

Co-authored-by: onionbutties <[email protected]>
2026-10-14 09:59:02 +02:00
Niko1221andonionbutties 6f0505c69e setup: opt-in gfx1150 (Radeon 890M / 880M, Strix Point) as a unified-memory AMD card
STRATA_EXPERIMENTAL_GFX1150=1 takes the Radeon 890M / 880M iGPU (Ryzen AI 300,
Strix Point, gfx1150) as a unified-memory card like Strix Halo: it joins AMD_ARCHS
and the UMA handling (new is_uma_apu helper), with notes of its own.  The engine
already builds and runs for gfx1150 (0.1.40.2, #1217; tools/hip/gfx1150-hipblaslt-100401.txt
is calibrated on it); no release ships a ready-made Windows engine for it yet, so
a Windows user builds one (STRATA_HIP_ARCHS=gfx1150 tools\hip\build_windows.bat) and
starts setup with --prebuilt dist\.  Unset, the default path is unchanged: the test
that asserts gfx1150 is not in AMD_ARCHS (tools/test_setup_strix_halo.py) still passes.

Co-authored-by: onionbutties <[email protected]>
2026-10-14 09:59:02 +02:00
Niko1221andAlexisCarbonell d3c4a6de77 setup: write the gfx1103 exact speed switches for the opt-in install
The engine turns 18 exact (bit-identical) switches on by default on gfx1151 (src/core/arch_defaults.cpp) but keeps
gfx1103 out of its arch defaults (src/core/arch_defaults_test.cpp asserts arch_default_env("gfx1103").empty()).  A
gfx1103 install gets the 17 that apply here - the gfx1151 set less STRATA_HCD_EXACT, which copies the gfx1151
hipBLASLt table's solutions 1176 / 1177 and so does not fit beside a gfx1103 table - written into the config's
"env" beside STRATA_HIPBLASLT_TUNING.  One 780M (Ryzen 7 8745HS, 60 GB, GTT 50 GiB, ROCm 10.1, Q2_0, 4K prompt)
measured the same greedy token ids with the 17 on and with them off, +13% to +15% prompt and decode +3.2% (the
decode half alone); bench/results/2026-10-12-community-gfx1103.

The set is setup's own (SETUP_ENV): a re-run refreshes it, and it does not linger after the opt-in is turned off.
The gfx1103 note now names the native dot4 / WMMA kernels and docs/RADEON_780M.md, which documents the opt-in
build, the table and the measured numbers.  Setup's suites pass (strix_halo, rocm_pins, amd, choices: 143 tests).
Refs #1423.

Co-authored-by: AlexisCarbonell <[email protected]>
2026-10-14 09:59:02 +02:00
onionbutties 7aa71b08e5 hip: add a gfx1150 hipBLASLt 100500 tuning table (Radeon 890M)
Calibrated on a Radeon 890M (Strix Point, gfx1150) with
tools/hip/tune_hipblaslt --workspace-mib 32 over the 16 dense GEMM geometries of
the shipped gfx1150-hipblaslt-100401.txt at T = 64, 128, ..., 16384: two passes,
each row keeping the solution with the lowest mean of all six repetitions.  The
three rows slower than plain hipBLAS (the tiny bf16 N=1 K=2560 shape at T = 64,
128 and 256) are left out, so those buckets route to the nearest good row.  Every
remaining row beats plain hipBLAS: 1.07x to 16.7x per GEMM, geometric mean 5.0x.
The solution IDs are scoped to hipBLASLt 1.5.0 (version 100500, the ROCm
10.2.0a20260930 nightly the engine here was built with).
2026-10-14 09:59:02 +02:00
AlexisCarbonell 0fb72e8b0e hip: a gfx1103 hipBLASLt tuning table (Radeon 780M)
The Radeon 780M (gfx1103, RDNA3, 12 CUs) builds and runs (issue #1423), but had no hipBLASLt table, so the prompt's
dense GEMMs ran on plain hipBLAS.  This adds tools/hip/gfx1103-hipblaslt-100401.txt, calibrated on one 780M with
ROCm 10.1's hipBLASLt 1.4.1 (version number 100401) over the gfx1150 table's 144 GEMM cases at T = 64..16384 with
the engine's 32 MiB workspace.

Two tune_hipblaslt passes back to back; each row keeps the solution with the lowest median of all six repetitions,
and a row is left out when no candidate beats plain hipBLAS.  143 rows, 1.05-23.30x per GEMM, geometric mean 4.95x;
the one shape left out is bf16 N=1 K=2560 T=64 (best 0.039 ms against 0.024 ms on plain hipBLAS, sub-100 us
timings), where the engine falls back to hipBLASEx.

Measured on the 780M (Ryzen 7 8745HS, 60 GB, GTT 50 GiB, Q2_0 + MTP, 32K context): a 4K prompt read 90 tok/s on
plain hipBLAS against 187 with the table (medians of an interleaved A/B; with the gfx1103 exact switches and the
WMMA opt-ins, 277 at 4K and 272 at 32K).  hip_prefill_hipblaslt_gemm passes with it (launches=4 fallbacks=0) and
hip_prefill_hcd_exact_parity skips, because the table picks no 1176 / 1177 for the HC down GEMM - the gfx1150 table
gets the same skip, which is why STRATA_HCD_EXACT is not in the gfx1103 switch set.  Setup installs for gfx1103 only
with STRATA_EXPERIMENTAL_GFX1103=1 and then picks the table up when the installed hipBLASLt reports 1.4.1.

Docs, the community report (runs.csv, tune-summary.csv, the server config, the engine's start-up lines) and the
setup tables list carry the numbers.  Refs #1423.
2026-10-14 09:59:02 +02:00
Niko1221andlibratechw 1b1bd4b4a8 prefill: STRATA_GDN_CHUNKED=1 on gfx11, the prompt's DeltaNet recurrence in 16-token chunks on the matrix cores (opt-in)
The switch exists for CUDA (#1372) and is compiled out on HIP.  On gfx11 it now runs the WY form of the gated delta rule in
chunks of 16 tokens, every product one 16x16x16 WMMA (FP16 operands, FP32 accumulation, the state in the accumulators), with the
output norm and the z gate fused: a prep kernel for T and P of every chunk, then a scan with a block per value head.
It rounds differently from the FP32 recurrence, so it is opt-in; the default path is unchanged.  A gfx1100, gfx1101, gfx1102, gfx1150 or gfx1151 part, 64 tokens and
more, an output row stride divisible by 8 and no STRATA_GDN_REC_HEADS take it; anything else runs the existing kernels.

tests/hip/gdn_chunked.cpp: bounds on the FP16 output and the carried state against the existing recurrence over 0-16,385 tokens
(chunk and stride edges, a split request), a double-precision CPU check, and bit-identical output with the switch off or unset.
tests/hip/gdn_rec_head.cpp unsets the switch, as its variants 0 and 1 are the references there.  hip_gdn_chunked --bench and
--bench-default time this path and gfx1151's default kernels, one per process (the switch is read once).

gfx1151 (Ryzen AI Max+ 395, UD-IQ4_XS): a layer's recurrence + norm takes 1.60x less time than the default kernels at 16K and
32K tokens; a whole prompt 1.4-2.1% less at 4K-128K after a warm-up prompt (11 pairs each: 1.8-2.1% with the docs/STRIX_HALO.md section-5 switches,
1.4-1.5% without them, 10 or 11 of 11 faster); first-token KL against the default 0.006-0.007 on average, the same top token in
12 of 12 prompts.  Run on gfx1151 only; kernels.cu compiles for gfx1030, 1100, 1151, 1152 and 1201 (docs/AMD_HIP.md).

Co-authored-by: libratechw <[email protected]>
2026-10-14 09:59:02 +02:00
libratechw feeb8dc808 docs: STRATA_PF_F16=1 on gfx1151 (what it covers, the layer times, first-token KL) 2026-10-14 09:58:51 +02:00
libratechw ed160b461a tests: hip_prefill_fused_iq also runs with STRATA_PF_F16=1 STRATA_PF_F16_KQ=1 2026-10-14 09:58:51 +02:00
libratechw 0c2608ae97 prefill: STRATA_PF_F16=1 takes Q8_0 down without STRATA_PF_FUSED_KQ on gfx11
Q8_0 has no minimum term and the F16 kernel's output already differs from the int8 kernels', so the KQ opt-in is
not needed for it.  UD-IQ4_XS has five Q8_0-down layers (4 IQ3_S / Q8_0, 1 IQ4_XS / Q8_0): they left the fused path
for MMQ, and since fused_ring() asks every layer, the pack kept MMQ's buffers.  d_covered_on() is the one test, for
native_supported() and experts_native(); gfx12 keeps d_covered().

UD-IQ4_XS, gfx1151, 3 interleaved rounds, prompt tok/s (v0.1.42 / 65d809c6 with STRATA_PF_F16=1 / this):
4K 926.6 / 992.2 / 1054.8, 32K 1257.4 / 1364.6 / 1397.1, 128K 1202.0 / 1309.4 / 1333.6.
2026-10-14 09:58:51 +02:00
libratechw bd5cc1f27b prefill: the other native formats on gfx11's F16 matrix cores (STRATA_PF_F16=1)
native_f16_kernel takes the IQ gate/up formats (IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ3_S, IQ4_XS) and the Q2_0, IQ4_NL
and Q8_0 down: load_unit / convert decode each sub-block once into an int8 weight tile in LDS with its scales as
half2, each wave turns its own rows into F16 (gufo's Q8_0 decode: sign flip, byte permute into 1024's mantissa,
one packed multiply), and the F16 WMMA accumulates all of K.  The work item, the activations staged as F16, the WMMA
loop and both epilogues move to moe_f16_common.cuh, shared with the K-quant kernels (moe_f16_kq.cu, now on it; its
output bits are unchanged).  The switch becomes STRATA_PF_F16=1 (was STRATA_PF_F16_KQ) and covers both; the K-quants
still need STRATA_PF_FUSED_KQ=1.  gfx12 keeps the int8 kernels.

hip_prefill_fused_iq (gfx1151, ROCm 7.14.1 clang 23, top 10 over 512 experts, one 128-expert launch, ms):
- IQ3_S / IQ4_NL at 16384 tokens: gate/up 11.65 -> 9.87, down 6.11 -> 4.71; layer at 8192 37.3 -> 32.2.
- every pair is faster at 4096 and 16384 (IQ2_S gate/up 14.38 -> 9.85, Q8_0 down 6.80 -> 4.79), and at 128-2048
  tokens equal or faster.
- FP32 reference rel RMS as the int8 kernels' (IQ3_S / IQ4_NL 1.456e-2 against 1.454e-2); against the int8 model
  and MMQ the F16 path sits at 2.4-2.7e-3, so both checks allow 5e-3 under STRATA_PF_F16=1 (f16_on).
- With the weights decoded to F16 in LDS (18 KB a block) gate/up was 9.83 and down 5.71.  Neither a VGPR cap nor
  dropping the sign table (4 blocks per WGP) nor decoding after the products moved gate/up; without the decode it is
  8.98, without the weight loads 9.30.
Off by default (the output differs from the int8 kernels').
2026-10-14 09:58:51 +02:00
libratechw 929010be61 prefill: gfx12 keeps the int8 kernels under STRATA_PF_F16_KQ=1
moe_f16_kq.cu's bodies are gfx11's WMMA layout (and trap on other targets); v0.1.42's gfx12 native kernels share
experts_native's launch, so the F16 path is taken only off gfx12.
2026-10-14 09:58:51 +02:00
libratechw b2492918f5 prefill: Q5_1 down on the gfx11 F16 matrix cores too (STRATA_PF_F16_KQ=1)
down_kernel takes gate/up's structure for UD-Q4_K_XL's Q5_1 down (47 of its layers): the codes and high-bit words
packed in LDS, w = q d + m per wave in F16, H staged once as F16 from its int8 blocks, FP32 outputs at the routed
rows.  The activation staging and the WMMA loop are shared with gate/up (ActStage, mma_tiles).  Q8_0 down stays on
native_w11_kernel.

hip_prefill_fused_iq (gfx1151, one layer, 16384 tokens x top 10 over 512 experts): down 7.10 -> 4.83 ms per
128-expert launch; Q4_K / Q5_1 layer 62.7 -> 53.5 ms (85.3 on the int8 kernels); FP32 reference rel RMS 1.304e-2.
2026-10-14 09:58:50 +02:00
libratechw df218bd8ae prefill: K-quant gate/up on the gfx11 F16 matrix cores (opt-in STRATA_PF_F16_KQ=1)
UD-Q4_K_XL's routed gate/up was 6/10 of the prompt-time gap to gufo on gfx1151 (128K: 23.1 s against 13.9 s).
moe_f16_kq.cu adapts gufo's paired RoutedF16GEMMKernel (97b8645f, MIT) to Strata's tables, int8 activations and H:
the Q4_K / Q5_K codes stay packed in LDS and each wave turns its own rows into F16 after the read, so the F16 WMMA
accumulates all of K in F32 with no per-block epilogue; the activations are staged once in LDS as F16 for every wave;
the epilogue writes H as int8 per 32 features for the existing down kernels.  One block per work item; two tiles of
an expert per item from 64 routed rows per expert on average.

hip_prefill_fused_iq (gfx1151, ROCm 7.14.1 clang 23, one layer, 16384 tokens x top 10 over 512 experts):
- gate/up kernel 14.63 -> 8.46 ms per 128-expert launch; Q4_K / Q5_1 layer 85.3 -> 62.7 ms, Q5_K / Q5_1 89.9 -> 66.2.
- against the FP32 reference the error is the int8 path's (rel RMS 1.305e-2 / 1.302e-2: the int8 activations
  dominate); the int8 model check allows 5e-3 under this switch (2.5e-3 measured with Q8_0 down).
- the output bits are the same with 64- and 128-row items.
Off by default; it needs STRATA_PF_FUSED=1 and STRATA_PF_FUSED_KQ=1, whose output already differs from MMQ's.
2026-10-14 09:58:50 +02:00
libratechw d573332130 prefill: refill a prompt's lent expert slots from the GGUF mapping's pieces (STRATA_REFILL_PARTS=1, gfx1151's table)
After a prompt the borrowed expert-cache slots are refilled from the experts' files.  With the GGUF read in place an
expert is three tensors, and the refill first assembled each 4 MB blob in a staging buffer (a copy on the calling thread)
and then copied that to the slot.  The pieces are contiguous in the mapping, so they can go straight to their offsets in
the slot: blob_parts() hands them out, ExpertCache::fill_slot_queued_parts copies them on the same queue.  The bytes in
the slot are the same.  Off by default; gfx1151's table turns it on.
2026-10-14 09:58:50 +02:00
Niko1221 37f2941fc2 docs/NVFP4.md: the CPU rows on a CPU without AVX-512 (AVX2 rows bit-equal to ggml's, measured on three CPUs) (#2347) 2026-10-14 09:45:18 +02:00
Niko1221 5e9bd5db4f docs/MAC.md: the M1 Max line quotes the 9,940-token prompt speed of the #2348 report, not a 24-token cold request 2026-10-14 08:05:07 +02:00
Niko1221 b1f7b4ac3a pytest: a session config home, so no test can write or migrate the user's real Strata settings (#198 data-folder migration) 2026-10-14 08:04:52 +02:00
Niko1221 3de6363749 Merge f-145-mac (the #2365 test no longer runs data_folder() for real) 2026-10-14 07:51:50 +02:00
Niko1221 98f5484c8d gr_upmix_parity: skipped (exit 77) below sm_80, where the fused read refuses by design (#1370 follow-up) 2026-10-14 07:46:45 +02:00
Niko1221andAndy Slater b558df67b4 cpu: the exact singleton rows only where ggml-cpu is itself the AVX2 build (not AVX-512, not an ISA-floor build)
STRATA_CPU_IQ_ALL_EXACT1 reproduces ggml-cpu's AVX2 single-token dot bit for bit, which only
holds if the ggml-cpu in this binary is that dot: a STRATA_ISA_FLOOR build runs ggml's AVX1/SSE
dots, and a native build on an AVX-512 CPU run with STRATA_FORCE_AVX2=1 runs its AVX-512 one.
The guard is #2215's (ggml_cpu_has_avx2 / ggml_cpu_has_avx512).

Co-authored-by: Andy Slater <[email protected]>
2026-10-14 07:20:05 +02:00
Niko1221 6eb34c7036 setup: --parallel is not offered where the card holds every expert for one session only (4 slots at 128K on a 32 GB card moved all but 15 experts per layer to the CPU: 81 tok/s against ~180) 2026-10-14 07:17:06 +02:00
Niko1221 093108c2e0 setup: the engine's own programs run with its rocm/bin on PATH (#2372: strata --families on Windows AMD showed hipblas.dll / libhipblaslt.dll not found and Qwen3.6 was refused) 2026-10-14 07:16:36 +02:00
Niko1221 8bfef49c5f q36_vision_cmp: --dump-ngl, llama.cpp's offloaded layers for a quant larger than the card 2026-10-14 07:14:54 +02:00
Niko1221 a6bdc3fb5b pf_wmma: compile the WMMA kernel bodies only for sm_70+ device passes (a multi-arch 60;70 build failed in the sm_60 pass: no nvcuda::wmma there) 2026-10-14 07:14:39 +02:00
Niko1221andtuandat3019 10552345e9 AMD HIP: the primary ordinal, and the helper caches beside the resident RAM mode
Three changes for two-card AMD PCs whose faster card does not come first in the
runtime's enumeration (Windows HIP orders the cards itself; HIP_VISIBLE_DEVICES
only filters):

- gfx1032 (RX 6600) joins the unvalidated build list - it shares the gfx1030 /
  gfx1031 / gfx1034 dp4a path.
- STRATA_PRIMARY_DEVICE=N picks the visible ordinal the engine's own session
  runs on; it is set before any context or allocation exists.  The helper
  caches take the other visible devices, the HIP arch check and the peer tier's
  device queries follow it, and the primary's session is created on the chosen
  device.  A layer split still assumes device 0 and is refused with this set.
  Unset or 0 is upstream's behavior unchanged.
- The helper-GPU caches are allowed beside the resident RAM mode: the experts a
  helper holds are left out of the RAM copy as a split stage's are, and the two
  adaptive tiers already keep each other's experts out.  RemoteExperts::finish
  spins on its stream like PeerExperts::finish - on HIP the CUDA branch's
  cudaInitDevice(ScheduleSpin) does not exist.

Measured on an RX 6800 (16 GB, gfx1030) + RX 6600 (8 GB, gfx1032), Windows 11,
32 GB of RAM, Swift 1.5 IQ3_XXS at 160K context, the resident RAM mode with the
helper's cache left out of the copy: decode 37.5 -> 44.2 tok/s on 8K-token
requests with a 3800-slot helper; the CPU pool's share of the routed experts
falls from ~2.9 to ~0.4 entries per layer-window.

Rebased onto 296b59d2 (conflicts in generate.cpp, peer_experts.cpp, hip_backend.cmake, docs/AMD_HIP.md resolved by keeping main's code and adding the PR's).  Changes: remote_caches is kept (still used by the --batch-mtp check); the spin in RemoteExperts::finish is HIP only (CUDA keeps its cudaInitDevice(ScheduleSpin) + blocking sync).  NOT COMPILED YET, not run.

Co-authored-by: tuandat3019 <[email protected]>
2026-10-14 07:14:17 +02:00
Niko1221 e8b20f2a6e setup: NVFP4 on a CPU without AVX-512 is offered, with a note on what its CPU rows are (#2347)
The menu never hid NVFP4 on an AVX2 CPU; the engine runs a native pack's rows on AVX2 kernels that are bit-equal to
ggml's. The NVFP4 menu line and the install step now say, on a CPU with AVX2 and no AVX-512, that the CPU's share of
the experts runs on AVX2 rows (measured about as fast when memory-bound, up to ~12% slower on 4-token windows).
A note only: nothing is refused or capped. tools/test_setup_menu.py: 2 tests.
2026-10-14 07:14:11 +02:00
Vop3p eb6e0f1e4d calibrate: measure the shared-expert stream fork on AMD (STRATA_SH_STREAM, off against on) and keep it per PC
The fork stays off by default on HIP (#816, #826): an RX 6800 on Windows and 4x R9700 decode 12-45% faster without it.
The same card family on another PC goes the other way (#1272: 2x RX 6900 XT on Linux), and the output is identical
either way, so it is a per-PC choice - the open item of #1272.

- tools/calibrate.py: step 6, HIP only. With everything chosen before, the engine is started twice, with
  STRATA_SH_STREAM=0 and =1 set explicitly, two rates each; "1" is kept only when it beats "0" by more than MIN_GAIN
  (the rule of the other steps). measure() returns the choice as "env"; a failed start drops that candidate (#1337).
  start_engine takes an optional env; apply_env() sets the chosen variables and removes the others of ENV_KEYS, so an
  older calibration's choice does not linger.
- setup.py: --calibrate writes "env" into the config and the settings file; a reinstall applies a saved calibration's
  "env" the same way (a calibration saved before this step leaves "env" as it is).
- docs/AMD_HIP.md: the SH_STREAM note says calibrate measures it.
- tools/test_calibrate.py: the step runs on HIP only, keeps the fork only on a win, falls back on a failed start;
  apply_env; setup writes and later removes the variable.

Measured with the real engine (0.1.42 tag, 2x RX 6900 XT gfx1030, Linux, IQ3_S layer split, setup-style config): the
whole calibration took 982 s, the new step chose STRATA_SH_STREAM=1 (off 62.2 / 61.3, on 67.2 / 66.6 tok/s, +8.3%).
Separately, with a fixed protocol, the fork's output is byte-identical to off (4 of 4 requests, one card and the split).
2026-10-14 07:12:58 +02:00
Niko1221andxjc10 4f21be8723 hip: the FP32 tiled QSA block scorer on HIP cards (opt-in STRATA_SELECT_SIMT=1, as on CUDA), tiled 32/128/16/8 on gfx103x
The scorer of 65ce329 is plain FP32 arithmetic, so it runs on any HIP card as it is; without matrix cores the
prompt path otherwise scores on the warp kernel.  Its tiling becomes template parameters: every tiling keeps each
score's fmaf chain over k = 0..127 in order and the same relu sum over the heads, so every tiling gives the same
bytes (RX 6900 XT, qsa_select_bench at 32K and 128K: 14 tilings, also on 60 and 72 of its 80 CUs through
HSA_CU_MASK, all byte-identical to the original kernel; the fastest tiling the same at every width).  CUDA keeps
the tiling it was written with; gfx103x runs 32/128/16/8: 256 queries at 128K 2.69 ms (warp kernel) -> 0.695 ms
(the original tiling) -> 0.587 ms.  FP64 gate passed, selections identical 256/256.  On the 2x RX 6900 XT layer
split a 128K-token prompt reads 7.4% faster, 32K 0.8%.  A HIP ctest runs it at 131K.

Rebased onto 296b59d2: the docs/AMD_HIP.md conflict (main's SGEMM/rocBLAS-table bullets from the ports before it) is resolved by keeping all bullets.

Co-authored-by: xjc10 <[email protected]>
2026-10-14 07:12:53 +02:00
xjc10 4b3e4d096f rocBLAS table docs: the shipped gfx1030 table was measured on an RX 6900 XT; other gfx1030 cards can make their own with tune_rocblas (reports welcome) 2026-10-14 07:12:35 +02:00
Niko1221andxjc10 57c95d5faf hip: a rocBLAS solution table for the FP16 prompt GEMMs on gfx103x (the GDN projection 7x below 1,152 tokens; short prompts -10 to -14%)
rocBLAS picks the kernel for each FP16 product from its own table by shape, and on gfx1030 that pick is far
from the best kernel the same library holds for some shapes: N = 10240 K = 2560 (the GDN input projection)
below T = 1152 runs at 5.2 TFLOPS where the best of rocBLAS's 237 solutions for n = 799 is 6x faster (not an
alignment effect: n = 800 is as slow as 799).

tools/hip/tune_rocblas enumerates the library's solutions per shape and token bucket, checks each against the
default kernel's output and that it also runs at smaller odd token counts, and writes the winners (and
`default` rows bounding their reach) as tools/hip/<arch>-rocblas-<version>.txt; Gemm::f16_inplace runs the
table's solution for a shape (STRATA_ROCBLAS_TUNING: a file, or the directory from which the engine takes the
file of its own rocBLAS build; the version string, tweak hash included, is checked). One tiny default product
at startup opens rocBLAS's kernel library (a solution-index call before that fails although the index is
valid); a refused solution falls back to the default kernel for that call, after three refusals for good.
setup points the engine at tools/hip when a table for the card's architecture exists. Shipped: the gfx1030
table for ROCm 10.0.0's rocBLAS 5.6.0.8d1ae90e (112 rows, 76 with a solution).

Measured (RX 6900 XT, 6f32ec0 + #835 + this, temperature 0, the same answers with and without): a 799-token
prompt's GPU time 3,329 -> 2,985 ms (-10.3%), 495 tokens 2,477 -> 2,139 (-13.6%), 8,874 tokens 11,978 ->
11,459 (-4.3%); a PCIe-bound 1,108-token prompt unchanged (its saving reappeared as `wait copy`). ctest 66 of
70 (the 4 failures are outside this change); hip_prefill_rocblas_gemm, the new smoke test, passes with 0
fallbacks. Builds on #835. bench/results/2026-10-05-rdna2-rocblas-table has the per-run lines and the tuner's
summary.

(cherry picked from commit e35e103604)

Rebased onto 296b59d2: conflicts in setup.py (SETUP_ENV gained STRATA_ROCBLAS_TUNING beside the 0.1.44 entries) and docs/AMD_HIP.md (kept both sections) resolved.

Co-authored-by: xjc10 <[email protected]>
2026-10-14 07:12:35 +02:00
Benjamin Larsson 2f44bf51dd STRATA_HC_REQ8: hyper-connection projections requantized to int8 + fp32 scale per 32 at load
The rebased #704 under the name asked for there (upstream's STRATA_HC_Q8 now
reads the GGUF's Q8_0 bytes, a different thing).  Opt-in.  The four BF16 hc_*
projections per layer (1,200 MiB) are skipped by the loader and uploaded as
int8 codes with one fp32 scale per 32 values (675 MiB); the freed VRAM goes to
the expert cache.

- Q8 is a template parameter of the down/up kernels (single- and multi-token):
  the BF16 instantiations compile exactly as before, the launch sites pick the
  Q8 ones when w_down / w_up carry scales.  The inject rows stay BF16.
- The staged / v3 / upmix / PF_HCDOWN / gfx906 split paths read BF16 rows and
  are skipped for it; the prompt path expands the codes into a bf16 scratch for
  its GEMM.
- The loader seeks with 64-bit offsets.
- A layer split keeps BF16 with a warning instead of exiting.
2026-10-14 07:11:52 +02:00
Niko1221 0c808eed00 RDNA2 SGEMM prompt GEMMs (#1123) are opt-in: STRATA_RDNA2_SGEMM=1 (gfx103x) / =all (any AMD card)
Fix-up to the ported #1123 commits.  The route is not bit-identical to the native call
(the author measured at most 4.3e-6 of the largest output apart), so unset keeps the native
path on every card, gfx103x included; =1 enables it on gfx103x devices only (a mixed layer
split keeps the native call on the other cards), =all on every AMD device for A/B arms and
tests.  The buffers are no longer freed under an in-flight GEMM: the stream is synchronised
before the FP32 copies grow, as the sm_60 FP32 route does.  docs/AMD_HIP.md says so.
2026-10-14 07:11:51 +02:00
vallicgrr 6ccdb14167 prefill gemm: a padded BF16 X (ldx, STRATA_PF_PAD) keeps the native call on the RDNA2 SGEMM route 2026-10-14 07:11:04 +02:00
vallicgrr 6dffb74a1f tests: hip_prefill_gemm crosses an RDNA2 SGEMM slice boundary (T 3300, K 10240; FP16 beta 1 with a padded ldy, and BF16) 2026-10-14 07:11:04 +02:00
Niko1221andSanghunYun d7298883b8 Let the resident RAM copy follow decode routing (opt-in)
Port of PR #1838 (commit fb868db01) onto 0.1.44.1. Test file merged with #2134's foresight test (both kept). replace_resident now drops a Foresight slot for the outgoing expert while its RAM place is rewritten (#2134's copies read those places).

Co-authored-by: SanghunYun <[email protected]>
2026-10-14 07:11:02 +02:00
Niko1221andvallicgrr 62a98cd353 docs: RDNA2 prompt GEMMs as SGEMMs (measured on an RX 6800), and a Windows gfx1030 report
Rebased onto 296b59d2: the docs/AMD_HIP.md conflict (the gfx1031/gfx1150/gfx1152 bullets main gained) is resolved by keeping main's text.

Co-authored-by: vallicgrr <[email protected]>
2026-10-14 07:10:58 +02:00
Rafa ad0582c3d4 #2146 tests: pa_bench (the prompt attention on a recorded selection); STRATA_QSA_DUMP_LAYER / _STEPS
STRATA_QSA_DUMP_LAYER=i keeps one QSA layer, STRATA_QSA_DUMP_STEPS=1 appends
the steps to each record.

pa_bench: the prompt sparse attention (qsa_prompt_attn_batch) on a
recorded selection - a STRATA_QSA_DUMP record of one layer and chunk with
its steps appended - random q and int8 K/V; median time and an output
hash for A/B of kernel variants (PA_BENCH_OUT=file writes the output).
2026-10-14 07:10:57 +02:00
Rafa 5d4c786760 #2146 sm_75: v2 prompt attention in one stage, three blocks per SM (bitwise)
On Turing the v2 int8 prompt attention (STRATA_PA_V2T=1) stages the next
chunk in registers, so its second shared-memory stage bought nothing - and
at 45.6 KB a block only one fit an SM (64 KB): four warps on four
schedulers, every latency exposed (ncu: 12.5 % warps active, tensor pipe
8 %).  Smem2T: one K/V stage, and the q.k partials of cell group nt
(8 cells) written over the 512 bytes of the warp's own K slice that held
those cells (read by then, a __syncwarp between) instead of their own
8.4 KB.  19.7 KB a block: three per SM (168 registers fit three).  The
same arithmetic in the same order - bitwise the same output.

RTX 2080 Ti, a recorded 130K selection (QSA layer 5, last chunk: 15871
queries x 2051 cells, tests/cuda/pa_bench.cpp): 113.8 -> 68.5 ms, same
output hash.  Engine (Q2_0 pack, 16K chunks, fused experts): 98K prefill
53.3 -> 49.9 s, 4K 3.54 -> 3.42 s; first logits bitwise identical at 98K
and 4K, the same generated text.
2026-10-14 07:10:57 +02:00
Rafa 0410d62aa9 #2092 sm_75: the v2 int8 prompt attention with register staging instead of cp.async (STRATA_PA_V2T=1)
On Turing the int8-KV prompt attention ran v1 because v2 stages its K/V
slices with cp.async. Here a warp loads its next slice into registers and
stores it into its other stage after the current chunk (the same bytes in
the same places); the MMAs are the m16n8k8 pairs v1 already uses. Accuracy
as v1 (qsa_prompt_attn_parity vs FP64: 3.7e-6 / 4.9e-6 / 4.5e-6 / 3.8e-6 of
3.5-3.7). RTX 2080 Ti, ms per chunk v1 -> v2: 32K x 1024 queries 13.0 -> 7.6,
64K x 512 6.7 -> 4.8, 1.5K 6.2 -> 5.0, 2.1K x 256 3.2 -> 2.3. Opt-in until
the engine A/B.
2026-10-14 07:10:56 +02:00
Rafa 8bdef963cc #2091 sm_75: the hyper-connection xn16 written as the FP16 its tensor-core GEMMs read (bitwise)
With STRATA_BF16_TC the prompt path converted xn16 (T x 10240) from BF16 to
FP16 before each of its two GEMMs (down and inject), in 16M-element slices:
~2.8 s of a 130K prompt read on an RTX 2080 Ti. The four xn16 writers now
store gemm.cu's conversion of the BF16 value (bf_to_hf_conv: the same clamp
and rounding as bf16_to_f16_kernel), and Gemm::bf16(..., x_f16) skips the
conversion on the tensor-core path (same slices, same GEMMs) and converts
back to BF16 on any other. Only when the stage runs BF16 on FP16 tensor
cores, without the BF16X2 low parts or a padded xn16 stride; the MTP draft's
own read keeps BF16. STRATA_XN_F16=0 turns it off.

First logits after a 98K prompt: bitwise identical; prompt read 1431 -> 1473
tok/s; a 4K run writes the same tokens.
2026-10-14 07:10:56 +02:00
imanu d976db3454 #2081 dequant: the prompt projections' BF16 dequant with 16-byte stores on sm_75 (bit-exact; STRATA_TC_SM75=0 keeps the per-group kernel)
The per-group dequant kernel writes 2 bytes every 64 bytes; 0.1.43 coalesced only the Q8_0 path (#1720). On compute
capability 7.5 the BF16 projections of the prompt path take a kernel that writes 16 bytes per thread; the values are
the same (a conversion, no arithmetic), so the output is bit for bit the old kernel's. Other cards are untouched.
2026-10-14 07:10:51 +02:00
Niko1221andvallicgrr 8f949956b2 HIP RDNA2 (gfx103x): run the prompt path's 16-bit GEMMs as FP32 SGEMMs
The rocBLAS the engine ships has tuned gfx1030 kernels for FP16->FP16, int8 and
FP32->FP32 only; every FP16/BF16-input, FP32-output GEMM (rocBLAS HS/BS) runs a
fallback kernel at ~5 TFLOPS. Widening the inputs to FP32 is exact and SGEMM
accumulates in FP32 like the native call.

Nothing passes through FP16, so unlike STRATA_HIP_PROMPT_F16=1 (#835) the
route cannot overflow: an FP16 GEMM output above 65504 becomes inf and can
spread as NaN, while here it stays a finite FP32 value. It is slower than the
FP16 route, and not bit-identical to the native call (at most 4.3e-6 of the
largest output apart on random inputs).

RX 6800, Coder IQ1_M, fresh 12K-token prompts (4 each, A/B in one binary via
STRATA_RDNA2_SGEMM=0): prompt 312 -> 452 tok/s, GPU timeline 37.2 -> 25.5 s
(gdn 9.4 -> 2.8 s, qsa proj 3.6 -> 1.4 s, hc read 3.85 -> 2.84 s). Decode
unchanged. A 9-task code-repair benchmark: 9/9 first try on both, model time
458 -> 374 s.

Default on for gfx103x only; N < 64 keeps the native call; if the FP32
buffers cannot be allocated the native call runs.

Rebased onto 296b59d2: the conflict in Gemm::f16 and gemm.hpp (the CUDA FP16 algorithm fallback and the sm_60 STRATA_DENSE_F32 block that main gained) is resolved by keeping both; the RDNA2 call sits before the CUDA/HIP GemmEx call.

Co-authored-by: vallicgrr <[email protected]>
2026-10-14 07:10:39 +02:00
Rafa d5b80b2e65 #2093 sm_75: the prompt path's top-k with one full read of the scores (identical ids)
block_topk_wide_kernel reads a query's scores six times (four radix passes,
a count, the emit): past ~115K cells that is DRAM traffic. block_topk_sample_kernel
takes a bound from every 32nd block (the KS-th largest of those keys), reads
all blocks once keeping those at or above it, and when they hold at least
`width` cells computes block_topk_kernel's threshold, `above` and tie budget
over them and emits them in block order (bitonic sort): every block at or
above the threshold is a candidate, so the ids are the reference's.
Otherwise the query is flagged and block_topk_wide_kernel<true> runs it.
qsa_topk_parity --selftest and qsa_topk_active_parity: identical ids
(continuous, 64-level, equal scores - the last ones take the fallback).

RTX 2080 Ti, 256 prompt queries under a 512K capacity: 131K 0.51 -> 0.45 ms,
262K 0.95 -> 0.59, 400K 1.48 -> 0.71, 520K 1.84 -> 0.79. Used above 28,672
active blocks; STRATA_TOPK_SAMPLE=0 keeps the wide kernel.
2026-10-14 07:10:32 +02:00
Rafa 050229ecda #2072 sm_75: the decode QSA selection at long context, split top-k and halving scorer (bitwise)
A verify window's 1-8 queries scored and selected on a handful of SMs; at
420K cells on an RTX 2080 Ti the two took ~6 ms of a ~52 ms window.

- block_scores_halve_kernel: the decode scorer's warp sums as one
  recursive-halving reduction (each value pairs lane l with l ^ o at offsets
  16..1, the butterfly's additions operand for operand), the lane's query
  values in registers (shared-memory reads were the limit, ncu mio_throttle),
  5-8 queries as two calls of up to 4. Same bits (qsa_select_bench
  QSA_BENCH_DECODE=1 hashes, 32K-524K, 1-8 queries). 420K: 1 / 3 / 4 / 6 / 8
  queries 117 / 265 / 347 / 510 / 774 -> 101 / 118 / 125 / 238 / 255 us.
  STRATA_SCORES_HALVE=0 keeps the old kernel.
- qsa_block_topk_split: the decode top-k over chunks of 2,048 blocks in six
  launches (four radix passes into per-query global histograms, a count, the
  emit); every CTA takes each digit from the complete histogram with
  block_topk_kernel's rule, so the ids are identical (qsa_topk_parity
  --selftest: continuous, 64-level and equal scores, 2K-524K). Decode calls
  at 420K: 0.21 -> 0.033-0.064 ms; short contexts unchanged within 0.01 ms.
  STRATA_TOPK_SPLIT=0 keeps the wide kernel (=1: on any card, for tests).

Both on sm_75 only by default.
2026-10-14 07:10:28 +02:00
Rafa 3f691eb345 #2071 sm_75: STRATA_SELECT_SIMT=2, the tiled FP32 block scorer with 32 x 128 tiles
block_scores_simt2_kernel: 8 x 8 products per thread, twice the blocks per
CTA and the next K chunk in registers; every dot is the same fmaf chain over
k = 0..127 and the same relu sum, so its scores are bitwise those of
STRATA_SELECT_SIMT=1 (qsa_select_bench QSA_BENCH_HASH=1, 131K-510K).
RTX 2080 Ti, 256 queries: 131K 1.21 -> 0.95 ms, 262K 2.01 -> 1.87,
400K 3.00 -> 2.82, 510K 3.77 -> 3.57 (the warp kernel: 7.7 / 15.3 / 23.2 /
29.6 ms). Opt-in like =1 (another summation order than the warp kernel).
2026-10-14 07:10:27 +02:00
Breno Perucchi 653f011360 prefill: gather experts in groups on short prompts too (resident and staged)
Below stream_all_min() the MMQ path gathered every expert with its own launch
and, for a staged one, its own wait and release event. The staged walk now
groups like the streamed walk: its ring is STAGE_GRP (32) slots, two groups, and
it stages at most 15 entries ahead, so a group's staged slots are not refilled
before the group is gathered (one wait on its last copy, one event releasing
them all). stage_slots() counts those 32 slots wherever the ring is counted or
taken, so a prompt borrows every byte it writes.
STRATA_PREFILL_GROUP_GATHER=0 turns grouping off everywhere, as before.
2026-10-14 07:10:25 +02:00
Rafa 80872d4349 #2070 sm_75: hc up kernel with batched row loads for 3+ tokens, down token pairs for T 4/6
gr_up75_kernel: two batches of 4 rows of w_up loaded before the dots, the
token activations read once per batch, the 4 xor reductions interleaved and
the row tails held 4 rows to a uint4 - bitwise gr_up_multi_kernel (gr_parity,
gr_multi_parity, fused_gr_bench hashes). Used from 3 tokens on sm_75
(STRATA_GR_UP75). The staged down kernel takes a token group from
blockIdx.y; sm_75 runs T 4 and 6 as pairs (STRATA_GR_DOWN_TG).

fused_gr_bench cold, RTX 2080 Ti, whole read: T3 53.2 -> 50.2 us,
T4 64.1 -> 57.6, T5 76.2 -> 69.3, T6 86.1 -> 70.9; T1-2 unchanged.
2026-10-14 07:10:24 +02:00
Rafa 26855986e1 #2069 sm_75: grouped native experts default to mode 4 (down: mode 0 in windows of 4+ tokens)
native_expert_bench on real Q2_0 rows of Flash-Next, RTX 2080 Ti, bitwise
equal to mode 0 in every case: 10 groups x 1 entry 47.3 -> 41.4 us,
20 x 1 83.7 -> 71.5, 30 x 1 120.2 -> 100.2, 20 x 2 124.2 -> 111.4. The down
projection of mode 4 loses with 3+ entries per group (12 x 3 36.5 -> 40.8,
16 x 4 56.9 -> 68.9), so cap_groups > 30 keeps mode 0 for it.
STRATA_SM75_EXP=0 keeps the CUDA layout; STRATA_EXP_MODE still overrides.
2026-10-14 07:10:23 +02:00
Niko1221andsergiywith 3f2db5e2c5 decode: the adaptive tier's evictions reach the device's residency table before the next window when the tier does not wait (STRATA_ADAPT_NOWAIT=1, STRATA_ADAPT_LAG > 1)
adapt() marks an evicted expert non-resident in host_res at once; d_res follows in apply_pending, once the round's
copies have landed.  When the tier does not wait for them (STRATA_ADAPT_NOWAIT=1, or STRATA_ADAPT_LAG > 1) the next
window runs before that.  doorbell_publish_res reads d_res, sees the evicted expert as resident, and when every routed
expert of a layer looks resident it does not publish the layer's activation.  The pool plans from host_res and
computes the evicted expert on the CPU from the activation published before.  Now every verifier launch (run,
run_slot_rows, batch_launch, pl_launch; a layer split's stages too) runs a pre-launch hook.  It patches the device
tables from a list of changed entries: res_patch, a small kernel on the verifier's stream reading mapped memory; the
whole table when the list overflows or a device does not map it at the host's address.  A failed whole-table upload in
apply_pending leaves its entries to the hook.  Only where the tier does not wait.  --batch-groups with a layer split
publishes every activation instead: set_always_publish, which now also turns E-6's device plan off.
--pipeline-windows keeps its fence.  STRATA_ADAPT_EVICT_SYNC=0 restores the old behaviour.  STRATA_ADAPT_CHECKRES=1
reads the tables back before each window, compares them with host_res, and counts the layer-windows that computed a
CPU expert with no activation published.

RTX 5090, IQ2_XS, --expert-cache 12000, 32K explain, 600 tokens: NOWAIT=1 34 of 222 windows with mismatched tables
(2470 entries), 1 such layer-window -> 0 / 0 / 0; LAG=2 55 of 223 (3222 entries: every swap), 5 -> 0 / 0 / 0.  LAG=2
is reproducible (two runs of each arm, the same tokens), and the fix changes its answer from token 228.  The hook's
cost under NOWAIT (6297 slots, 3 pairs): ms a round -1.1% chat, -1.2% 32K, noise.  The default (the tier waits)
installs no hook: tokens and first-token logits identical to d5ea7133 (p2k 200 tokens, 32K explain 600).

Port of PR #1517 (commit eb4796045) onto 0.1.44.1. Conflicts with the 0.1.43 adapt overlap gate and the tail-skip kind -2 resolved; set_adapt_gate and set_always_publish kept side by side. Docs line added.

Co-authored-by: sergiywith <[email protected]>
2026-10-14 07:10:19 +02:00
Niko1221 80bed76c8a tools/test_setup_mac: the --source test no longer runs setup.data_folder() for real
The real data_folder() moves the model files of this checkout and of the data folder named in the user's settings.json into --data-dir and saves that folder as the new data folder. With a TemporaryDirectory the models are deleted on exit and settings.json is left pointing at a deleted directory (found on the M4 and on the PC: the test wrote %APPDATA%/Strata/settings.json and ~/.config/strata/settings.json). The test is about --source, so data_folder is mocked.
2026-10-14 07:10:16 +02:00
Rafa 004efd4606 #2069 sm_75: interleaved small-batch matvec table for Turing (kIlRows_75)
The 2-4 column dense projections of the verify windows ran the plain multi-column kernels on Turing, which are
bound by dp4a throughput there from 2 columns on. mmvq_il_parity --bench --emit-table on an RTX 2080 Ti (SM 1650
MHz) gives a table like the Volta/Pascal ones: IQ4_XS 2560x10240 at 4 columns 81.1 -> 37.4 us, Q4_K 74.5 -> 39.1,
the Q5_K head 1671 -> 1045 us. Bitwise equal (the decode writes the same tokens). No one-column table: the
engine's dedicated single-column kernels sum in another order, for a 0.5% gain. STRATA_SM75_IL=0 turns it off.

Q2_0 512K resident, 4K prompt: one-shot decode 52.6 -> 55.5 tok/s; serve steady state ~69 -> ~77 tok/s.
2026-10-14 07:10:01 +02:00
sergiywith af0e9d3c1b prompt attention: STRATA_PROMPT_ATTN_IMMA=1 - the int8-KV kernel on INT8 tensor cores (opt-in, closer to FP32)
prompt_attn_i8v3_kernel: QK^T and PV as mma m16n8k32 on the int8 K / V codes as stored; the other operand in three
exact 8-bit parts (q scaled per head and 64-dim group, p * v_scale per row and chunk; X = rint(x * 65536) split into
h, l signed and m unsigned bytes, a u8 x s8 MMA): 24 bits, exact int32 sums, combined in FP32.  One K and one V stage
per warp, 12-row partials, launch bounds (128, 3): ~25.5 KB, 3 blocks an SM.  sm_80+ (STRATA_PA_SM80); Turing,
Volta and HIP keep their kernels.  Unset: v2, byte-identical.

qsa_prompt_attn_parity: against FP64 IMMA 2.15e-6 - 2.64e-6, v2 2.78e-6 - 3.30e-6, the FP32 kernel 1.90e-6 - 2.51e-6.
IQ2_XS, first-token KL to the FP32 kernel (STRATA_PROMPT_ATTN_OLD=1): median 0.0027 (IMMA) against 0.013 (v2);
2K 0.0026 against 0.0248.  1.35-1.53x a launch; a 32K prompt in 8K chunks 358 -> 312 ms of attention.
2026-10-14 07:09:47 +02:00
Niko1221 f288cbfea9 F16_RANGE counters (#1149): HIP only, and the gfx906 compat layer maps cudaMemcpyFromSymbol
Fix-up to the ported #1149 commits.  The range counters and the saturating to_f16
are compiled for HIP only, so the CUDA kernels (hf_sat in the SwiGLU product, the
embedding image) stay exactly as they were, with no flag read; set_f16_range /
f16_range_read are no-ops there.  strata_hip.h (the STRATA_HIP_GFX906 compat layer,
which goes first on that backend's include path) gets the cudaMemcpyFromSymbol
mapping the HIP shim already has, so the counters also compile for gfx906.
2026-10-14 07:09:41 +02:00
Niko1221andsergiywith 04be241a5a decode: STRATA_PCIE_BALANCE=1 picks each layer's PCIe count from measured costs
The verify window reads pcie_frac of a layer's missed experts over PCIe, a share
the link probe sets; the CPU's speed does not enter it.  With a slow CPU the GPU
spins on the pool's rows: IQ2_XS with one AVX2 pool worker (emulated) runs
16.8-17.2 ms a round at the probe's 0.55.

Opt-in, STRATA_PCIE_BALANCE=1 (one GPU, no batch slots, no --pcie-frac):
- wait_flag_ge_stamped: the flag spin kernels also write the GPU clock as the
  plan arrives, as the PCIe part starts and as the GPU waits for the CPU's rows
  (no extra launch; captured only with the switch on);
- the host times its pool per step; after each window, least squares with
  forgetting fit the pool's a + c n_cpu + d n_pcie (d: the link's DRAM reads
  slowing the pool - on a DRAM-bound pool moving experts to PCIe saves nothing)
  and the GPU's g0 + g n_pcie;
- a layer then reads over PCIe the m of its misses that minimizes the longer
  side; every 64th layer that would keep them all on one side moves one, so
  both fits stay measured.

IQ2_XS, RTX 5090 + 9950X3D, chat to the end of the turn:
- one AVX2 pool worker: 16.77 / 17.17 -> 12.98 / 12.82 ms a round
  (fit: CPU 42-60 us an expert, GPU 28 us a PCIe expert; 0.8 of the misses
  move);
- the 9950X3D's 16 workers: 3 pairs 12.19 -> 12.17 (pool DRAM-bound, few move).
Off (the default) the logits and 32 tokens equal v0.1.39 (2K, fixed cache).

Port of PR #1548 (commit a513394c4) onto 0.1.44.1. Conflicts with the 0.1.43 tail skip, PCIE_MIN1 and the fused waits resolved: the fused wait+copy launches stay off while the balance stamps are on. Docs line added.

Co-authored-by: sergiywith <[email protected]>
2026-10-14 07:09:32 +02:00
sergiywith cffe31d43e prefill: opt-in STRATA_HC_UPMIX=1 on CUDA - the hyper-connection up projection with gr_mix_r as its epilogue (sm_80+)
The up GEMM of each hyper-connection read writes `gated` (T x 10240 FP32, 336 MB at an 8K chunk) and gr_mix_r reads
it back with R at ~96% of the bandwidth - only fewer bytes help.  gr_upmix_cuda_kernel (128 tokens x 32 columns x 4
streams a block, K = 320 in cp.async-staged slices of 32, BF16 mma.m16n8k16 with FP32 accumulation, the mix in
registers in gr_mix_r's order writing all its outputs) never writes or reads `gated`.  Behind the gfx11 switch;
sm_80+ per device (STRATA_EMULATE_CC honoured), chunks of >= 33 tokens; else the pair.  Unset: unchanged.

gr_upmix_parity (new ctest): bitwise the cuBLAS + gr_mix_r pair at T >= 33 on an RTX 5090 (cuBLAS 13.3), <= 1.8e-7 at
T <= 32, both as close to FP64.  1.42x / 1.81x / 2.19x the pair at 2K / 8K / 32K; IQ2_XS: an 8K prompt -4.4%, 32K
-3.9% (the hc read -19% / -24%).  Logits identical with it on (600, 2K, 32K); the 32K greedy tokens moved only through
the time-driven suffix-draft policy (identical with --suffix-draft 0).
2026-10-14 07:09:30 +02:00
Niko1221andsergiywith c675f5b787 prefill: quantize each token's MoE input once and scatter it to its k rows (the same bytes)
mmq::quantize(m.mixed, m.src_dev, ...) read and quantized every token's row of `mixed` once per routed expert (K =
10).  mmq::quantize_scatter uses llama.cpp's quantize_scatter_mmq_q8_1_cuda (the pinned quantize.cu strata_mmq
already compiles): each token quantized once, its block written to all of its rows through slot_dev, the inverse of
src_dev.  The same q8_1 bytes, so the default; STRATA_QUANT_GATHER=1 keeps the gather.  No new device code (CUDA and
HIP compile the same kernels); SYCL's prefill.cpp untouched.

prefill_mmq_scatter_test (new ctest): every MMQ q8_1 layout (D4, DS4, D2S6), T = 1..2099: 0 differing bytes.  IQ2_XS
600 / 2K / 32K and Q2_0 2K against e8ca9afd: identical first-token logits and 32 greedy tokens.  RTX 5090 (96 MB L2):
the quantizer 2x at 2K, even at 8K (the chunk's input fits in L2), 1.4-1.9x at 32K; a 32K prompt with
--prefill auto:32768: 77.5 -> 58.1 ms of quantization.

Co-authored-by: sergiywith <[email protected]>
2026-10-14 07:09:21 +02:00
ruibeikaa a87fb80e13 sm_60: stage the IQ3 tables by default on compute capability 6.0 only
The staged IQ3_XXS / IQ3_S tables trade a per-block copy and barrier for
cheaper lookups, which pays only where the grouped expert kernels are
decode-bound. On GP100 (6.0: no __dp4a, 70-140 GB/s) it does; on an RTX 3090
Ti (sm_86), forced on with STRATA_IQ_STAGE_TABLES=1, the same real Swift 1.5
IQ3_S layers (20 experts, 24 entries) took 3-13% longer (71-76 -> 75-84 us;
those kernels read at 490-665 GB/s there) and IQ3_XXS was unchanged, the
output bitwise the same. 6.1 cards (P40, GTX 10) have __dp4a and are
unmeasured, so the default now asks for 6.0 rather than any 6.x;
STRATA_IQ_STAGE_TABLES=1 still turns it on anywhere.

docs/OLDER_GPUS.md says so, and that neither STRATA_Q8_SM60 nor the tables
is measured on 6.1: both build for sm_61, and tools/q8_sm60_check.cu passes
on an RTX 3090 Ti, where the kernel's dot is the hardware __dp4a.
2026-10-14 07:09:00 +02:00
ruibeikaa ac35eebf30 sm_60: stage IQ3_S's codebook and sign masks in shared memory too
IQ3_S was the one i-quant gate/up format whose codebook the grouped expert
kernels still read from global memory (IQ2_XXS / IQ2_XS / IQ2_S stage theirs
everywhere, IQ3_XXS on 6.x since the first commit of this branch). Its 512
codebook words are now staged per block with two 256-word tables of byte
masks - IQ3_S keeps one explicit sign byte per 8 values, so the tables hold
the masks of a sign byte's bits 0-3 and 4-7 - applied as
(g ^ mask) + (mask & 0x01010101): every iq3s_grid byte is odd, so a negated
byte never carries into the next. Bitwise the same output (iq_multi_parity
and native_grouped_parity on a GP100 die, with and without
STRATA_IQ_STAGE_GRID_MMVQ=1).

One GP100 die, a verify window's layer of real Swift 1.5 IQ3_S experts
(20 experts, 24 entries): 517 -> 393 us with Q2_0 down, 492 -> 369 us with
IQ4_NL down. Swift 1.5 IQ3_XXS (13 of 48 layers IQ3_S), four GP100 dies,
1M context, decode tok/s, medians of three interleaved engine starts:
36.0 -> 37.2 (10.7K document), 36.3 -> 37.4 (cached), 43.1 -> 44.5
(reasoning), 41.6 -> 43.0 (code), with the same drafts accepted.

The switch covers both formats and is renamed accordingly:
stage_tables_on(), STRATA_IQ_STAGE_TABLES=1|0 (was STRATA_IQ_STAGE_GRID18).
Still the default on compute capability 6.x only.
2026-10-14 07:08:59 +02:00
ruibeikaa 080abf86a4 tools/q8_dense_gguf.py: convert every shard that holds dense matrices
The tool converted only the shard it was given and hard-linked the others, which is enough for Flash-Next (every
natively served projection and the head are in shard 1) but not for Swift 1.5 IQ3_XXS, which keeps 78 of them in
shard 1 and 223 in shard 2: one run left three quarters of them in their K- / i-quants, silently, and the kernel
only runs Q8_0 matrices. Now any shard of the model may be named; every shard that holds such a matrix is written
as before (same tensor-info length, Q8_0 blocks appended, offsets checked) and the others are hard-linked.

Checked: Swift 1.5 IQ3_XXS in one run gives both shards byte-identical to two single-shard runs (301 tensors);
Flash-Next IQ3_XXS converts 301 tensors in shard 1 and links shard 2.
2026-10-14 07:08:59 +02:00
Niko1221andruibeikaa 6e29a5d314 sm_60: a Q8_0 decode GEMV for Pascal GP100 (STRATA_Q8_SM60=1, opt-in)
GP100 has no __dp4a, and on a GP100 die the verify window's dense projections read their weights at 40-80 GB/s
of ~730. With STRATA_Q8_SM60=1 every Q8_0 dense matrix (any shape) and the head get the packed planes of
STRATA_Q8_PACKED and run src/kernels/cuda/q8_sm60.cuh: a warp (half / quarter warp for narrow rows) per row,
16-byte weight loads through the read-only cache, the window's q8_1 activations copied once per workgroup into
shared memory as an int8 plane plus float scales, STRATA_DP4A (vmad on sm_60), a shuffle sum with no barrier per
row, persistent workgroups (4 per SM, STRATA_Q8_SM60_BPS). Shapes it declines (> 48 KiB of shared memory: a
6144-wide row with 7-8 columns) keep the usual kernel. Not bitwise the exact kernels (summation order); the
packed copies cost their size in VRAM (Flash-Next: 2.9 GiB of dense projections over the stages, 0.6 GiB head).

The GSQ-RCO packs have no Q8_0 dense matrices: tools/q8_dense_gguf.py writes a shard 1 whose natively served
projections and head are Q8_0, keeping the tensor-info block's length so every expert keeps its offset (the native
pack is reused) and hard-linking the other shards beside it.

PH402 SKU 200 (four GP100 dies), clocks locked at 1050 MHz, Flash-Next IQ3_XXS, 32K, layer split 12,24,36,
--prefill 2048, greedy, decode tok/s (medians, interleaved starts): GSQ-RCO shard 28.3, Q8_0 shard with
STRATA_Q8_PACKED 31.1, Q8_0 shard with STRATA_Q8_SM60 38.9 (+37%); per window dense 19.2 -> 11.2 ms, head
4.25 -> 2.1 ms; prompt reads unchanged. tools/q8_sm60_check.cu: 57 shape x column cases against a CPU reference,
max relative error < 1e-5. docs/OLDER_GPUS.md has the table, the conditions and how to use it.

Co-authored-by: ruibeikaa <[email protected]>
2026-10-14 07:08:58 +02:00
Niko1221 f01b989264 sycl: mmvq_bench checks the STRATA_Q6_PLANAR copy and a gathered subset bitwise against the GGUF-layout result (#2340) 2026-10-14 07:08:52 +02:00
Francesco Albano 051b970303 cpu pool: --host-core sibling, the host on the first core's other hardware thread (hybrid CPUs too)
Windows sends a GPU's interrupts to one logical processor, usually the first,
and the host thread spins there (--host-core first, the default). --host-core
last moves it to the last physical core, but not on a hybrid CPU, whose first
core is its best P-core, so a 12th-14th gen Intel or a Core Ultra keeps the
host on logical processor 0 with the interrupts.

--host-core sibling (or STRATA_HOST_CORE=sibling) keeps the host on that same
first core but on its other hardware thread. The interrupts keep the first
logical processor, which nothing else is pinned to, and the workers keep
every other core, since they take one logical processor per core (with
--pool-affinity auto / p-cores, which also use the P-cores' siblings, the
host's sibling is taken out of their list). Without an SMT sibling it is
--host-core first. It works the same on Linux (the sibling from sysfs), where
the interrupts are usually spread anyway.

detect_cpu_topology records the host core's sibling (CpuTopology::host_sibling)
in each layout; pool_affinity_test checks, in every affinity mode, that
sibling puts the host there with no worker on its core, and that it is first
without SMT. The startup log names the placement.
2026-10-14 07:08:35 +02:00
xjc10 c211cb3ebe hip gfx103x FP16 prompt: STRATA_F16_RANGE counts what enters and leaves FP16; the range measured (outputs peak at 410 of 65504) and the FP32 route compared
The README's range check said "none beyond 65504" over 110K tokens without
the numbers. The engine now has the counter: STRATA_F16_RANGE=1 records,
per device, the largest |value| and the count beyond 65504 (or not finite)
where a value enters or leaves FP16 - the activation images (hf_sat, the
image writers' one funnel), the BF16 weights converted to FP16
(bf16_to_f16_rows) and the FP16 GEMM outputs as they are widened
(widen_rows_f16: an output rocBLAS wrote beyond the range is Inf there) -
and prints one line per prompt. Off, the cost is one flag read per value;
on, a 34.4K-token prompt read at 889.8 tok/s against 888.5. to_f16 (the
hc image for the FP16-weight GEMMs, the PLE key image) saturates like the
other image writers now, so every FP16 image is under the same rule. The
HIP shim gains cudaMemcpyFromSymbol.

Measured on one RX 6900 XT over 466K tokens in nine prompts (corpus
slices, two prompts built to push activations - delimiter floods, a token
repeated 4,000 times, 6,000-character runs, base64, five scripts - and
four batches of real requests): outputs peak at 409.5 (160x below 65504),
activations at 112.2, the BF16 weights at 11.56; nothing beyond the range,
nothing non-finite. The prompts built to push activations stayed below
the ordinary corpus. One model (GSQ-RCO IQ3_S); the table is in the
README's range section.

The README also gains the same-card comparison with #1006, which takes
these GEMMs as FP32 SGEMMs: 8.4K / 32.8K / 104K-token prompts 412 / 455 /
456 tok/s on the old path, 622 / 742 / 750 with #1006, 730 / 900 / 904
with this change (the card's FP32 rate against its FP16 rate), with the
phase times and what the SGEMM route covers that this change does not.

(cherry picked from commit e780c8581e)
2026-10-14 07:08:33 +02:00
xjc10 d0838c6c3a tests: hip_prefill_gemm runs the FP16-io prompt GEMMs (set_f16_io) on any HIP card, and its BF16 inputs are no longer zero
The test ran only the FP32-out GEMMs, so the gfx103x prompt path's FP16-out
GEMM (f16_inplace, widen_rows_f16) and bf16()'s W conversion through the
scratch were not under ctest. Every shape now runs a second time with
set_f16_io(true), forced on whatever HIP card runs the test: X as the FP16
image for the BF16 products, a scratch of 64 rows so a 96-row weight converts
in two slices (the second written at Y + 64), ldy > N, an odd N (257), T = 1,
and a beta = 1 f16() call, which keeps the FP32-out GEMM. FP16-out results are
checked against the double reference at FP16's rounding (6e-4 relative per
element plus 1e-4, rel L2 < 1e-3); the row padding and the guards around Y
must be untouched, as before. The FP32-out runs keep their tolerance.

Found while adding them: the BF16 inputs were all zero. hip_bfloat16(float)
produced data 0 in this test's build (ROCm 10.0, gfx1030; not traced further),
so the two BF16 cases passed on nothing (rel L2 printed 0). The BF16 bits are
now made by hand (round to nearest even on the float's bits); with real inputs
the BF16 cases pass at rel L2 1.9e-07 (FP32 out) and 2.2e-04 (FP16 out).

On an RX 6900 XT (gfx1030, ROCm 10.0): 11 of 11 pass.

(cherry picked from commit 4c6c9fe374)
2026-10-14 07:08:32 +02:00
libratechw b6ccf873b8 prefill: the register top-k from 16K cells of context on gfx1151 (STRATA_TOPK_REG_MIN_BLOCKS, gfx1151's table sets 4096)
On HIP a prompt batch with fewer than 7,168 blocks (28K cells of context) took the original top-k kernel, a line measured on
gfx1201 before the cooperative scan.  With the scan the register kernel is faster from about 15.5K cells on gfx1151 (2,048 queries:
original / register kernel 0.57 / 0.82 ms at 13K, 1.01 / 0.87 at 16K, 2.4 / 0.93 at 20K, 3.9 / 1.00 at 24K).  The line is now
STRATA_TOPK_REG_MIN_BLOCKS (default 7168, as before) and gfx1151's table sets 4096.  The selected cells are the same
(qsa_topk_parity, also run with the register kernel always on: ctest qsa_topk_parity_reg_always).
2026-10-14 07:08:28 +02:00
0FL01 557b79e78f test: cover exact multi-row and fused HIP paths on gfx906 2026-10-14 07:08:27 +02:00
ruibeikaa 96cb96aced sm_60: stage IQ3_XXS's codebook and sign masks in shared memory (grouped expert kernels)
Ported from shinbunbun/llama-cpp-p100-patches (MIT), patches 29 `mmvq-iq3xxs-grid-smem` and 30
`mmvq-ksigns-smem`: the grouped expert kernels stage the 256-word iq3xxs_grid and two 128-word tables of per-byte
sign masks once per block, and apply a mask as (g ^ mask) + (mask & 0x01010101) instead of the per-lookup
unpack_ksigns / __vcmpne4 / __vsub4 chain (iq3xxs_grid's bytes are all >= 4, so no carry crosses a byte).

Bitwise the same output: checked end to end on a PH402 (GP100), greedy, three prompts x three interleaved rounds,
identical replies with and without. Measured only on GP100, where it is within noise (+0-4% decode), so it is the
default on compute capability 6.x only; STRATA_IQ_STAGE_GRID18=1 / 0 forces it on / off on any card.
2026-10-14 07:08:24 +02:00
Warabii1248 1327c0916c fused_gr: say in the comment that CUDA opts in to the register split norm 2026-10-14 07:08:24 +02:00
libratechw 6ef23b8966 prefill: the QSA top-k digit by a cooperative scan (same ids)
block_topk_reg_kernel picked each radix digit with thread 0 walking up to 255 bins while the other threads waited.
Threads 0-255 now prefix-sum the bins in descending order, and the one where the count above reaches `width` takes the
digit (bin 0 when none of 255..1 does). The digit and the count above are the same, so the selected ids are the same.

Measured on a Ryzen AI Max+ 395 (gfx1151, ROCm 7.14.1), qsa_topk_parity with 2,048 queries (CTX 2048 10 <ctx> 1),
ms per call:
  32K  2.19 -> 1.11 (-49%)
  64K  2.54 -> 1.63 (-36%)
  128K 4.83 -> 3.79 (-22%)
qsa_topk_parity --selftest: 19 PASS. UD-Q4_K_XL end to end, 3 interleaved rounds: the same text in every round; the
128K prompt moved +0.2%, inside the noise, so the gain is the kernel's and not yet visible end to end.

CUDA (RTX 3060, Windows, CUDA 13.0, sm_86), same call, ms: 4K 2.12 -> 0.975, 8K 2.04 -> 1.03, 16K ~1.9 -> ~1.1, 32K ~2.0 -> 1.27, 64K 2.53 -> 2.15, 128K 7.29 -> 7.29 (no change); qsa_topk_parity --selftest passes, ids identical in every case. Not built: SYCL.
2026-10-14 07:08:23 +02:00
Niko1221andnekomario28 719b733e01 dq_q2_0: keep the Q2_0 signed-zero fix on every HIP arch/version (#1540, #1474)
The negative-scale-times-zero workaround was gated to an arch list (gfx1012 on
HIP < 7, gfx1151, gfx1200/gfx1201 since 0ff45cffb); the list only ever chased
the newest report. The workaround writes the scale's sign bit for code 1, which
is value-identical on targets whose compiler never folds: keep it on every HIP
build. Ported from PR #1540 onto the tree that already carries the gfx1200/1201
entries; the comment text is adjusted.

Co-authored-by: nekomario28 <[email protected]>
2026-10-14 07:08:23 +02:00
Warabii1248 fec7d1cae2 fused_gr: STRATA_GR_NORM_FAST=1 on CUDA too (opt-in; the register split norm is bitwise the default; decode +0.5-0.6% on a V100 + PG500 split) 2026-10-14 07:08:23 +02:00
Warabii1248 ab7c977216 verify: say in the comments that sm_7x takes argmax_rows with STRATA_ARGMAX_ROWS_SM70=1 2026-10-14 07:08:23 +02:00
Warabii1248 83081577e5 sm_70: the multi-block greedy pick (argmax_rows) as an opt-in, STRATA_ARGMAX_ROWS_SM70=1 (bitwise the one-block pick; decode +0.7-0.8% on a V100 + PG500 split) 2026-10-14 07:08:22 +02:00
Warabii1248 ab5ea838de pf_wmma: say in the comment where the WMMA2 tile layout comes from 2026-10-14 07:08:12 +02:00
Warabii1248 dca6a6532f pf_wmma: STRATA_PF_WMMA2=1 (opt-in), a second Volta prompt-expert kernel for IQ2_S gate/up and IQ4_NL down - the dequantization spread over the whole block from a raw stage in shared memory (bitwise dq_iq2_s / dq_iq4_nl; prompt +2.5-4.3% with the IQ3_XXS pack on a V100 + PG500 split) 2026-10-14 07:08:12 +02:00
Warabii1248 4dc3206402 pf_wmma: set the tiles shared-memory attribute on each card, not once per process (a layer split stopped on its second card: "the WMMA expert tiles could not launch") 2026-10-14 07:08:11 +02:00
Warabii1248 caa3c8425a pf_wmma_parity: run on every sm_70+ card in one process, in turn (a layer split launches the tiles on its second card after the first) 2026-10-14 07:08:11 +02:00
g0 2d9b1b348a pf_wmma: take the FP16 prompt path for IQ4_NL down too (mmq_dt == 20) 2026-10-14 07:08:10 +02:00
g0 cb73a0538a pf_wmma: IQ4_NL (type 20) WK=128 lane for the down product (K=640); parity cases 2026-10-14 07:08:10 +02:00
NoxestNoz 140f011030 bench/results: V100 prompt experts on FP16 tensor cores (+9.6% / +10.3% prompt speed); docs/NVIDIA_V100.md 2026-10-14 07:08:09 +02:00
NoxestNoz fb433c3a57 pf_wmma_parity: gemm_iq_f16_grouped against a double reference over the FP16 path's weights 2026-10-14 07:08:09 +02:00
Niko1221 cb2740a8c2 prefill: Volta runs the prompt experts on FP16 tensor-core tiles, a group per launch (STRATA_PF_WMMA)
gemm_iq_f16_grouped: one launch per MMQ expert group, a block = 128 weight rows of one
expert x up to 256 of its token rows, each weight superblock dequantized once into shared
memory by dq_dispatch (the FP16 path's formulas), FP16 activations, WMMA with FP32 sums,
the next activation tile prefetched into registers.  sm_70 defaults to it for the layers
MMQ takes and for IQ1_M (which rides the MMQ grouping); STRATA_PF_WMMA=0 keeps MMQ.

Co-authored-by: NoxestNoz <[email protected]>
(merge-resolved onto 0.1.44.1: the MMQ gather now zeroes the group tail itself and the quantizer takes a scale buffer)
2026-10-14 07:08:08 +02:00
Xiaomi2008 1ab0fbd02a --pool-affinity measure: on a hybrid CPU, time the all and p-cores pool layouts at start and take the faster (opt-in)
The pool's default layout puts a worker on every physical core, and a layer's
experts are done when the slowest worker is. On a Core i7-13700K (8 P + 8 E
cores) the E-cores make the verify window wait, by an amount that changes from
run to run.

`--pool-affinity measure` (hybrid CPUs, native packs) times both layouts on
the pack's own experts before the pool is built: two experts of four tokens
per layer through run_split_multi_native, every layer in turn, the host
spinning ~150 us between two layers, three interleaved blocks per layout,
about a second in all. p-cores is taken unless all is more than 10% faster.
One line says what it read and took. It is one reading at start and is not
repeated.

Measured on this branch (0.1.42 + this change), Core i7-13700K, RTX 5090,
64 GB, Windows 11, Swift 1.5 IQ3_XXS, 128K context, one 8K prompt and the
same 1,500 forced tokens (--spec-follow), three interleaved rounds:
  default (all, 15 workers)   146.6 / 151.8 / 158.0 tok/s, pool 6.3-7.6 ms per window
  --pool-affinity measure     200.0 / 201.5 / 204.8 tok/s, pool 2.1-2.2 ms (read all at 16.4-17.6 ms, took p-cores)
  --pool-affinity p-cores     200.3 / 203.9 / 204.9 tok/s, pool 2.1 ms
Over four server starts of one day on 0.1.41 it read all at 14.8, 21.9 and
25.3 ms against 6.1-6.2 ms for p-cores, and once all at 6.6 ms against 8.6 ms,
where it took all.

Without the value nothing new runs: the option parses as before and the
function is not called. The layout changes no result, only where threads run.
Built for CUDA only (sm_120, CUDA 13.0.88, MSVC 14.50 with nvcc's
--allow-unsupported-compiler); HIP and SYCL were not compiled (the change is
in generate.cpp, which both build). Not measured: other hybrid CPUs, Linux.
2026-10-14 07:08:06 +02:00
aangelinsf fde1d795ef SYCL verify: batch windows keep the doorbell nodes under the lean window
0.1.43 made the decode window lean under STRATA_VERIFY_NO_HOST with the device
plan (46a77907): the doorbell publish, flag waits and plan copy are dropped
because no host answers them. capture_batch() records batch windows through the
same record_window(), so a batch graph was also recorded lean, but the host batch
path (run_slot_rows / batch_poll) still waits for each layer's doorbell unless the
stage is all-resident. On a stage with mirrored experts the graph finished with
the ring silent: "verify batch: layer 0 never rang (graph finished)", then
UR_RESULT_ERROR_UNINITIALIZED and exit 139 at the first shared window.

Leave lean off while recording a batch window (batch_rec_). The solo window
stays lean. Batch tokens are identical to a run with STRATA_VERIFY_LEAN=00.
2026-10-14 07:07:51 +02:00
W1nge ccba2d5fd3 core: write reduced helper output directly to mapped host memory 2026-10-14 07:07:49 +02:00
Zack ea08bd6e81 decode: size --lookup-chain's window cap from the cap in force, not the stale mtp_max_t
With the default --suffix-draft 3 the suffix block above already raises --spec by 2
(4 -> 6) and writes mtp_max_t = 4. The --lookup-chain block then computed
min(mtp_max_t + K, 8) = min(4 + K, 8), so with --spec 4:

  K=1 -> max(6, 5) = 6   (unchanged: the flag does nothing)
  K=2 -> max(6, 6) = 6   (unchanged: the flag does nothing)
  K=3 -> max(6, 7) = 7

Measured on 0.1.40 by reading the cap back from /metrics engine.spec after a restart
per row; K=1 and K=2 were indistinguishable from not passing the flag.

Size from the cap actually in force instead, so K adds up to K of headroom while the
kVerifyMaxT = 8 ceiling and the mtp_max_t bound on the MTP's own drafts are preserved.
Behavior is unchanged when --suffix-draft is 0 or when the cap is already at 8.
2026-10-14 07:07:36 +02:00
yjamil8 c41c0ebbd1 KV streaming under WSL: pin the K/V host copy with cudaHostRegister
Under WSL2 the NVIDIA driver caps what cudaHostAlloc(Mapped) can pin at about
1 GiB in all, so the KV streaming host copy (layer.cpp) cannot be allocated
and setup turns streaming off there: the whole KV cache must sit in VRAM,
beside the expert cache. cudaHostRegister of ordinary memory is not capped the
same way; the expert arena already registers tens of GiB with it on the same
machines (pinned.cu).

kv_host_alloc() maps anonymous memory and registers it Portable|Mapped, before
any page is touched, when /dev/dxg exists (WSL2, detected as generate.cpp's
under_wddm() does). Elsewhere cudaHostAlloc stays first. Either way the other
method is the fallback when the first is refused, and
STRATA_KV_HOST_REGISTER=1/0 forces the order. On native Linux and Windows the
behaviour is unchanged unless cudaHostAlloc is refused (by inspection; not run
there). The failure message no longer says WSL pins only 1 GiB.

Measured on one RTX 5090, Ryzen 9 9950X3D, WSL2 (157 GiB), Windows driver
591.86, IQ3_S, --max-context 1048576 --rope-scaling yarn --rope-scale 4
--kv int8 --kv-resident 32768, with this commit:
- start: "the K/V in 12.38 GiB of pinned RAM"; expert cache 11,609 slots
- tools/needle_bench.py: 4/4 found, 128K and 1M (1,015,070 prompt tokens) at
  depths 10 and 90
- decode 99-124 tok/s with 0.5-1M tokens in context (4 runs); 150-191 tok/s
  medians from 55-token prompts to 128K (single runs 113-196); prefill about
  4,200 tok/s for a cold 1M prompt
kv_stream_parity still passes; it allocates its own host pools, so it checks
the streaming kernels, which this change does not touch.

Not tested: layer split or remote experts under WSL, Docker Desktop.
setup.py still never writes --kv-resident on WSL and removes it there
(upgrade_config); until that changes it has to be added by hand.

Port of PR #1498 (commit 79b43be18).
2026-10-14 07:07:32 +02:00
Evan Lightcap f2e142f444 resident RAM mode: a budget keeps the prompt path's lend region when it has room (#1683)
pin_cache_complement set lend_from_slot = -1 whenever --resident-budget-gib was given, so the experts of the cache
slots the prompt path borrows were in neither VRAM nor RAM during a prompt and were read from the model files on every
chunk, and again when the slots were refilled - even with room left in the budget. The budget branch now appends the
lend region's experts after the ranked ones, from the last slot down (the order the borrowing takes them), while they
fit; the budget stays a hard cap, and the ones past it keep the file fallback. The start-up line reports the count, and
the lend warning names a larger budget as the remedy when one is set.

Measured on an RX 6900 XT 16 GB (HIP gfx1030), Ryzen 9 5950X, 125 GB RAM, UD-Q4_K_XL, --resident-budget-gib 71 (the
copy held all 63.47 GiB outside the cache, 7.5 GiB unused; 1,231 lent slots = 3.60 GiB), engine 0.1.42, one run per
case, greedy: file reads of the lent experts per request 0.5-2.6 GB -> 0; SSD reads per fresh 8K / 16K prompt 2.8 /
1.9 GB -> 0.3 GB; prompt-path host staging 0.8-2.1 s -> 0.06-0.5 s per request; fresh prompts 887 / 7.8K / 15.8K
tokens 180 -> 194, 633 -> 676, 677 -> 698 tok/s. Decode unchanged.

Port of PR #1896 (commit 29063eb2b).
2026-10-14 07:07:32 +02:00
Xiaomi2008 fa1d1d1ebb STRATA_EXCHANGE_ROTATE: rotation for packs whose layers differ in expert block size (spares kept per size)
Rotation hands the evicted expert's exchange buffer over as its place in the
RAM copy instead of copying the bytes there. It required one block size for
all layers and fell back to the copy path on any other pack: Swift 1.5 IQ3_S
has seven sizes (1,510,400 to 2,662,400 bytes) and IQ3_XXS nine.

A swap exchanges two experts of one layer, so the slot the promoted expert
leaves has the size the evicted one needs. ExchangeStorage now keeps its
spares per block size (initialize_sized): an exchange takes a spare of its
layer's size and hands back one of the same size. commit() refuses a spare of
another size and an expert of another layer. initialize() keeps the uniform
geometry and its checks. exchange_buffer() takes the layer; reserve_exchanges()
takes the number of trailing buffers that only stage copies (--adapt-async's
second half), kept in the largest size alone. With --adapt-async 1 a rotated
round flips in the tick that sees the copies in: there is nothing for the job
to move, so the round ends one verify window sooner.

Still opt-in (STRATA_EXCHANGE_ROTATE=1); unset, nothing changes. The exchange
allocation grows to one spare of every size per swap of a round: 1.06 GiB of
page-locked RAM on IQ3_S with --adapt-swaps 80 (1.26 GiB with --adapt-async 1)
against 0.20 and 0.40 GiB.

Measured on a Core i7-13700K, RTX 5090 32 GB, 64 GB, Windows 11, Swift 1.5
IQ3_S, 128K, --resident-experts, --adapt-every 1 --adapt-swaps 80, p-cores:
  one 8K prompt, the same 1,500 forced tokens, blocking tier, rotation off / on:
    161.6-165.1 tok/s (five runs) / 184.9-188.7 (four runs); two runs of each
    arm with the same cache size (10,681 slots) wrote identical --window-hashes
    for all 434 windows
  --serve, 23 coding-agent turns, greedy, three interleaved rounds, blocking tier:
    147.8 / 159.2 / 158.4 (the build before) -> 175.2 / 184.9 / 172.8
  the same turns with --adapt-async 1, one binary, two rounds:
    166.0 / 170.1 -> 185.6 / 182.5
The tier swaps about 30 experts per verify window on this box (44% of the
experts fit in VRAM), so the copy path moved about 50 MiB of RAM per window.

tests/core/exchange_storage_test.cpp gains a three-size case (6,144 exchanges)
and passes built alone with MSVC 14.50. Built for CUDA only (sm_120, CUDA
13.0.88, MSVC 14.50 with nvcc's --allow-unsupported-compiler); HIP and SYCL
were not compiled (expert_source.cpp and generate.cpp are built by both). Not
measured: Linux, AMD cards, a layer split.

Port of PR #2042 (commit 8db375018).
2026-10-14 07:07:31 +02:00
Katsumi Takeuchi 9de548d198 sycl opt-in: STRATA_IQ3S_WIDE=1, a wide mmvq kernel for the dense IQ3_S matrices
The multi kernel reads the dense IQ3_S matrices (attn_q, attn_gate, ssm_out, attn_k, the shared experts' gate/up)
at 76-114 GB/s on the B70. The new kernel has the shape of the Q4_K..IQ4_XS wide kernels. unitrace, one card,
IQ3_XXS, T=1: worst attn_k (512 rows) 77 -> 123 GB/s, best attn_q (12288 rows) 114 -> 307 GB/s; these matrices take
0.60 ms of kernel time per token instead of 1.36 ms.

Unset or 0: the multi kernel, output identical to main. With the option the greedy output parts from the default at
token 198 of 256 (the sub-blocks are summed in another order).
2026-10-14 07:07:25 +02:00
libratechw edf61f18e9 prefill: the VGPR-capped gfx11 fused expert kernels are on again for gfx1151 on clang 23, except IQ2_XS and Q8_0 (#1180)
Since 0.1.40.2 the waves_per_eu(8) variant of native_w11_kernel is uncapped on clang 22 and newer, because clang 22
computes wrong results with it on gfx1100. On gfx1151 with clang 23 (ROCm 7.14.1) the capped kernels give the same
bits and take less time, as alexdns1 measured in #1180. The cap now applies there again, per format
(hip_prefill_fused_iq, one layer, 2048 / 3584 / 8192 tokens, the same output bits hash in every case):
- gate/up IQ2_XXS / IQ3_XXS / IQ3_S / IQ4_XS and down Q2_0 / IQ4_NL: capped, 8-15% less time per layer.
  IQ2_S gate/up is capped too, but its LDS leaves room for 3 blocks per WGP and the compiler keeps its 238 VGPRs;
  IQ2_S / Q2_0 takes 2-4% less time from the Q2_0 down.
- IQ2_XS gate/up: not capped. Under the cap it spills 104 VGPRs and takes 61-66% more time.
- Q8_0 down: not capped. It spills 18 VGPRs and took -1.9% to +1.9% time (with Q4_K gate/up).
The test is the clang major version: other clang 23 builds are not measured, and clang 24 and newer keep the clang 22
rule. The template argument is the same in the host and every device pass; only the device pass for gfx1151 decides
the attribute (nw_capped). Other architectures, other compilers and -DSTRATA_W_LB=N behave as before: the gfx1100
disassembly is identical to before (kernel names aside). The file builds for gfx1012 / 1030 / 1100 / 1150 / 1151 /
1200 / 1201 and with STRATA_HIP_GFX906 (gfx906); no CUDA build here, and the changed lines are inside
`#if defined(__HIPCC__)`. SYCL has its own copy of this file, not touched. The path is opt-in (STRATA_PF_FUSED=1), so
default output is unchanged.

End to end on a Ryzen AI Max+ 395 (128 GB, gfx1151), UD-IQ4_XS, the section-5 switches of docs/STRIX_HALO.md with
--prefill 16384, 3 interleaved rounds of fresh servers: a 128K prompt read 1,126.1 -> 1,153.4 tok/s (+2.4%; +1.8%,
+2.4%, +2.6% per round). At 4K (868.0 -> 894.9, +3.1%) and 32K (1,182.9 -> 1,207.0, +2.0%) the medians moved too, but
within the spread of 3 runs. Decode unchanged; the same text for all 9 prompts.
2026-10-14 07:07:17 +02:00
libratechw acc94b2171 prompt attention (gfx11 / gfx12 WMMA): issue a chunk's reads before its compute
prompt_attn_wmma_kernel interleaved a chunk's global reads with its
compute: the V slice was staged first, then each 16-cell tile loaded its K
scale and fragments between the q.k WMMAs, so a wave waited on each read
in turn (a thread trace on gfx1151: 56% of the kernel's time in
s_waitcnt vmcnt, 0.4% in WMMA). The chunk's reads now go out together
(fetch), then the same compute runs on them (chunk). The values and the
order of the arithmetic are unchanged.

gfx1151, ROCm 7.14.1 SDK (clang 23), hip_prompt_attn_wmma with
STRATA_PA_FAST=1, three interleaved runs against the unchanged kernel,
ms per chunk: 32K context / 2,048 queries 12.50 -> 11.29 (-9.7%),
1,500 / 1,500 3.39 -> 3.20, 2,100 / 256 1.30 -> 1.23. The output hashes
(NEWHASH) match with STRATA_PA_FAST=1 and =0. gfx12 compiles the same
change; it was not run there.
2026-10-14 07:07:13 +02:00
Andy Slater db0b4d315c sycl/A770: wide MMVQ kernels at 2-8 columns in large-GRF mode on Arc A-series
The wide kernels (Q6_K a2 and the generic native_mmvq_wide) spill at 2-8 columns with 128
registers per thread on the A770. They now launch with grf_size<256> there. Large GRF halves the
threads resident per Xe core, so it is the default on Alchemist only (strata::intel_gpu_gen);
Battlemage and other devices, and 1 column on every device, launch the same kernels as before.
STRATA_MMVQ_LARGE_GRF=1 forces it on, =0 off.

mmvq_bench now prints a checksum of the output (FNV-1a over the float bits) so the two modes can
be compared bit for bit.

Measured on an Arc A770 16 GB (xe, oneAPI 2026.1 JIT), same day. mmvq_bench Q6_K, 2560x2560 / 2560x10240 /
10240x2560, 1-8 columns: 1 column unchanged; 2 columns 1.3-2.0x, 4 columns 2.3-3.1x, 6 columns 3.0-5.0x, 8 columns
3.7-7.4x faster (e.g. 2560x10240 at 4 columns 289.3 -> 92.9 us, 74 -> 232 GB/s of weights); checksums identical
with STRATA_MMVQ_LARGE_GRF=0 and =1 in all 15 cases. End to end, a0116b27 + #2190 + #2193 + #2203 with and
without this commit, Coder IQ1_M, 128K context, int8 KV, --resident-experts, STRATA_VERIFY_STEPPED=1, 8K prompt,
256 tokens: decode 23.47 / 23.50 / 23.73 / 23.77 -> 25.99 / 27.10 / 27.49 tok/s (+10-17%), prompt unchanged,
output ids identical where the expert placement matched. Not built or run on Battlemage.
2026-10-14 07:07:12 +02:00
yi-sudo f629fb0417 Batch decode: one launch for all rows in QSA attention and GDN recurrence (opt-in)
With --batch N the verify window ran QSA block scoring, top-k and decode
attention as one launch per row, and the GDN recurrence (window and 1-row
commit) as one launch per slot.  At 4-8 live slots these serial launches are
most of the window's cost beyond the expert reads.

Two env switches, both off by default (off = the same launches as before):

  STRATA_BATCH_ATTN_ROWS=1   block scores, top-k and decode attention for all
                             rows of the window in one launch each; every row
                             reads its own slot's indexer keys and K/V pools
                             (passed by value as kernel params, so the graph
                             captured per row layout stays valid).  A slot that
                             streams KV (kv_mode 1) keeps the per-row path.
  STRATA_BATCH_GDN_GROUPS=1  GDN recurrence in the window and the 1-row commit
                             for all slots in one launch (grid.y = slot group).
                             Conv commits stay per slot.

Same arithmetic per row: 4 concurrent temperature-0 prompts give the same
bytes with and without the switches (--batch 4 and --batch 8).

RTX PRO 6000 Blackwell, IQ3_S, all experts in VRAM, a0116b2, same day,
total tok/s (docs/BATCHING.md has the table):
  --batch 4, 4 clients: 268.4 -> 311.7
  --batch 8, 8 clients: 308.4 -> 374.8 (reserve 6144 MiB)
Window: 4 rows 14.3 -> 12.2 ms, 8 rows 24.6 -> 19.9 ms.

Built and run with CUDA 13.0 (sm_120) only.  The .cu files are also in the
HIP build, which was not built here (no ROCm).  The SYCL port has its own
copies of these files and is untouched.
2026-10-14 07:07:10 +02:00
imanu 8e68aa5879 OPUS-MTP-MMVQ multi-colonna con attivazioni riusate fra le righe del blocco (STRATA_B6_MMVQ_ROWS) e micro-bench
Q4_K, Q5_K, Q6_K e IQ4_XS: il frammento di attivazione di ogni colonna si carica una volta per blocco
di pesi e serve tutte le righe del blocco; le scale Q6_K stanno in registro. Stesso ordine di accumulo e
di riduzione: uscite identiche bit per bit al layout esatto e alla chiamata a colonna singola
(b6_mmvq_bench, memcmp su entrambe le schede). Default spento.
2026-10-14 07:07:09 +02:00
Stuart Chapin 77ddc2ff14 cpu: IQ2_S in the bit-exact singleton rows; the singleton path only below the #152 rule
Stacked on #1415. IQ2_S (type 22, 122 gathered) joins STRATA_CPU_IQ_ALL_EXACT1's list: ggml's AVX2
iq2_s dot sums the same int32 lanes as row_dot_ggml_one and ends with 0.125f * hsum_float_8.
The singleton gate now also requires nt < native_gu_mt_min, so with STRATA_IQ_MT_MIN=1 a lone token
keeps the multi-token kernel and native_expert_parity's width invariance passes with the switch on.
2026-10-14 07:07:08 +02:00
leadgtr7 6befb3d65d sycl mmvq: Q6_K decode kernels no longer ALU-bound; output heads as aligned planes; int4 activation loads
- Q6_K wide kernel (and the planar one): the six-bit offset (v - 32) was dpct's per-byte saturating char4 subtract, a per-byte loop on Xe;
  no byte can saturate, so a 4-op SWAR form is bit-exact. Q6_K dense matvecs 59.5 -> 30.8 us (1 row), 77.4 -> 45.5 us (4 rows).
- STRATA_Q6_PLANAR (default on, =0 off): the Q6_K LM head and the MTP draft head are converted at load to ql/qh/scales/d planes (same bytes, the
  GGUF-layout allocation is freed: no VRAM change) so a lane's loads are single aligned 16-byte loads; the draft head subset is gathered from the planar
  head. native_q6_k_mmvq serves every registered matrix at any column count. Bitwise equal outputs.
- ld_i4_a4: the Q8_1 activation qs (4-byte aligned) as one dword-vector load in the Q4_K / Q5_K / IQ4_XS wide kernels (Q5_K 4 rows 81 -> 68 us).
2026-10-14 07:07:08 +02:00
Niko1221 db568409fc tools: batch_test.py takes up to 32 prompts (folds in havok2-htwo's batch_test16.py and batch_test32.py) 2026-10-14 07:07:06 +02:00
InB4DevOps 8ee2005dae cpu: add bit-exact AVX2 singleton expert rows 2026-10-14 07:07:04 +02:00
KarlGrier 1a2f95c332 QSA decode attention: split each query's cells across blocks on the int8 mma path (STRATA_QSA_SPLIT_ATTN=1, opt-in)
With STRATA_QSA_SPLIT_ATTN=1 the verify window's and the draft layer's QSA attention run through the prompt path's int8
tensor-core kernel with each query's cells split among up to 20 blocks, and the splits are merged with the log-sum-exp
rescale. qsa_decode_attn_split() returns false (nothing launched) unless the KV is int8 on sm_80 or newer, or when its
partials would not fit the caller's scratch, and on HIP; the caller then keeps qsa_decode_attn_batch(). The sums run in
another order, so with the switch on the output is a rounding-level change. Unset, nothing changes.

The idea is from Eddoursul's fork (fd23d91); the kernel is written against this tree's qsa_prompt_attn.cu.
2026-10-14 07:06:38 +02:00
KarlGrier ffc30b8d76 Verify window: the mixer's input-only work on graph branches (STRATA_SIDE_MIXER=1/2, opt-in)
STRATA_SIDE_MIXER=1 runs the GDN gates and the QSA indexer (key projection, tail snapshot, query chain, block scores,
top-k) on a branch of the window's graph beside the attention's projections, joined before the recurrence and the
attention; =2 also runs the GDN gate projection and the QSA queries' chain on a second branch. A branch that projects
reads F4's interleaved copy, written on the main stream before the fork, and qkv is recorded before the branch so its grid
is dispatched first. Every result is bitwise the same as in line; STRATA_DF_BRANCH is off while this is on. Unset, the
window runs in line as before.

The idea is from Eddoursul's fork (34a35db, b063c13).
2026-10-14 07:06:34 +02:00
imanu f7eaf2de9a Verify window: the routed experts' gate/up with the group's q8_1 rows in shared memory (STRATA_EXPDEC=1, opt-in, bitwise); RTX 2080 Ti gate/up kernel -11/-23% at T 1-4, decode +1.1% on a 131k layer split
native_expert_grouped already decodes each weight part once per block, but the apply re-reads every entry's q8_1 block from global memory with 9 scattered 4-byte loads, so an extra row of the window costs ~60% of a whole expert. expdec_gu_kernel stages the group's entries in shared memory (two conflict-free 16-byte loads per call, the fp16 scale as a float), takes rb rows per block and stages the IQ3 codebooks too; the same lane does the same calls in the same order with the same operations (apply_u = apply on the block's words), so the output is bitwise the default kernels' (expdec_bench: identical on every cell, 2 cards). Gate/up IQ2_XXS/IQ2_XS/IQ3_XXS/IQ3_S/IQ2_S at n_embd 2560; the down (Q2_0, IQ4_NL) behind STRATA_EXPDEC_DOWN=1 (in the engine the Q2_0 down was ~10% slower). Knobs STRATA_EXPDEC_R/_DR/_MINB/_RB/_DRB/_MINBLK/_D20; one log line per (role, format) at the first launch. docs/EXPERTDEC.md.
2026-10-14 07:06:21 +02:00
Alex Sadleir 79376a2732 cpu: the AVX2 expert-row dots keep their accumulators in registers too
q5_1_dot and q8_0_dot in kq_avx2.cpp held their per-token sums in the same
runtime-indexed __m256 acc[MAXT] that made bf16_rows_dot_multi spill: with the
array live the compiler leaves it in memory, and every token's FMA reads and
writes a stack slot.  Both are now template <int NT>, dispatched 1..4 and, past
that, run four tokens per pass with the row's bytes re-read from L1 - the same
idiom the router change above and kq256_gu_rows already use.

Ryzen 7 9800X3D (Zen 5), GCC 15.2.0, build/native_expert_parity --synthetic
(ggml-quantized weights and activations), one thread pinned to core 0, at the
verify window's four tokens:

  Q5_1 down rows [2560 x 640]: 190-197 us -> 155-156 us  (1.24x)
  Q8_0 down rows [2560 x 640]: 135 us     -> 104 us      (1.30x)

Same sums: 0 of 36 token-sets differ in any bit against ggml's own vec_dot over
1..8 tokens, both before and after; native_expert_parity reports 0 failures.

Left alone: q4k_dot keeps its runtime-nt array.  Its token accumulators are not
what spills - the unpatched function carries 8 ymm stack ops for the sums
against 24 (q5_1) and 59 (q8_0); what q4k_dot spills is its weight-side unpack
arrays (q4l/q4h/scale_l/scale_h), which token chunking does not touch and
re-does.  The same change measured +5% at four tokens and -4..-7% at six to
eight, so it is a wash at the window and a small regression above it, and q4k is
the gate/up format that runs most.  The AVX1 path, the iq_avx2/q2_avx2 rows and
the engine's dispatch are untouched.

Backends: CUDA and HIP do not compile this file.  The SYCL port does; not built
on this machine (no icpx here) - the router change in this branch built there.

Tests: native_expert_parity --synthetic q4_K/q5_1 and q4_K/q8_0 (0 failures,
0 bits differ), iq_avx2_parity, iq_avx1_parity, q8k_quant_parity,
router_dot_parity - all exit 0.
2026-10-14 07:06:19 +02:00
Alex Sadleir 2a669f3c29 cpu: the AVX2 router dot keeps its accumulators in registers (2.1x on a Zen 3)
bf16_rows_dot_multi held its per-token sums in __m256 acc[MAXT] indexed by a
variable. With eight 256-bit accumulators live GCC also unrolled the column loop
and left the whole array in memory, so every token's FMA read and wrote a stack
slot (vmovaps 0xc0(%rsp) ... vfmadd213ps 0xe0(%rsp)). On a Ryzen 7 5700X3D one
layer's router [512 x 2560] at 4 tokens took 0.41 ms against 0.25 ms for the AVX1
copy in kq_avx1.cpp that kept its accumulators in registers, so every AVX2 CPU ran
the slower of its two kernels (router_dot_parity --cpu 2, median of 3). Pragma
GCC unroll 1 on the column loop changes nothing; the variable-indexed array is
what forces the array into memory.

Four tokens per pass, in four named accumulators. A row above four tokens is read
a second time, from L1: nt=8 measures 0.37 ms, the same as the spilled form. On
the same CPU the AVX2 kernel goes 0.41 -> 0.19 ms at 4 tokens and nt=2 0.36 ->
0.16 ms, and it now runs 1.3x ahead of the AVX1 copy instead of 1.6x behind it.
The loop body carries no stack slot any more (was 21 memory ops per iteration).

Untouched: the arithmetic and the reduction. Each token's FMA chain still runs
over the same columns in the same order and ends in the same hsum_float_8, so
every sum is bit-identical - the output for nt 1/2/4/8 at cols 2560/5120 hashes
the same before and after (FNV-1a over the whole out[]), and router_dot_parity
reports 0 failures with the AVX2 rel error unchanged at 3.16e-07.

Left alone: the expert row kernels (iq_avx2.cpp, q2_avx2.cpp, kq256_gu_rows,
kq256_rows) and the AVX1 path. The iq_avx2_parity bench over four format pairs x
1..8 tokens moved within 1.6% (noise), and the CPU parity tests (8 targets) pass.
The router stays what it was, an estimate for which experts to prefetch. CUDA and
HIP do not compile this file; the SYCL port does, and it builds there under icpx
2026.1 with no new warnings and no spills either.

The AVX1 copy in kq_avx1.cpp kept a comment saying it reads its rows "exactly as the AVX2
kernel does"; with the row loop chunked four tokens at a time that is no longer the
AVX2 kernel's shape, so the clause is dropped there.  Comment only - no code change.
2026-10-14 07:06:18 +02:00
imanu 731528beba QSA decode attention: PR #540's two-cells-per-step chunk kernel on every CUDA card and a bounded merge (STRATA_ATTN_DAILY=1, opt-in, bit-exact); parity harness
attn_chunk_kernel_pre75 (PR #540 by sskver: two cells per step with a 12-shuffle reduction, cell-major
probabilities, V loaded four at a time) is compiled in every CUDA build, not only with STRATA_EXPERIMENTAL_SM60, and
STRATA_ATTN_DAILY=1 selects it on any card for the decode attention (single query and batch: verify window and MTP
drafter) together with attn_merge_v2_kernel (Hardin22), a merge that skips the chunks the chunk kernel marks empty.
Bit for bit with the default kernels by construction: reduce12 adds the same pairs as warp_sum's butterfly at every
level, the V cells fold in the same order with the same fmaf, the masked cells stay out; the merge does the same fmaf
in the same order. Shared memory 15,872 B like the default kernel (spt shares sq's storage). STRATA_ATTN_MERGE_V2=0/1
and STRATA_ATTN_PRE75=0 switch the two halves separately. HIP unchanged.

qsa_merge_daily_parity (tests): memcmp of both kernels against the default ones on four KV formats, widths at the
chunk borders and non-resident pages; --bench times the chunk kernels back to back.
2026-10-14 07:06:09 +02:00
imanu 5b6c93987f QSA decode: the window's block scores with the queries in registers and six shuffles per block (STRATA_QSA_ROWS=1, opt-in, bitwise); RTX 2080 Ti 101 -> 55 us a call at T=4, decode +3.2/+0.1% on a 131k layer split
In a 131k decode block_scores_multi_kernel costs ~20 us more per call for every row of the verify window: not the keys (read once for all the queries) but the shared-memory and shuffle pipes, 4 x LDS.128 of the query slices and 20 shuffles per (block, query). block_scores_rows_kernel<NQ> keeps the NQ <= 4 queries' slices in registers and reduces the four heads' lane products with a scatter that keeps warp_sum's pairings (xor 16 and 8 leave one head per lane, then 4, 2, 1): every addition adds the same two operands of the same level, so scores and top-k ids are bitwise the multi kernel's. qsa_rows_bench measures the window's QSA kernels row by row and, with --rows, checks the parity (16 cases per card, 0 differ) and prints both kernels' microseconds: 2080 Ti 47/57/78/101 -> 36/38/49/55 us at T 1-4, 3060 73/109/152/194 -> 63/76/98/130. docs/QSAROWS.md.
2026-10-14 07:06:00 +02:00
ATIVX928 f1a9eaa47e gdn chunk: the smem opt-in and the workspace are per device (layer-split fix)
justxiami reported (#1098) a 4xV100 layer split dying with
STRATA_GDN_CHUNK=1.  Reproduced on 2xV100: the second card launched the
chunked kernels at the 48 KB default (a process-global std::call_once set
the opt-in on the first card only) and the engine exited with
"gdn_chunk wmma: invalid argument".

Two more per-process assumptions in the same path:
- gdn_chunk_cc() cached the first card's capability for the whole process
- the chunked workspace was thread_local with no destructor: the per-request
  stage threads leaked ~50 MB per card per request (the second card ran out
  of memory on the next prompt) and one card's pointers could reach another

Now: the opt-in and the capability are recorded per device (each attempt
clears the runtime error, and a refused wmma opt-in falls back to the FMA
path), and the workspace is one per (thread, device) with a destructor.
Verified on 2xV100 32K: STRATA_GDN_CHUNK=1/2 run to completion over repeated
rounds (1: 1900 tok/s prefill, 2: 1766), gdn_chunk_parity 0 failures.
2026-10-14 07:05:19 +02:00
ATIVX928 fd13adf248 bench (v100): gdn_chunk_bench - the GDN prompt recurrence variants and the chunked path, real shapes
CUDA events over one gdn_recurrence call per side (S=128, 16 q/k heads, 48 v heads, C=10240,
T=1024/4096/8192), all four paths and the max abs diff between the chunked wmma output and the
pipelined recurrence.  RESULTS.md records the table and the error bounds for
bench/results/2026-10-02-gdn-chunk (measured on the V100-SXM2-16GB, CUDA 12.8, sm_70 build).
2026-10-14 07:05:18 +02:00
ATIVX928 c0a4a9e657 tests (v100): gdn_chunk_parity - the chunked and Volta GDN recurrences vs a double-precision host reference
GPU, synthetic, no model.  Compares through the real dispatch (STRATA_GDN_CHUNK /
STRATA_GDN_REC_WARP A/B) at T=64/128/200/320 - 200 exercises the < 64-token tail - and asserts
measured bounds, not equality: the recurrence is the FP32 floor (y abs 1.1e-6, state 3.0e-8), the
chain-split and the chunked FMA path are FP32-level (~3e-6), the wmma chunked path is the
f32->f16 staging level (y abs 2.3e-3, state 8.9e-5).  The sync-free warp recurrence is asserted
BIT-EXACT against gdn_rec_cols_pipe (memcmp of y and the final state).  A NaN or inf anywhere
fails the check outright - an unordered comparison would otherwise leave the max at zero.

On main's 0.1.38 recurrence the out norm keeps its raw FP32 output in place and writes only the
FP16 copy, so the kernel comparisons use that raw output (ref_full's yraw_out) and the norm is
checked once through its FP16 store.
2026-10-14 07:05:18 +02:00
ATIVX928 7c4ff2de72 prefill (v100): the chunked GDN prompt recurrence (FlashQLA gdn-chunk) + Volta recurrence variants
The prompt GDN recurrence is a per-token chain; on a V100 that chain is what the 36 GDN layers
cost.  This adds the chunked form (cumsum / kkt / fwd over 64-token chunks, chunk-serial scan,
the state in registers - the gdn-chunk-sm70 structure of llama.cpp-v100, hand-written CUDA):
a chunk is a triangular solve and a batch of matmuls instead of 64 sequential rank-1 steps.
Two variants: an FP32-only FMA scan and an m16n16k16 wmma one with the reference's f32->f16
staging.  Both are FP32-level, NOT bit-exact (the sums are reordered; the wmma path stages
q/k/v and the state through f16) - gdn_chunk_parity bounds them.

Also two Volta variants of the recurrence itself: a sync-free warp-shuffle reduction (BIT-EXACT
vs gdn_rec_cols_pipe, but measured 2.6x slower - kept as the bitwise control) and a 4-accumulator
chain split of the pipelined kernel (FP32-level, measured +6-7% at T=4096/8192, the V100 default
now).  All of this is compiled only into the STRATA_EXPERIMENTAL_SM60 build and dispatched only
at cc==7.0; sm_80 and up keep the trunk's kernel bit-for-bit.  The env switches
(STRATA_GDN_CHUNK=1/2 chunked, STRATA_GDN_REC_WARP=0/1/2) still override.

Measured on the V100 (bench/gdn_chunk_bench, real shapes, one gdn_recurrence call): the chunked
path is correct but 4-6x SLOWER than the recurrence as implemented (4-warp CTAs underfeed the
tensor cores - see bench/results/2026-10-02-gdn-chunk/RESULTS.md), so it stays opt-in.
2026-10-14 07:05:18 +02:00
ATIVX928 d284809fb0 V100: operator fusions for the decode GEMV / MoE paths, bitwise
Fusions (all bit-exact vs the pipelines they replace; bench/micro/fusions_parity.cu
checks every one bitwise and times the before/after at decode shapes):

- gate/up dual GEMV + SwiGLU in ONE kernel (llama.cpp's mul_mat_gated):
  s_gemv_pair_silu, one read of the activation for both projections and the
  SwiGLU in the epilogue - the up intermediate never lands in memory.  Rows are
  accumulated by the shared row functions (s_rowdev.cuh) exactly as the
  standalone kernels accumulate them.  shared_expert uses it whenever both
  projections are S-family (bitwise checked for all three silu kinds).
- silu(gate)*up + the intermediate quantizer fused (was 2 kernels):
  swilu_quantize_q8_0/_scaled/_q8_K, warp-parallel swilu then the verbatim
  quantizer body.  Wired into the S2 expert entry points and shared_expert.
- every activation image of x in one pass (was 2-3 kernels): quantize_act_images
  produces block_q8_0 + block_q8_K + bf16 from one read of x.  Wired into
  gdn_layer, qsa_layer and moe_shared.
- multi-weight bf16 GEMV (llama.cpp's mul_mat_multi): bf16_gemv_pair computes
  two matrices from one read of x; used for ssm_alpha/ssm_beta and for the
  indexer's q/k projections (q and k now one launch).

The call sites are gated on STRATA_EXPERIMENTAL_SM60 (the Pascal/Volta build),
so the ready-made engine runs exactly the pipelines it did before; the fused
kernels are the same code either way.

Measured (V100, CUDA events, n_embd 2560 / FF 640, 200 reps):
  expert intermediate (swilu+q8_0)   15.7 -> 11.6 us  M=1,  18.8 -> 11.4  M=8
  activation images (q8_0+q8_K+bf16) 46.2 -> 30.4 us
  dual GEMV + swilu (gate,up)        30.4 -> 14.8 us
  bf16 pair (alpha,beta)             24.2 -> 14.4 us  M=1,  23.7 -> 14.1  M=8

Parity: fusions_parity 0 failures (bitwise), plus s_gemv/s_gemv_q8k/s2_gemv_q8/
s2_expert_grouped/shared_expert/quantize_act/bf16_gemv/elementwise/mmvq_multi/
iq_multi/native_expert parity suites all pass unchanged.
2026-10-14 07:05:13 +02:00
ATIVX928 1e14e6469e kernels (sm_70): the prompt attention's latency pipeline (mma884 kernel)
Port of oldlocal commit 4672cf9 ("kernels (sm_70): the prompt attention's
latency pipeline - 2.2x over the FP32 kernel at 32K") onto the #600
prompt_attn_v70_kernel, the m8n8k4 Volta kernel now inline in
qsa_prompt_attn.cu.  Still gated on STRATA_EXPERIMENTAL_SM60, so the
ready-made engine keeps exactly the code it had.

Two of the commit's three changes port; the third (splitting the dependent
WMMA chains into f0/f1 and fh/fl) is already inherent to the mma884 kernel -
its scores run two tg[rb] chains and its p.v four tmp[rb][nt] chains - so
porting it would only change the summation order.  The two ported, measured
on a V100-SXM2-16GB (CUDA 12.8):

* the staging loop issues every global load of a thread before any of its
  stores (2-4 in flight), instead of paying the HBM latency per piece;
* the next chunk's ids/page-table lookups double-buffer (row[2][CH]) and run
  just before the previous p.v, where their latency hides.

The original commit measured, on the #371 kernel, 2,048 queries of a 32K
selection: 34.4 -> 15.8 ms (2.18x) int8, 33.5 -> 15.4 ms (2.17x) fp16 KV,
35.6 -> 20.3 ms (1.76x) K8V4 (its q4_0 V dequantizes to FP16 at staging, the
sm_75/80 kernel's same trade).

This port, same 2,048 queries of a 32K selection (qsa_prompt_attn_sm70_bench,
which also gained the K8V4 pools): the Volta kernel goes int8 12.11 -> 10.25 ms,
fp16 KV 25.70 -> 17.39 ms, K8V4 25.71 -> 24.12 ms, so against the FP32 decode
kernel int8 rises 1.73x -> 2.05x and fp16 0.80x -> 1.18x (K8V4, 0.85x -> 0.91x,
keeps main's #600 dequant-to-FP16 trade).

qsa_prompt_attn_parity passes all four cases.  An A/B against the unpatched
kernel shows the kernel's output is unchanged - every FP64 and new-vs-old
figure is identical before and after (int8 5.08e-6, fp16 5.29e-6 vs FP64,
within the 4x gate).
2026-10-14 07:05:12 +02:00
ATIVX928 132faae7cf kv_q8: the gather decodes 8 int8 per thread (bitwise, +11% on the 32k microbench)
The experimental build (-DSTRATA_EXPERIMENTAL_SM60=ON, Pascal/Volta) gains a
kv_gather_q8_wide_kernel: one uint2 (8 B) load of codes, one uint4 (16 B) store of
the dequantized fp16 and one scale per thread instead of two.  The 8 values lie in
one 64-value group and the arithmetic is unchanged ((float) code * scale ->
f16_from_f32), so the output is byte-identical.  The ready-made engine keeps exactly
the 4-wide kv_gather_q8_kernel and launches it unchanged, so its gather is bit for
bit the release engine's.  On a device below sm_75 the wide kernel is selected at
run time inside the experimental build (STRATA_Q8_GATHER_WIDE=0 opts out).

Measured on 2x Tesla V100-SXM2-16GB (SM 7.0), CUDA 12.8, one card: +11.4% on the
32k gather microbench (15.85 -> 14.05 us/call, 401 -> 452 GB/s).  kv_q8_parity
stays OK at worst INT8-vs-FP16 error 0.590 quantization steps.
2026-10-14 07:05:04 +02:00
ATIVX928 6a5be385c6 audit: where the int8 KV read decodes, and whether it is vectorized 2026-10-14 07:05:04 +02:00
Niko1221 8ccc4a0fe6 Verifier::ar_live and no Prefill::cuda_err_ in the hand-off worker (the two pieces of the unported STRATA_PIPE_OVERLAP series that STRATA_ADMIT_MERGE_FIRST and STRATA_HANDOFF_PIPE name) 2026-10-14 07:03:56 +02:00
havok2-htwo ed4fa42a80 serve: STRATA_ADMIT_SHORT_FIRST=<tokens> admits short prompts before long ones (opt-in)
New requests took the control lines in whatever order the lock fell, so a 500-token chat question could wait
behind several agent reads. With the env, a prompt of at most that many tokens goes first (arrival order within
each class); a longer one that waited STRATA_ADMIT_MAX_WAIT_S (30 s) counts as short. A request already in
acquire() gives the lines back when a shorter one came. Only the order changes, never what is read.

Ported from havok2-htwo/Strata 08830ddf onto 0.1.44.1: _take_control has no image-request line here, so its
gate is only the ordering; the first admission passes new=True.
2026-10-14 07:02:42 +02:00
Niko1221 ed7c7f339d engine: a pack folder without native_experts.txt says so before the AVX-512 refusal (#2347)
The refusal 'this CPU cannot run the expert kernel' belongs to the canonical Q2_0 pack. An NVFP4, i-quant or
Unsloth pack is a native pack (native_experts.txt in the --pack folder) and its rows have AVX2 kernels, but a
config whose --pack points at the wrong folder reads like a CPU that is too old. On a CPU without AVX-512 the
engine now says the file is missing, and where it sits when it is one level down (<pack>/pack).
2026-10-14 07:02:11 +02:00
havok2-htwo 1d9758f897 prefill: STRATA_HANDOFF_PIPE=P hands a layer split's chunk on in P pieces, gated by events, on a persistent thread
A layer split hands each prompt chunk's rows (21 MB per 512 tokens) to the next card through a pinned
host buffer: one D2H, a host stream sync, a new std::async thread, then one H2D before the next stage's
first kernel.  The 3090s here have no P2P (topo: CNS) and cudaMemcpyPeerAsync is driver-staged and
slower (3.4 ms vs 3.2 ms for D2H + H2D), so the copy stays on the host route; nsys showed 3.8-4.0 ms per
512-row hop, of which ~1-1.4 ms was the thread start alone.

With STRATA_HANDOFF_PIPE=P (2..16, opt-in; unset = unchanged):
- the sending stage records an event when its chunk's work is done and one after each of P row-piece
  downloads into the same pinned buffer, and waits on the host only for the first (on_stage_chunk's
  checkpoint parts and the error check need the state, not the copy);
- the next stage waits on the GPU for each piece's event (cross-device cudaStreamWaitEvent) before that
  piece's upload, so its uploads overlap the remaining downloads (separate PCIe links); a smaller
  receiving chunk waits for every piece, then copies as before;
- the next stage's chunks run on one worker thread per sending stage, started and pinned once, whose
  packaged_task futures carry the same results and exceptions as std::async's (it clears the CUDA error
  state after each job, as a thread's end did).  STRATA_HANDOFF_PIPE_WORKER=0 keeps std::async (A/B).

Same bytes in the same rows before the next stage's first kernel: exact.  No device memory (events only,
the existing two pinned buffers per stage).  Measured (T16-143l5, P=8): 512-row hops 3.8-4.0 -> 1.85-1.89
ms, 143-row hops 1.6 -> 1.0 ms; reads 1.2K about -1.5 %, 150 about -1 %, 9K (agent, --kv-resident) -3 %;
decode rows/s unchanged.  batch_test16/32 (greedy and temperature 0.7), batch_interleave_test and
batch_overlap_test (+ the stage fault arm) byte-/token-identical to the reference dumps; an agent-config
run with checkpoint parts byte-identical to the reference binary.
2026-10-14 07:00:13 +02:00
havok2-htwo 6959adf404 tests: qsa_slots_parity skips where a slots launcher declines by design
STRATA_SCORES_MULTI=0, the pre-sm_75 attention (STRATA_EXPERIMENTAL_SM60 builds on a
V100/P100) and the gfx906 query swizzle make qsa_block_scores_slots or
qsa_decode_attn_slots decline; the engine keeps its per-row calls there, so a decline
is no failure. A declined case prints SKIP, the test exits 77 when every case was
declined, and STRATA_SCORES_MULTI=0 skips up front.
2026-10-14 07:00:13 +02:00
havok2-htwo a71ee46cd2 verify: STRATA_QSA_SLOTS=1 runs a batch window's QSA block scores, top-k and attention as one launch over its rows
In a batch window every row reads its own slot's indexer keys and K/V, so each QSA layer ran the
selection and attention once per row: four rows (16 slots) or eight (32 slots) times qsa_block_scores,
qsa_block_topk and qsa_decode_attn_batch, each launch small (the top-k on one CTA, the attention on
cap/64 x 2 blocks).  With the switch, per window (up to 8 rows per launch):

- block_scores_slots_kernel: grid (256, rows), blockIdx.y runs the single-query block_scores_multi_kernel
  body with that row's slot keys (block_scores_multi_kernel is now a wrapper around the same body)
- qsa_block_topk over the rows in one call (it only reads the rows' scores and steps)
- attn_chunk_slots_kernel: attn_chunk_kernel's body with the row's own QsaAttnPools (an 8-entry table
  passed by value, blockIdx.z picks), then the existing batched merge

Each row is computed by the same kernel body with the same reduction order as its own call.  The
launchers decline (the per-row calls run) where that call would pick another kernel: STRATA_SCORES_MULTI=0,
the pre-sm_75 / gfx103x attention, the gfx906 query swizzle.  A streamed KV cache keeps the per-row
resolve-then-attend loop.  Default (switch off): unchanged.

Checked on 4x RTX 3090, T16-143gsc setup (layer split 12,24,37, int8 KV):
- SASS (sm_86) of every existing kernel in qsa_select.cu and of the default attn_chunk_kernel<M> and the
  merge is identical before/after; the opt-in lane-cell instantiations differ only in register allocation
- new qsa_slots_parity (ctest, also with STRATA_ATTN_LANECELL=1): fp16, int8, q4_0 and int8-K/q4_0-V
  pools, 1/2/4/8 rows of different slots with their own page tables and contexts 37..32768: scores, ids
  and attention bitwise identical to one call per row
- batch_test16/32 (greedy and t=0.7), batch_interleave_test and batch_overlap_test (STRATA_PIPE_OVERLAP=1)
  token-identical to the build without the change; with the switch off batch_test16 is byte-identical too
- batch decode rows/s (engine level, 30 s, back to back with the reference): 16 slots 683 -> 726 (+6.3 %),
  32 slots 750 -> 802 (+6.9 %); STRATA_VERIFY_PROFILE "QSA projections + attention" per stage-window
  0.89 -> 0.52 ms (16) and 1.52 -> 0.79 ms (32)

HIP: the new code uses only what the existing kernels use; SYCL has its own sources and only sees the new
declarations.  Neither was built here.
2026-10-14 07:00:13 +02:00
Niko1221 3bfa088e46 setup: Qwen3.6 offers --parallel on small cards too (the price said: a request alone ~5% slower, 4 together ~1.5x), and asks how many requests at once; Enter keeps one at a time 2026-10-14 07:00:10 +02:00
havok2-htwo 2f4fe20a9a serve: STRATA_SLOT_CKPT_MOVE honours the #879 flush (no resume from a tainted slot's checkpoint)
Upstream's non-finite flush only marks running slots `tainted`; their checkpoints stayed.  With
STRATA_SLOT_CKPT_MOVE a running slot's checkpoints are a resume source (active_source) and a slot's
moved turn checkpoint can be restored into the main session (lent_slot), so a request right after the
flush, logged as "reading this prompt from token 0", still continued from a pre-flush checkpoint.

- the flush drops a running slot's checkpoints too (only with STRATA_SLOT_CKPT_MOVE, the one reader of
  them; they go back to the checkpoint pool)
- active_source and the lent_slot search skip tainted slots

Checked with STRATA_NAN_INJECT=600 on the T16-142gsc setup (next turns of chats still running in
slots): before, the two requests after the flush resumed 1628 tokens from slots 0 and 8 ("gave back ...
its turn checkpoint"); now both read 1661 tokens from token 0.  A next turn before the flush still
resumes from its running slot.  Default path (switch off) unchanged.
2026-10-14 07:00:09 +02:00
havok2-htwo f32663a8b3 tools: batch_interleave_test runs on a pipelined batch (no give-way required there) and checks a turn checkpoint taken from a slot
On a layer split with more than one group a batch read never gives way to a
BYIELD, so step 4 always failed with 'no such free slot' and steps 5-7 never
ran.  --yield auto now expects the give-way only on a non-pipelined batch;
there D is read whole and E admitted beside it, both still equal to solo.
New step 7b: A ends in slot 1, --flush unrelated solo GENs, then F2 can only
start from slot 1's turn checkpoint.  --dump writes every token list for an
A/B of two builds.
2026-10-14 07:00:09 +02:00
havok2-htwo bec9edf1e0 prefill: decide STRATA_MMQ_GROUP_SORT per layer once, in the MMQ plan
mmq::whole_tiles walks every visible GPU and every tile width through the
MMQ config table; the sort hook called it twice for each MoE layer of each
prompt chunk, on the host path between the routed-ids download and the
expert launches.  Its verdict depends only on the layer's weight formats and
the device set, so MmqPlan now keeps it as a per-layer flag (taken only when
STRATA_MMQ_GROUP_SORT=1 and STRATA_MMQ_RESIDENT_SORT_NE is unset) and logs
how many layers qualify.  Same order, same bits; without the switch nothing
new runs.
2026-10-14 07:00:09 +02:00
havok2-htwo 69a05bfb59 verify: STRATA_BATCH_SELF_COMMIT=1 lets a one-row-per-slot batch window keep its rows itself
A pipelined batch window is followed on its stage's stream by the layout's commit graph, and the stage counts
as done only after both. When every row of the window is another slot's one token, that commit is
unconditional (each slot keeps its one row; commit_slot_prefixes allows nothing else), and most of the
graph only repeats what the window wrote: it puts each indexer tail back from its snapshot and appends the
same row again, and copies each PLE history back from the window's own snapshot of it. What it adds is the
GDN conv history and recurrence state, which the window ran without writing.

With the switch, such a window advances the GDN conv history (gdn_conv_l2_multi with commit) and state
(gdn_step_norm_multi with n_keep = 1, the kernel the commit graph runs for a one-row group) itself, as a
one-token solo window does, and takes no tail or history snapshots. No commit graph is captured for the
layout; batch_launch and commit_slot_prefixes launch none, layout_captured and ensure_batch_layouts do not
ask for one. Layouts with proposal rows (--batch-mtp) keep the commit graph. With the switch off every path
is unchanged.

Measured on 4x RTX 3090, UD-IQ4_XS, T16-142fcb (split 12,24,37), decode of 16 / 32 slots, 30 s each,
reference and new build back to back: 653.7 -> 686.0 rows/s (+4.9 %) and 714.1 -> 752.6 rows/s (+5.4 %);
the group step period 24.5 -> 23.3 ms and 44.85 -> 42.6 ms. STRATA_VERIFY_PROFILE: the window's GDN
recurrence takes 0.09 ms (16) / 0.18 ms (32) longer per stage-window with the state written in it.

batch_test16 (16 slots, a 1,200-token prompt) is token-identical to the reference dump; batch_test32
32/32 identical and its dump equal to the switch-off run; batch_overlap_test (STRATA_PIPE_OVERLAP=1) passes
and is token- and RESUME-identical to the switch-off run; batch_interleave_test parts 1-3 identical (its
part 4 fails at the same point with the reference build).
2026-10-14 07:00:08 +02:00
havok2-htwo 62fc7c8a3c prefill: STRATA_MMQ_GROUP_SORT=1 sorts the full MMQ groups by row count, keeping the partial group
STRATA_MMQ_RESIDENT_SORT_NE=1 puts a fully resident layer's experts in row-count order, so each
16-expert group's max_rows (its tile padding) is close to its experts' own. It also moves the hottest
experts into the last, partial group, whose few tiles take MMQ's stream-k split: the K boundaries move
with the group's tile count and the prompt's bits change.

The new switch sorts only order[0, n - n % 16] (mmq_resident_sort_span) and leaves the partial group's
experts, and so its rows, where they were. A full group's launch has 160 (gate/up) or 320 (down) tiles
per column tile; mmq::whole_tiles checks on every visible GPU that every such launch runs one block per
tile (tile efficiency >= 90 %, NVIDIA) and that all tile widths share the I/K tiles and SRAM layout. Then
each expert's rows get the same bits in any full group whatever its group-mates and max_rows. Where it does
not hold (e.g. 108 or 128 SMs, AMD's stream-k grid) the switch does nothing. RESIDENT_SORT_NE takes
precedence when both are set; with neither the code path is unchanged.

prefill_mmq_tab_test part 4 runs a layer's groups as prefill.cpp does (src-gathered q8_1 rows, absolute
bounds, the group's max_rows) in ID order and in both sorted orders, for IQ3_S / IQ4_XS gate/up and
IQ4_NL / Q8_0 down: with the partial group kept every token's rows are bitwise equal (16 cases); sorting
the partial group too changes 133-1881 rows per case. 890 comparisons, 0 failed (RTX 3090).

Measured on 4x RTX 3090, UD-IQ4_XS, T16-142fcb (split 12,24,37, every expert resident), read time,
medians of 6, back to back: 150-token prompt 145.8 -> 136.2 ms, 1,200-token prompt 428.6 -> 363.4 ms
(RESIDENT_SORT_NE: 134.9 / 358.6 ms). The final residuals of five prompts (300-1,200 tokens,
STRATA_PREFILL_DUMP_R_ALL every position and STRATA_PREFILL_DUMP_R fp32) are byte-identical to the
reference build; with RESIDENT_SORT_NE every row differs. batch_test16 (16 slots, a 1,200-token prompt)
is token-identical to the reference dump.
2026-10-14 06:59:57 +02:00
havok2-htwo a923889bd2 generate: review fixes for the admission switches
STRATA_SLOT_CKPT_MOVE: an active slot is a slot source only when the main
drafter's rows come along or are not drafted from (no --mtp, --batch-mtp,
or a batch admission).  The slot copy takes the main layers' cells only, so
a solo GEN beside active slots would draft from another conversation's
drafter rows below its start; it reads its prompt as with the switch off.

STRATA_CKPT_THREADS: a checkpoint thread widens its CPU mask before it
goes to the aux CPUs, so the mask the aux placement restores (a request
with aux_cpus=0) is the wide one, not the host thread's single CPU.

STRATA_ADMIT_MERGE_FIRST: a merged first window counts as one row in the
STRATA_DECODE_TIMING window statistics (its prompt rows are not decode rows).
2026-10-14 06:59:57 +02:00
havok2-htwo ca239cdabc generate: STRATA_SLOT_COPY_BATCHSYNC=1 queues the direct slot copy and synchronizes each device once
The direct slot copy (STRATA_SLOT_COPY_DIRECT) waited for each of its ~116 copies on the private copy
stream.  With the switch, conversation_checkpoint_copy and conversation_kv_copy only queue them (new
`queued` argument, default off), and the slot copy's closing sync_all is the one wait per device.  A
streamed K/V state (kv_mode != 0) has its residency refresh on the legacy stream, which is not ordered
after a non-blocking stream: conversation_detail::order_after_queued records an event on the private
stream and makes the legacy stream wait for it first.  The copies stay on the private non-blocking stream.

Same copies in the same order on one stream: tokens are identical.  Measured on 4x RTX 3090 (T16-142,
24 admissions): the slot copy takes 0.5 ms instead of 1.3 ms; batch_test16 16/16, batch_overlap_test
and batch_interleave_test token-identical to the references; a --kv-resident 20480 run with a 21K
prompt is identical with the switch off and on.
2026-10-14 06:59:57 +02:00
havok2-htwo 139d702ebd generate: STRATA_SLOT_CKPT_MOVE=1 moves a batch admission's turn checkpoint into the slot
A batch admission copied the main chain's longest checkpoint into the slot (~112 MB of host vectors, 7.6 ms, ~60 ms
on fresh pages).  With the switch, the request's own turn checkpoint (or one it took out of the slot it is admitted
to) moves instead; a root, a pinned prefix and any other checkpoint the chain shares are still copied.

The chain then no longer holds it, so the request path also looks in the slots:
- while the main session holds its tokens (live, its cells below it), a slot's checkpoint is restored on the main
  session as a chain one would be, with no K/V copy; when that slot is the one the request is admitted to, it moves
  back into the chain first (the admission drops the slot's checkpoints), else the slot keeps it and the admission
  copies it as it copied the chain's;
- an active slot's checkpoints are a slot source like an idle slot's (its windows write its cells from p on): a
  second request with the same history after the main session moved on restores instead of reading it all;
- a slot source that is the admit slot moves its checkpoint into the chain instead of copying it.

The switch also keeps the dropped checkpoints' storage as STRATA_CKPT_POOL=1 does: a moved checkpoint frees
nothing at the next prune, so without it every save faulted in fresh pages (17 -> 38-60 ms, more than the copy saved).

Off by default: nothing changes.  A session SAVE or a parked conversation right after an admission lacks the moved
turn checkpoint (the slot holds it).
2026-10-14 06:59:25 +02:00
havok2-htwo bebeb791a4 generate: STRATA_CKPT_POOL=1 saves checkpoints into dropped ones' vectors, STRATA_CKPT_THREADS=1 saves a split's parts at once
STRATA_CKPT_POOL=1 (off by default): the serve loop keeps the running-state
vectors of the checkpoints it drops (LRU eviction, the tail one, the prune at
a new request, a batch slot's old one), at most two, and the next checkpoint
or slot copy goes into them.  A save then resizes nothing: no zero-fill of
~118 MB of fresh vectors.  Every byte is overwritten by the copy.

STRATA_CKPT_THREADS=1 (off by default): a layer split's checkpoint saves the
first stage's part on the calling thread and every later stage's on a
persistent thread of its own (the copies' private streams are per thread, so
short-lived threads would leak them).  The threads leave the host thread's one
CPU first (to the aux CPUs with --aux-cpus, else to every CPU the process may
use): on it the copies only took turns.

Measured on a 4-stage split (T16, 24 batch admissions per arm): a turn
checkpoint takes 17.2 ms today, 10.4 ms with the pool and 5.6 ms with both;
the batch tests, the overlap and interleave tests and a driver run give the
same tokens with the switches on as off.
2026-10-14 06:59:13 +02:00
havok2-htwo 0cf1a9b7c8 generate: STRATA_ADMIT_MERGE_FIRST=1 reads a batch admission's last prompt tokens and its first token in one window
A batch admission read its header through the verify windows (T=4 + T=2 for the
usual 6 tokens) and then its first token in a window of the last prompt token
alone. With the switch the last windowed part leaves its last tokens (up to the
verifier's max_t, 8, the last prompt token included) to the first window: the
part's earlier tokens are read as before, the first window commits every row,
the drafter gets the rows of the prompt tokens in it as the prompt windows give
them, and only row T-1's output is emitted. Row T-1 sits at the first window's
position with the same history and Philox draw, and a row computes what it
computes in any window while every expert is in VRAM, so the token and the
state are the same.

Only for a BGEN admission with every expert of every stage in VRAM (no loan,
no swap pending, the zero-doorbell graphs), and not with --batch-mtp, the
pipelined windows, pictures, the elastic K/V, a peer device, remote caches,
foresight swaps, STRATA_LOGPOS or a checkpoint at the last token. The merged
window takes no drafter q (STRATA_SPEC_PROB), adds nothing to the draft
counts, and the "PP <n-1>" line is printed before it. Verifier::max_t() is new.

Off by default: the same path as before. Tokens (4x RTX 3090, split
12,24,36, 16 slots, STRATA_IQ_MT_MIN=1): 24 admissions of prompts of 3 to 65
tokens and chat headers with and without an assistant prefill (merged T 3 to
8), greedy, seeded and with penalties, on vs off identical; batch_test16 16/16
and equal to the earlier dump; batch_overlap_test token- and RESUME-identical;
batch_interleave_test equal; 47 sampled driver requests equal.

Measured (24 admissions, ~131 and ~1195 tokens): header windows 23.5 plus
first window 11.5 ms become one 7-token window of 27.6 ms, 7.4 ms less per
admission (send to first batch token 246.8 -> 239.4 ms short, 656.1 -> 648.5
long). A window row costs ~2.4 ms up to T=6 and ~4 ms at T=7 and 8 (T=1 11.4,
T=7 28.6, T=8 32.1 ms).
2026-10-14 06:58:52 +02:00
havok2-htwo a62bd38eba generate: STRATA_PIPE_BYIELD_NODRAIN=1 drops a BYIELD without draining the pipelined groups
A pipelined read never gives way (batch_read_piece returns 0 when piped), so a
BYIELD that reaches the main loop is for a read that has ended and is dropped.
The main loop still drained the groups for it first, as for a request, unless
STRATA_PIPE_OVERLAP was set. With the switch the BYIELD is dropped as it comes.

A BYIELD the server sends during the read is queued and taken right after BADM,
before any group is restarted, so that drain costs nothing today (0 groups in
flight, 0.0 ms). Only a BYIELD that arrives once the groups run again (the
server reads the control lines late) paid for it: measured with one sent 60 ms
after BADM, drain 26 ms with 4 groups in flight and the first rows 50 ms after
it, a 23.5 ms gap in the slots' rows and about 14 rows lost (at 576 rows/s). With
the switch the longest gap is 7.1 ms, as in plain decode.

Off by default: the same path as before. Tokens: batch_overlap_test (a BYIELD
during B's read) on vs off token- and RESUME-identical, batch_interleave_test
output byte-identical, batch_test16 16/16, and 41 sampled requests of the
admission driver (8 with a BYIELD while 4 groups were in flight) equal.
2026-10-14 06:58:39 +02:00
havok2-htwo bc761d06b1 generate: STRATA_TRACE times a batch admission's phases
One stderr line per batch admission ("strata trace: admission of slot N: ...")
splits it into the drain before BGEN, the prompt (batched read, header windows,
the checkpoints' device syncs, copies into RAM and eviction, the rest), the first
window, the slot copy, freeing the slot's old state and the copy of the turn
checkpoint into the slot. Each checkpoint gets its own line, and with pipelined
groups "first rows N ms after the admission" times the refill after BADM. A drain
for another line says how many groups were in flight and what it cost.

Only with STRATA_TRACE set; without it the log and the tokens are unchanged.

Measured (4x RTX 3090, split 12,24,36, 16 slots in 4 groups, 24 admissions):
drain 25-26, header windows 23.5, checkpoint 17.3 (sync 0.1, save 17.2), first
window 11.5, slot copy 1.3, slot freed 0.0, checkpoint copy 7.8 (60 on a slot's
first fill), refill after BADM 23.7 ms.
2026-10-14 06:58:16 +02:00
havok2-htwo 6ab4c31243 generate: more than 8 batch slots are held to 8 rows per group window once the slot sessions that fit are known
The >8-slot cap ran only before the slot sessions were carved. When fewer
sessions fit than asked for, the groups were resolved again with no row bound:
auto could pick a group of more than kVerifyMaxT slots (PGroup's rows overrun,
every window refused) and a --batch-groups N that no longer divides the slots
fell back to one serial group whose windows reach only the first 8 slots.

slots_in_windows() picks the most slots whose groups, resolved as
batch_groups::resolve does (a number given, auto, the default's one group for
--adapt-async 1), divide them with at most kVerifyMaxT rows each and at most
one group's worth per stage. It runs before the groups are resolved and again
on the slots that fit (the extra sessions are dropped like the per-stage trim);
the pump refuses a group over kVerifyMaxT rows as a safety net. The hand-off
buffers hold one row per slot only for pipelined groups, so --batch-mtp keeps
upstream's kVerifyMaxT rows. Log and comment wording without the g3 tags.
2026-10-14 06:57:02 +02:00
havok2-htwo 574dcbf9c3 docs: STRATA_MMQ_DIRECT needs -DSTRATA_MMQ_TAB=ON; its VRAM and build cost; the hack is no bound; engine exactness
- The build option is off by default and why (36-38 MiB of VRAM per GPU under EAGER, ~70% of the upstream MMQ
  instances' compile time per arch); the nine types with a pointer table.
- The skip-gather hack's 225/577 ms are no bound: it changes the routing and does less work.
- Engine exactness as measured: first-window logits (STRATA_DUMP_FIRST_LOGITS) and greedy tokens byte-identical,
  2 vs 1 vs 0, on the 4-GPU split with --prefill 2048 (the streamed walk), with 9 GB less expert cache per GPU
  (mixed in-place, copied and mid-group-flushed groups), and on one GPU with --expert-cache-per-layer.
2026-10-14 06:56:59 +02:00
havok2-htwo 02d0c23a59 docs: STRATA_MMQ_DIRECT (opt-in) and the STRATA_MMQ_TAB build option, with the measurements 2026-10-14 06:56:59 +02:00
havok2-htwo a2d979b999 prefill mmq tab: moe_mmq_tab.cuh carries ggml's MIT notice
The file mirrors parts of llama.cpp's ggml-cuda mmq.cuh; like the other
ggml-derived kernels (native_moe.cu etc.) it now includes the copyright and
permission notice the MIT license asks for. Comment only.
2026-10-14 06:56:49 +02:00
havok2-htwo c2b44ca5b8 expert cache: slot_guard() reports the guard applied; a test for slot_tail on uniform and sized caches
- ExpertCache::slot_guard() returns the guard open_sized applied (guard_, reset by close), not the request: 0 for
  uniform slots, when closed, and on SYCL, whose open_sized applies none.
- tests/core/expert_cache_slot_guard_test: slot_stride, slot_room and slot_tail on a uniform cache (empty off_,
  including the last slot) and on sized caches with and without the 256 B guard; no guard gives today's offsets;
  close and reopen.
- generate: the "+256 B guard per sized slot" line only for a sized cache; a uniform one says it has no guard.
- prefill: STRATA_MMQ_DIRECT_LOG=1 (with STRATA_MMQ_DIRECT=1) prints each stage's totals of experts read in place,
  resident and streamed after every prompt, so an engine run can show which walk read what.
2026-10-14 06:56:49 +02:00
havok2-htwo a5ad4077ef prefill mmq tab: STRATA_MMQ_TAB off by default, the nine tested types only, not on gfx906
Review fixes for the pointer-table MMQ build:
- STRATA_MMQ_TAB defaults to OFF, like STRATA_MMQ_KQUANTS: the engine loads every kernel at start
  (CUDA_MODULE_LOADING=EAGER), and the nine tab instances took 36-38 MiB of VRAM per GPU (sm_86, measured with
  a cudaMemGetInfo probe linked with and without the tab objects; equal under LAZY), with STRATA_MMQ_DIRECT unset
  too. prep_group_kernel is compiled only with STRATA_MMQ_TAB, so a build without it has the same device code as
  3adce85 (the 58 cubins and PTX of build/strata are byte-identical to engine/strata-g3b's).
- The fingerprint block and the tab TUs are skipped on STRATA_HIP_GFX906: that build compiles no .cu from here, so
  the define, the uncompiled TUs and an unlinkable prefill_mmq_tab_test no longer appear.
- Tab instances only for Q2_0 IQ2_XXS IQ2_XS IQ2_S IQ3_XXS IQ3_S IQ4_NL IQ4_XS Q8_0, the types prefill_mmq_tab_test
  checks bit for bit; Q5_0 and the K-quants (STRATA_ORCA_Q4KS_MMQ, STRATA_MMQ_KQUANTS) keep the gather.
- overread_bytes takes the largest over-read of any config, not the over-read of the largest K tile (the over-read
  is not monotonic in the tile length).
- The test fails (instead of skipping) when a tested type has no tab, checks that Q5_0/K-quants report none, and
  pins the sm_86 over-reads (0, 72, 136 B); the dead K-quant scale branch is gone.
- SYCL's moe_mmq.dp.cpp gets the STRATA_MMQ_DIRECT stubs (false/0), for a future migrated prefill.cpp.
- moe_mmq.hpp points at docs/DETAILS.md.
2026-10-14 06:56:49 +02:00
havok2-htwo 0f10feb4d2 prefill_mmq_tab_test: pad the identity row map for memcheck
MMQ loads a whole tile's ids, and its stream-k fixup one per column tile of max_rows, past an
expert's last row (upstream behaviour on both paths; the values are not used). With the map
padded, compute-sanitizer memcheck runs the whole test clean: 0 errors, 874/874 bitwise equal.
2026-10-14 06:56:49 +02:00
havok2-htwo 20f75a7474 prefill mmq tab: the table as 16 restrict kernel parameters, so the weights keep LDG.CONSTANT loads
A pointer taken from a dynamically indexed array parameter (mmq_x_tab by value) made nvcc copy
the table to local memory and read the weights through generic LD.E loads instead of
upstream's LDG.E.*.CONSTANT: the products ran 1.2-2.5x slower than upstream's in the unit test
(Q8_0 down 2.2-2.6x, IQ4_NL 1.7-2.1x, IQ3_S 1.2-1.35x), which ate most of the gather saving.
The kernel now takes x0..x15 as `const char * __restrict__` parameters (like upstream's x)
and picks an expert's with a select chain: same SASS load mix as upstream (no local memory),
kernel time within 1% of upstream over every type and J in prefill_mmq_tab_test (69.7 vs
69.3 ms in total), and still 874/874 comparisons bitwise identical.
2026-10-14 06:56:49 +02:00
havok2-htwo 7d9b50dc1a prefill: STRATA_MMQ_DIRECT=1 reads resident experts in their cache slots (pointer-table MMQ)
mmq_plan marks a native layer direct when the pointer-table MMQ covers both products, gate/up
reads nothing past its matrix, the down over-read (72 B IQ4_NL, 136 B Q8_0) fits the 256 B
guard, and the blob is exactly [gate | up | down] in MMQ's matrix sizes; it says once how many
layers qualify.

In the MoE walk (both the staged and the streamed walk) compute() takes the expert's cache slot:
a resident expert whose slot has room for the over-read (slot_tail) and a 16-byte aligned blob
is not gathered - the group's pointer tables point at its gate|up and down in the slot, and the
guard goes into the group's GuardTab. Streamed, peer and unguarded experts are gathered as
before, at compact positions c = 0..ns-1 (gather_native, or gather_native_group with its
mid-group flushes unchanged); the end-of-group flush runs even when the last entry was direct.
At the group end one prep_group launch zeroes the direct experts' guards and the tails after
position ns, replacing the two memsets; both products run with the same n, bounds, ids,
max_rows and buffers plus .tab. STRATA_MMQ_DIRECT=0 (default) and 2 run the existing body.

PrefillStats::experts_direct, said on the generate summary when non-zero.
2026-10-14 06:56:48 +02:00
havok2-htwo b82cdb1090 expert cache: a slot guard for STRATA_MMQ_DIRECT (set_slot_guard), the sizing sites use slot_stride
ExpertCache gets inline set_slot_guard / slot_guard / slot_stride / slot_room / slot_tail; the
4-argument open_sized declaration is unchanged (the SYCL engine defines it out of line against
the same header and ignores the guard). open_sized lays slots out at slot_stride(blob, guard);
blob_ stays the largest blob, so fills, verify_slot and the HIP staging read no further.
slot_tail works on uniform caches too (through slot_offset).

generate.cpp: slot_guard = Prefill::slot_guard_bytes() (mmq::direct_slot_guard(): 256 when
STRATA_MMQ_DIRECT is 1 or 2 on a pointer-table build, else 0) for native packs; the seven
round-256 slot costs (split-skip-if-fits, split placement, per-layer sum, profile sizing,
shrink_to, cache_bytes, stage sizing) use ExpertCache::slot_stride(b, slot_guard), and the
primary and stage caches get set_slot_guard before they open. Guard 0: the same arithmetic as
before, so the default layout is unchanged byte for byte.
2026-10-14 06:56:15 +02:00
havok2-htwo 86f677b4a9 prefill mmq: pointer-table MMQ (STRATA_MMQ_TAB), prep_group, the bitwise test
A Strata-side copy of llama.cpp's MMQ outer scheduler (mul_mat_q, launch_mul_mat_q,
mul_mat_q_switch_J; fallback=false only) in src/prefill/moe_mmq_tab.cuh: each expert's base
address comes from a 16-entry table passed by value instead of x + z * stride_channel_x.
The launch geometry (J, tiles, stream-k grid and k split, fixup) is computed from the same
mmq_args, and process_tile / load_tiles / vec_dot / write_back / the fixup kernel are
upstream's own templates, so the results are the same bits. Instantiated per type from
mmq_tab_instance.cu.in in strata_mmq (same flags); moe_mmq.cu sees declarations only.

CMake turns it off unless the SHA-256 of mmq.cuh's mirrored region (mul_mat_q_process_tile
through mul_mat_q_case) is a verified one: 3cf03257 (the fetched pin; the vendored tree's
mmq.cuh is byte-identical). HIP and SYCL get nothing; Product::tab is a default-null field.

Also: overread_bytes, tab_fits, direct_mode (STRATA_MMQ_DIRECT, read once, 0 unless the tab
is built), direct_slot_guard, and prep_group (one launch: zero the direct experts' guards and
the two group tails). Nothing calls them yet; the default path is unchanged.

tests/cuda/prefill_mmq_tab_test.cu (STRATA_BUILD_TESTS): 874 comparisons on an RTX 3090, all
bitwise identical - IQ3_S gate/up, IQ4_NL and Q8_0 down (1..16 experts, rows
{0,1,7,8,9,31,64,100,512,1000}, every J 8..128, stream-k fixup on and off, max_rows above the
largest count), every other instanced type, mixed direct/gathered groups (compact c != q,
mid-group flush, last entry direct); negative control: guards left as fp16 NaN give non-finite
down outputs.
2026-10-14 06:56:14 +02:00
havok2-htwo dd5443646c batch: device-to-device slot copy for admissions and slot resumes
The main sessions and the batch slot sessions of a stage live on the same device and are
carved alike, so copy_to_slot / copy_from_slot (without a checkpoint) now copy the running
state and the QSA K/V directly (conversation_checkpoint_copy, conversation_kv_copy:
the snapshot code's private non-blocking copy stream, one device sync before and after) instead of the
checkpoint/K-V image round trip through host memory with a sync per layer. Same bytes into
the same places in the same order; the residency refresh for streamed K/V is shared with
conversation_kv_restore. Opt-in: STRATA_SLOT_COPY_DIRECT=1 (default: the host path,
as upstream).

4x RTX 3090, UD-IQ4_XS, --batch 16, split 12,24,36 + --trim-stage-weights: admission copy
65.6 -> 34.0 ms average (pure copy ~8 ms); tools/batch_test16.py 16/16 and
batch_interleave_test.py (incl. next turn from its slot) identical to solo.
2026-10-14 06:55:40 +02:00
havok2-htwo ee2a907e30 batch: more than 8 slots with a layer split (8 rows per pipeline group), hand-off sized per slot
Measured on 4x RTX 3090 (UD-IQ4_XS): 16 and 32 slots decode bit-identical to solo
(tools/batch_test16.py / batch_test32.py); 16 slots 603 tok/s batch decode vs 345 with 8.

Rebased on 0.1.42: the stages' batch row check (batch_rows_error, upstream 48f2c47d) bounds
hbase + S by the hand-off rows; set_handoff_rows passes the per-slot size there (only above 8 slots).
2026-10-14 06:55:26 +02:00
Niko1221andDemetree Kallergis b8ec9bbf94 telemetry: the Mac's Monitor cards - GPU load and VRAM from IOKit, the rest said out loud
On a Mac the Monitor tab's GPU cards were empty: telemetry had NVML (NVIDIA) and the amdgpu
sysfs files (#301) and nothing else, so the Metal backend read nothing.  The Apple GPU's load
and memory are in the IOKit registry's IOAccelerator nodes (ioreg -l -w0 -r -c
IOAccelerator, 22 ms), and the device's memory budget is the same Metal property the engine
reads (metal/src/mtl_rt.mm's recommended_working_set, the engine's "GPU working set" log line
- 26,800,603,136 bytes here, which is 25.0 GiB).

_Metal (a new reader beside _Nvml and _Amd): the index'th accelerator node's PerformanceStatistics
give Device Utilization % (older chips: the renderer or the tiler) and the in-use system
memory; the total is the working set, read through the Metal framework and objc_msgSend (no
PyObjC).  The temperature and power sensors are not readable
without root (powermetrics), and an Apple Silicon GPU sits on the die, so there is no PCIe
link: the cards show a dash and static gpu_note says why, the same way #1380's note does for a
Windows AMD card.  gpu_reader takes `amd` first, so the AMD trees still read on a Mac for the
tests, and a missing selector is asked for before it is sent (an unanswered one raises an
Objective-C exception no ctypes call can catch).

Service.free_vram_mib now goes through gpu_reader instead of _Nvml directly, so the Mac gets
that reading too (it used to return None there).

Measured on an M1 Max (macOS 26.6.2): through the server, GET /metrics during a request -
gpu_name "Apple M1 Max", gpu_vram_total_mib 25559, gpu_util 74-77 % while decoding (9 % once
idle), gpu_mem_used 10.5 GiB in use of 24.96 GiB, cpu 12.6 %; the plain-text
/prometheus-format metrics carry the same series.  serve/test_server.py: MacTelemetry
(fixture registry: two accelerators, index selection, renderer/tiler fallback, a raising
runner, the note and the samples) plus AmdTelemetry, TelemetryNoDisks, Monitor and the full
serve/test_server.py (318 tests) pass on the same box.

What it leaves alone: the NVIDIA and AMD paths (a test-only gpu_reader order change on
macOS), and the web app - the cards read the series it already had.

Co-authored-by: Demetree Kallergis <[email protected]>
2026-10-14 06:52:38 +02:00
nekomario28 be225ae20c setup: honour --source on a Mac, whose hand-over skips setup.py's parser (#2363)
setup.py returns from mac_setup() before its own argparse runs, so a Mac never
sees the --source that sets STRATA_SOURCE, and the Mac parser did not define it
either: the flag the download failure recommends landed in parse_known_args'
leftovers and was dropped with "not used on a Mac".

Define it in the Mac parser and set STRATA_SOURCE from it, mirroring setup.py.
2026-10-14 06:52:38 +02:00
Niko1221andDemetree Kallergis 2e98a86344 mac docs: the two prompt opt-ins an M1 Max cannot run (measured)
Recorded with the hardware they were measured on, per the measured-numbers rule, because
neither can be fixed here: mm_gu3's fault is a threadgroup-count threshold of the
cooperative-tensor path on this GPU (the fix shape would change the M5's register-operand
GEMM), and router_route's race has no acquire atomic to fix it with on this Metal
(device-scope atomics are relaxed-only, macOS 26.6). Both are opt-in and off by default;
the engine's M1 defaults are the v2 expert GEMMs and the separate gemv + moe_route, which
run on this box.

Measured on an M1 Max (macOS 26.6.2, Q2_0): mm_gu3 with 35 threadgroups passes and 40
faults (data-independent; the engine's tile 16x32x32 SG 4 and the v3 test's 190 tiles both
fault). router_route's logits are bitwise gemv's, but two identical runs give different
ids, weights and plan.

Co-authored-by: Demetree Kallergis <[email protected]>
2026-10-14 06:52:38 +02:00
Niko1221andDemetree Kallergis ad7345311a moe_mm_parity: the default tile that did not exist, and --v3's v1 argument list
Two stale defaults, both of which kept the test from covering what the engine ships.

The default tile was TM 16 TN 64 TK 16, but the v1 kernel set only instantiates TK 64: the
default run died at pipeline creation on any GPU (measured on an M1 Max:
"mtl: no function mm_gu_42_16_64_16_2"). TK is now 64, which both the v1 and the v2 sets
instantiate, so the default run and --v2 both work; the engine's v2 tile for Q2_0
(16,64,16,2) stays reachable with --tile.

--v3 bound the v1 kernels' argument list ({W, x, H, stok, tiles, ma}) to the v3 kernels:
tiles was read as the sorted-token buffer and MmArgs as raw int2 pairs, so every stride,
n_embd and expert id was garbage. On the M1 Max that is a deterministic GPU
command-buffer error in the GEMM pass; with the right arguments the v3 kernels run, and a
separate M1 limit shows (≥40 threadgroups of mm_gu3 fault, 35 pass, data-independent)
that the engine does not hit: gu_v3 = dn_v3 = coop_k4, which is false on the M1.
v3 now binds {W, XS, HS, tiles, ma} as the engine does, fills XS for v3 too (v3 is v2's
gather plus the register-operand GEMMs), and binds HS - not H - to down3 with DSG
simdgroups.

Measured on an M1 Max (macOS 26.6.2, Q2_0, T 256, 64 experts): default (v1) PASS, rel L2
2.53e-06 vs the fp16-operand reference and 2.37e-04 vs exact; --v2 PASS, 4.42e-06 and
2.37e-04, also with the engine's tile (16,64,16,2).

Leaves alone: no kernel, engine or other test change.

Co-authored-by: Demetree Kallergis <[email protected]>
2026-10-14 06:52:38 +02:00
Niko1221andDemetree Kallergis 831ba47afd metal: the M1 Max prompt kernels - a select with a device load in the arm loses the threadgroup
qsa_attn_mm and qsa_flash (metal/kernels/qsa_prompt.metal) fill their single-simdgroup
matmul2d right inputs with

    rk[i] = id >= 0 ? half(float(kq[row * QD + k]) * float(ksc[...])) : half(0.0f);

and that shape - a select with the device load in the arm - loses the threadgroup on an M1
Max: every dispatch of the two kernels fails with
kIOGPUCommandBufferCallbackErrorInnocentVictim, deterministically, and they run only under
MTL_SHADER_VALIDATION=1. qsa_prompt_parity dispatches qsa_attn_mm the way the file intends
(it is the matrix-unit attention of a prompt chunk) and faulted on its first dispatch before
this change. The engine reaches the same fills through qsa_flash, which it dispatches under
STRATA_METAL_PROMPT_QSA=flash; its default on an M1/M4 is the fp32 qsa_attn, which never
touches these kernels - that is why the default prompt path kept running while the parity
test could not, and why this is measured on the test rather than on a request.

The row is now clamped with max(id, 0), the load is unconditional, and the select is on the
value. For an id >= 0 the value is what it was; for an id < 0 it was discarded either way, so
the results do not change.

Measured on an M1 Max (macOS 26.6.2, Metal compiler 32023): qsa_prompt_parity ALL PASS -
qsa_attn_mm vs qsa_attn rel L2 3.09e-04 (pos0 0) and 3.56e-04 (pos0 30000), qsa_flash 3.72e-04
and 4.22e-04, qsa_score_mm worst rel 2.66e-06 (T 96, within budget and with 2051 selected
cells a token); before the change the test faulted on its first dispatch and reported nothing.

The compile failure that started this - qsa_flash's cooperative_tensor copy, "cannot
initialize a parameter of type 'void *' with an rvalue of type 'const void *'" - is already
fixed in main (#2323); this change is the runtime half of the same two kernels' M1 Max
bring-up.

What it leaves alone: the COOP_K4 path (a function constant: compiled out of the M1/M4
pipelines), qsa.metal, and every host file; no CUDA, HIP or SYCL backend builds these kernels.
docs/MAC.md records the M1 Max with the measured tok/s (the hardware's medians, from the
benchmark report: chat decode 10.6, 4K prompt 223.7, 16K prompt 228.6 tok/s).

Co-authored-by: Demetree Kallergis <[email protected]>
2026-10-14 06:52:38 +02:00
Kingdon Barrett f037612bad fix(metal): the prompt's buffers get their own allocation when no arena chunk is big enough for them
stream.cpp: the standalone-mmap fallback (the prompt's buffers as their own allocation) sat inside the chunk loop behind the slot-class check, so it only ran when a chunk was big enough for the prompt's 1.60 GB of buffers. When the arena shrank below that at the start (the 24 GB M2: chunks of 0.8-1.0 GB), every chunk was skipped and the prompt stopped with 'no arena region for the prompt buffers'. Both standalone sites - a fitting chunk whose slots a class cannot spare, and no fitting chunk at all - now share standalone_region(); the fitting-chunk carve and the slot-class rule are unchanged.

This is the half of #2323 that 0.1.44.1 could not see: the hotfix's reporter ran a 96 GB M2 Max, where the arena's chunks are several GB and the prompt region always fits. On a 16-24 GB Mac running --mmap-experts, 0.1.44.1 still stops at the first prompt with 'metal prompt: no arena region for the prompt buffers'; this makes the standalone allocation run there too.

Verified on a 24 GB M2 (macOS 26.6.2, Qwen3.8-Flash-Next GSQ-RCO-Q2_0 streaming): the fallback line appears where the error used to be, prompts run end-to-end (3.7-4.1 tok/s steady in web chat, no crashes). That run predates the standalone_region() refactor, which moves the same statements into one function both call sites use.

docs/MAC.md: the small-arena note says the prompt's buffers also get their own allocation when no chunk of the arena is big enough for them.
2026-10-14 06:52:38 +02:00
Tsury e69cbfa568 fix(serve): preserve explicit false against reasoning defaults 2026-10-14 06:52:38 +02:00
nekomario28 69fe40553d tools: poll the server address the config binds, not loopback (#2330)
strata_bench starts the server on the config's `host` but polled readiness at
a hardcoded 127.0.0.1, so a LAN bind (192.168.1.10 in the report) printed
"still loading" while the server was answering on that address.

Use the configured host, mapping the wildcard binds to loopback and bracketing
IPv6 literals; an explicit --url still wins. Patch from the report, with the
mapping lifted into server_base so a test can reach it.
2026-10-14 06:52:37 +02:00
nekomario28 a091cdf681 serve: average the throughput gauges over the window between scrapes (#2341)
/metrics/prometheus exposes llamacpp:prompt_tokens_seconds and
llamacpp:predicted_tokens_seconds as averages of the cumulative totals, so an
idle server keeps advertising the speed of its last request.

llama.cpp reads those two names from a bucket it resets on every scrape ("the
gauges are averaged over the window between two scrapes", server_metrics::bucket
in tools/server/server-common.h) and reports 0 when the bucket is empty, so a
llama.cpp dashboard on Strata froze at the last value where llama.cpp decays to
0. Keep a bucket per scrape and expose its delta.
2026-10-14 06:52:37 +02:00
Niko1221 db4dd65e56 Merge f-q36-vp (Qwen3.6 parallel requests and pictures) 2026-10-14 06:51:43 +02:00
Niko1221 1a900ae160 Merge f-14412 (setup: UD-Q5_K_XL metadata-only shard check, #2346) 2026-10-14 06:51:43 +02:00
Niko1221 6ca7e92968 setup: a metadata-only GGUF shard that ends at its header is whole (#2346: Unsloth's UD-Q5_K_XL 00001 is 10,946,618 bytes, its aligned data start 10,946,624) 2026-10-14 03:17:43 +02:00
Niko1221 95eefd51f0 tools/qwen35: q36_vision_proto says to start the engine without --mtp for its cache check 2026-10-14 02:40:00 +02:00
Niko1221 1e13cbc8a5 qwen35: keep the base engine's end-of-answer accounting (hybrid text identical), a stop keeps only x, the held tokens list every accepted draft
An earlier commit capped the accepted drafts at the end of an answer; that is an exact state, but it moved the hybrid mode's expert-slot
counts, so a sequence of requests with --mtp on a card that does not hold every expert differed from the base engine from the fourth
request on (found by comparing the 10 parity prompts against the base goldens on the 12 GB card).  Now the speculative step commits what
the base engine committed; a stop seen before the state moves keeps only x (the server's hand-over of a stopped request to a batch slot then
reuses the held state: its text equals the run alone), and the held tokens list x and every accepted draft, so a conversation that continues
from them matches the state exactly.  Setup says what parallel measured on the 12 GB card; docs/QWEN36.md updated.
2026-10-14 02:28:13 +02:00
Niko1221 a7027766ff docs/QWEN36.md: the long picture, two pictures, reads beside decoding slots, the hand-over; the picture comparison takes several pictures 2026-10-14 01:54:21 +02:00
Niko1221 296b59d2f7 Merge f-1441-perf (Responses/Codex honours reasoning_effort; setup keeps the installed size on a re-run) into f-1441 2026-10-14 01:51:12 +02:00
Niko1221 3921f57eea qwen35: a conversation's held tokens match its state exactly (no drafts past the end of an answer or a stop), reads give the decoding slots windows, pictures x2 check
spec_step takes no draft past an end-of-answer token or past the room max_new leaves, and a stop asked for before the state moves keeps
only x: every token emitted but the last is in the state, so a conversation that continues from the held tokens (the server's solo ->
slot -> solo hand-over, a chat's next turn) neither misses nor reads a token twice.  Before, an answer ending inside accepted drafts left
the state ahead of the held tokens (found by tools/qwen35/q36_serve_handoff.py: the first request's text now equals its run alone).
STRATA_Q36_READ_WINDOWS sets the decode windows of the other slots between two chunks of a prompt read (default 1).  The mtmd dump takes
several pictures; the engine logs each request's reuse; a 1,024-token test picture.
2026-10-14 01:47:11 +02:00
Niko1221 0ca1dbbaa2 pool_tasks_test: skipped on a CPU without AVX (its activation quantiser is the AVX one) 2026-10-14 01:19:42 +02:00
Niko1221 944c88314b Merge f-1441-mac (Metal: gdn_step fits 512 threads, every pipeline's thread limit checked, qsa_prompt compiles on the macOS 26.3 headers; #2323) into f-1441 2026-10-14 01:19:41 +02:00
Niko1221 dd9cbce1a2 Version 0.1.44.1: project version and the setup minimum engine 2026-10-14 01:19:41 +02:00
Niko1221 9aa674edf8 serve: the illegal-instruction note says what the CPU reports (a missing AVX/AVX2/FMA/F16C, or that it has them all and the engine is at fault) 2026-10-14 01:19:41 +02:00
Niko1221 d981a51013 serve: a crashed engine start is not retried as a GPU runtime start; the start error names how the process ended (signal / NTSTATUS) and the CPU's AVX flags after an illegal instruction (#2324) 2026-10-14 01:19:41 +02:00
Niko1221 0b6fc1cec7 nvfp4 CPU rows: no static initialisers in the AVX2/AVX-512 files (an illegal instruction before main on a CPU without AVX/FMA, #2324); check_isa_guard flags AVX-class static initialisers 2026-10-14 01:19:41 +02:00
nekomario28 ab638a5684 qwen35: widen F16 by hand so upload_f32 builds on CUDA 12.1
upload_f32 runs on the host and widens F16 with
__half2float(__ushort_as_half(...)), but __ushort_as_half is a __device__
intrinsic: nvcc 12.1 and 12.4 refuse the call ("calling a __device__ function
from a __host__ function is not allowed") and q36_engine.cu does not compile.

Widen the bits here instead, the way the neighbouring bf16_to_f32 already
does for BF16.
2026-10-14 01:19:41 +02:00
Niko1221 de9115fac7 setup: run again over an installed model keeps its size; the three env entries 0.1.44 made setup's own stay the user's in a GSQ-RCO config
0.1.44 made IQ3_S the recommended Qwen size on a PC with 64 GB or more (IQ3_S: 9 to 15% slower decode than IQ3_XXS, more RAM).
SETUP.bat / --setup on a folder that already holds a model of the family now pre-selects the installed size (and --yes takes it);
the menu marks it "(installed)" next to "(recommended)", every size stays a choice, a first install takes the recommendation.

SETUP_ENV gained STRATA_MMQ_KQ, STRATA_HC_Q8 and STRATA_ROUTE_TAIL_SKIP, which setup writes for Unsloth's files only; a setup
run again dropped a user's own value of them in any other config (a hand-set STRATA_ROUTE_TAIL_SKIP=0).  They are setup's only
in an Unsloth config.
2026-10-14 01:16:56 +02:00
Niko1221 5d08c753c2 serve: the config's reasoning_effort reaches the Responses API too; reasoning_effort false is a level; serve/test_effort_paths.py pins every request form's prompt
/v1/responses (Codex) read only the Chat settings shared with apps, so the config's "reasoning_effort" (#1641) never applied
there (it always thought at xhigh); Service.effort_default() is now the one place that says which level a request that names
none gets, for the chat, Messages and Responses routes.  openai_to_messages dropped "reasoning_effort": false (an `or` between
the two fields); effort_kwargs already treats false as thinking off.

The new test renders the prompt of every way a client sets the level (OpenAI reasoning_effort / reasoning.effort /
chat_template_kwargs, Messages API output_config.effort / thinking budgets at the 2048 and 8192 limits / disabled, Responses),
the config key, the shared Chat settings and their precedence, the web app's request shape, tools and a conversation.
2026-10-14 01:16:56 +02:00
Niko1221 cfc7686c00 tools/qwen35: the Qwen3.6 pictures and parallel-requests gates (engine protocol harness, solo / batch identity, protocol edges, llama.cpp picture comparison, throughput A/B) 2026-10-14 01:11:42 +02:00
Niko1221 a9adfa5dad Qwen3.6-35B-A3B: setup asks for pictures and recommends parallel requests; docs
Setup: the images question for qwen36 (GPU encoder on an NVIDIA card of 16 GB or more, the CPU encoder otherwise, AMD under Linux on the
CPU; --vision gpu kept as asked with a note), the pinned mmproj (size and SHA-256) beside the GGUF, the vision section and --vision in
the config; --parallel N is written as asked with what a slot costs, and the recommendation is said where every expert still fits.
docs/QWEN36.md: pictures and several requests at once with the measurements.
2026-10-14 01:11:11 +02:00
Niko1221 7419037255 qwen35: --batch-mtp (each slot verifies an MTP proposal in the batch window) and a clear refusal when the slots' caches cannot fit
A slot's window carries its next token and, for the greedy slots that fit beside one row per slot, the proposal of its MTP block;
the proposals for the next window come right after the window from its own hidden states (the first from the state the prompt
left).  Grouped GDN rows (a slot's rows in sequence on its own state), verify windows and their commit run per slot; every slot's
tokens equal its solo run.  The engine also says what the sessions take and stops with the numbers when they cannot fit.
2026-10-14 01:11:11 +02:00
Niko1221 b4ef48e941 qwen35: sessions, parallel requests (--batch N) and pictures (--vision, M-RoPE) in the Qwen3.6 engine
One conversation state per session; --batch N keeps N of them and decodes one token of each in a single window (rows of
several sessions in one forward, bit-equal to the solo rows), reads a prompt between the other slots' windows and takes a
conversation over from the session that holds its start.  GENI / BGENI pictures: the encoder's rows replace the embeddings
of the <|image_pad|> cells, positions as llama.cpp's mtmd (interleaved M-RoPE 11/11/10, text after a picture continues at
p + max(nx, ny)).  tools/qwen35/q36_mtmd_dump.cpp is llama.cpp's side of the picture check.
2026-10-14 00:46:16 +02:00
Niko1221 5ce999455b Metal: gdn_step fits its launch size, every dispatch checks maxTotalThreadsPerThreadgroup, qsa_prompt compiles on macOS 26.3 (#2323) 2026-10-13 23:21:28 +02:00
Niko1221 ed4773eb7c README translations: Mac install steps and the two arena opt-ins (#1904, #1905) 2026-10-13 21:23:37 +02:00
Niko1221 2c4005cee0 README: how to install on a Mac, and how to turn on the two arena opt-ins (#1904 --adapt-async 1, #1905 STRATA_PIN_BY_NEED) 2026-10-13 21:05:49 +02:00
Niko1221 c7cbf787ab Merge f-144-ruben (RubenTorney's #1904 --adapt-async in the arena and #1905 pin by need, as opt-ins) into f-144-rel 2026-10-13 20:22:43 +02:00
Niko1221 3b788e183f docs: STRATA_ADAPT_OVERLAP never runs on a layer split, in 0.1.43 either (#2281)
0.1.43's gate already had 'not on a layer split' as its first predicate; it printed the line only when the variable was set. A default 0.1.43 split ran without the overlap, so 0.1.43.1's opt-in default changes nothing on a split. Measured on 2x A4000 (IQ3_XXS, layer split, 5 interleaved pairs): v0.1.43, v0.1.43.1 and v0.1.43.1 + STRATA_ADAPT_OVERLAP=1 decode within 0.6% of each other.
2026-10-13 19:32:36 +02:00
Niko1221 af452c1ecb pinned: load_experts_ranges with a PadMap is an overload, not a default argument - the SYCL build's own pinned.dp.cpp defines the 7-argument signature only (the b70 link failed with an undefined reference) 2026-10-13 16:52:15 +02:00
Niko1221 f087f780ed arena: pin by need - the cap's leftover goes to the pairs taken as GPU-resident (a second tier of ranges, never merged with the first: the prompt read streams whatever the adaptive tier has swapped out of the cache) 2026-10-13 15:59:05 +02:00
Niko1221 d6bd7c1d62 Merge f-mac-rel ccfe67aa (Mac memory-safety fix for big Macs) into f-144-rel 2026-10-13 15:10:49 +02:00
Niko1221 ccfe67aa07 metal_repack.py: say what F_NOCACHE does on macOS (writes uncached, reads still cached: measured) 2026-10-13 15:08:55 +02:00
Niko1221 55e1283174 arena: pin by need - blob_at (the async tier's source) follows the padded layout; no padded layout with a shared arena file (one layout for every process) 2026-10-13 14:53:19 +02:00
RubenTorney ab6bce05de arena: pin by need (STRATA_PIN_BY_NEED, opt-in) + PCIe share floor (STRATA_PCIE_MIN1, opt-in)
On Windows cudaHostRegister of the whole arena fails (the WDDM shared budget is about half the RAM) and the
sliced fallback registers the first ~30 GiB: with Qwen3.8-Flash-Next IQ3_S layers 0-31, so layers 32-47 never
get a PCIe share, while part of the registered prefix holds GPU-resident experts that DMA never reads.

STRATA_PIN_BY_NEED=1 (the arena on one GPU):
- every blob gets a page-aligned stride (PadMap in pinned.hpp, ArenaExpertSource::arena_off, +0.15 % RAM), so any
  set of blobs can be registered exactly; the loaders place each blob at its padded offset
- the expert profile's pairs past the estimated GPU-resident count are merged into ranges by rank and registered
  before any page is touched, up to STRATA_ARENA_PIN_GIB (29 GiB by default on Windows) or the first refusal
- pinned() / device_alias() / pcie_layer() work on the ranges (a blob is pinned only if wholly inside one);
  lock_resident_ranges() keeps the holes between them resident
- one start-up line: "pin by need: N ranges, X GiB pinned, Y % of the non-resident experts"

STRATA_PCIE_MIN1=1: the PCIe share (nmiss * pcie_num) >> 8 rounds to 0 at 1-2 misses per layer; with two or more
misses one goes over PCIe (when its blob is pinned).

RTX 3090 + Ryzen 5 5600X: 28.96 GiB registered = 92 % of the non-resident experts, all 48 layers with a PCIe
share; prompt read +5-7 % at 56K-117K tokens, decode level alone and +2-3 % with --adapt-async 1 and
--pcie-frac 0.6 (bench/results/2026-10-11-community-rtx3090-5600x-arena-pin-by-need, docs/DETAILS.md).

Both opt-in; without them the layout, the loaders and the sliced pin are those of v0.1.42.
2026-10-13 14:50:46 +02:00
RubenTorney 1aa0abf63a serve: --adapt-async 1 in the arena mode (one GPU, copy-in only, opt-in)
In the arena mode the blocking adaptive tier creates and joins a thread every verify window and the next window
waits for its copies (#1863: the time outside the decode timing breakdown). --adapt-async 1 now also runs with the
arena on one GPU, not only in the resident RAM mode. The arena holds every expert, so a round is a copy-in only:
no copy back, no RAM move, no exchange buffers.

- gates for the arena; ArenaExpertSource::blob_at() reads a blob from the job thread without the read counter
- the evicted slot is marked non-resident and uploaded before the round's copies are posted; an expert becomes
  resident only once its copy's event has completed
- STRATA_ADAPT_ASYNC_WAIT=1 (arena default): the next tick waits for the round, so the window an expert turns
  GPU-resident does not depend on timing; STRATA_ADAPT_PICK_INLINE=1 (arena default): one round per window;
  STRATA_ADAPT_ASYNC_BATCH=N caps the copies per tick (default 0 = all); arena rounds are not capped at 96
- adapt_quiesce(): every K/V grow, kv_quiesce and the VRAM command land the round in flight (they only called
  apply_pending(true)); a failed round still drains its copies. This also covers the resident mode's async tier.
- STRATA_DECODE_TIMING: "adapt wait X join Y, Z swaps/window"; --adapt-async help text

RTX 3090 + Ryzen 5 5600X, cold prompts: x1.07-1.11 decode, faster on 12 of 12 prompts; 25.9 -> 21.6 ms per window
(bench/results/2026-10-11-community-rtx3090-5600x-arena-async, docs/DETAILS.md).

Opt-in (--adapt-async 1, default 0); the default path is that of v0.1.42. With the tier on, answers are not
bit-exact run to run (which window first computes a swapped-in expert on the GPU).
2026-10-13 14:50:46 +02:00
Niko1221 62be0f3843 metal stream: wire the filled arena with mlock (a file write beside the idle engine froze the M5 Max in a control run)
The GPU's own wiring of the arena ends seconds after its last command buffer, and a Metal residency set did not hold it either
(wired memory 75 -> 3 GB between requests, measured with vm_stat and a probe). Idle, the arena is ordinary anonymous memory, and a
20 GB file written beside it made macOS compress tens of GB of it: the third panic (wired 2.4 GB, file cache 50 GB, compressor 1.9M pages).
mlock keeps it wired (STRATA_METAL_WIRE_ARENA=0/1, default on from 64 GB; not when the fill stopped early). Same write, arena wired,
full speed: compressor +0.7M pages (the engine's non-arena memory), 408 pages swapped, nothing else.
docs/MAC.md: the footprint cap, the wiring and the new variables.
2026-10-13 14:04:22 +02:00
Niko1221 caac711395 setup: UD-* on a card of about 8 GB - estimate the VRAM the engine needs, write the least harmful config that starts (4-bit KV, smaller context, then no draft layer on the GPU), say so, warn that UD-Q4 is very slow there and suggest IQ3_XXS / IQ3_S; tests 2026-10-13 13:51:01 +02:00
Niko1221 e2bf3d36a2 metal: a hard cap on the engine's whole resident footprint, and no engine beside a repack (the two M5 Max watchdog panics)
Both panics (watchdogd starved 92 s) had the streaming engine at 95.3 / 89.9 GiB resident, free memory 13 MB, the compressor at 2-4M pages.
macOS keeps a file-cache floor (vm.vm_page_filecache_min, 37.2 GB on the 128 GB M5 Max) and compresses anonymous memory instead of dropping
file cache below it; the arena is anonymous and the GPU wires it only while a command buffer runs, so during the fill and when idle it is
pageable. Measured with a 1 s memory watchdog: old build + 100 GB arena, or auto arena + 40 GB file write + a 16 GB program: compressor 21k -> 4.07M
pages in 10 s, 29,000 pages swapped, pressure warn. The 67% rule counted only cx.allocated() (7.5 GB) while the process held 11-15 GB more, and
67% was too much anyway.

- arena_budget.h: footprint_cap (anonymous memory: arena + Metal buffers + what the process holds, task_info phys_footprint) = 60% of the RAM on
  a Mac of 64 GB or more (and at least the file-cache floor + 12 GB left); resident_cap (a resident pack's mapped file pages, measured: UD-Q4 resident,
  wired 91.7 GB, 40 GB write + 16 GB program, nothing compressed) = 70%; STRATA_METAL_LEAVE_SHARE for measuring. The explicit
  STRATA_METAL_ARENA_GB is kept and flagged when it is over the cap.
- window.cpp logs the planned total (arena + held + n-gram reserve against the cap) and the footprint after the fill.
- the engine refuses to start while a repack or setup's model preparation holds a lock beside the model (STRATA_METAL_IGNORE_REPACK=1 overrides).
- metal_repack.py: unbuffered write(), F_NOCACHE on the shards and the output, fsync every GiB, a pid lock (file cache stays flat; the old tool filled all 128 GB).
- setup_mac.py: no engine while preparing a model (stops setup), start() waits on a live repack lock, resident rule = the engine's resident_cap.
- tests: arena_budget_test.cpp (47 checks), tools/test_setup_mac.py (repack guard).
2026-10-13 12:47:21 +02:00
Niko1221 d54e73fad5 setup: image generation (Qwen-Image) hidden from --help (options still work); README: Apple Silicon Macs line 2026-10-13 12:31:31 +02:00
Niko1221 226e061756 Mac: link the merged tree (commit batch off on Metal, stubs for the 0.1.44 entry points, version 0.1.44-metal) 2026-10-13 12:07:31 +02:00
Niko1221 0e7a4694e5 mtp: the draft graph node listing (STRATA_VERIFY_NODES) is CUDA/HIP only; the Metal build had no cudaGraphNodeGetType (merge fallout of f-latency into the Mac branch) 2026-10-13 12:03:42 +02:00
Niko1221 6a74320caa Merge f-mac-rel (Mac Metal support) into f-144-rel 2026-10-13 11:39:25 +02:00
Niko1221 081daa3078 docs/MAC.md: final M5 numbers for UD-Q5/Q6, prompt chunks, the M4 with a small arena 2026-10-13 11:30:05 +02:00
Niko1221 d1f4f44845 metal: a streaming Mac with 96 GB or more reads prompts in chunks of 4096 (UD-Q5 on the M5 Max: 4K and 32K prompts 700 -> 1,100 tok/s warm; the decode is unchanged) 2026-10-13 11:19:48 +02:00
Niko1221 54b61dbb11 pool: STRATA_POOL_UMWAIT only on MSVC proper; the Windows HIP build (clang with the MSVC ABI) has no WAITPKG intrinsics (#2174 follow-up) 2026-10-13 10:59:31 +02:00
Niko1221 107f9b62b3 setup: Unsloth defaults (owner): STRATA_MMQ_KQ=1 for UD-Q5/Q6, STRATA_HC_Q8=1 on CUDA for the UD-* files, STRATA_ROUTE_TAIL_SKIP=0 where every expert is in RAM; golden configs re-recorded, tests 2026-10-13 10:58:40 +02:00
Niko1221 edb0870fe3 docs/MAC.md: Unsloth UD-Q4/Q5/Q6 on the M5 Max, the recommended sizes, the arena's wired cap 2026-10-13 10:50:55 +02:00
Niko1221 43d14715bd metal stream: the arena plus the engine's own buffers stay under 67% of the RAM (wired through the residency set; UD-Q6 on the 128 GB M5 Max with a 90 GB arena made macOS swap and then freeze, 84 GB ran clean) 2026-10-13 10:33:44 +02:00
Niko1221 f5a6fb47fd metal stream: a small arena gives the prompt's buffers their own allocation instead of a region of its slots (a class left under 512 slots stopped the engine with 'no free slot' at the first prompt; seen on the 32 GB M4 with 14 GB available at start) 2026-10-13 10:25:04 +02:00
Niko1221 82fa92ff33 isa guard: the NVFP4 AVX-512 row functions are wide on purpose (both callers test cpu_avx512_ok) 2026-10-13 10:21:06 +02:00
Niko1221 f3ee1819e6 setup: IQ3_S (Qwen, Swift) and UD-Q4_K_XL / UD-IQ4_XS (Unsloth) are the recommended sizes and the defaults where the RAM fits them; the largest that fits otherwise. Golden configs re-recorded 2026-10-13 10:06:10 +02:00
Niko1221 c4427238f9 setup on macOS: IQ3_S and UD-Q4_K_XL are the recommended sizes; IQ3_S stays the default where it fits, UD-Q4_K_XL is named as the higher-quality choice 2026-10-13 10:02:57 +02:00
Niko1221 201eb32ff5 setup on macOS: a big Mac that streams (UD-Q5/Q6 on 128 GB) keeps the MTP draft layer with --spec 3 and skips the experts' copy; UD-Q5/Q6 marked measured 2026-10-13 09:52:56 +02:00
Niko1221 cb88d3d2e3 isa guard knows nvfp4_avx512.cpp; check_architecture leaves other architectures to read_geometry's by-name refusal 2026-10-13 09:52:06 +02:00
Niko1221 289f8f03a7 0.1.44: version, engine minimums, strata_kq packaging, the model menu
- CMake project version 0.1.44; setup MIN_ENGINE (0, 1, 44); NVFP4_ENGINE (0, 1, 44) (UD-Q5/Q6 were already 0.1.44).
- setup: a prebuilt zip without strata_kq leaves no stale library next to the new engine (kept in .previous); the
  Dockerfile copies libstrata_kq.so into engine/.
- Model menu: families in the order Qwen (GSQ-RCO), Swift, Coder, Unsloth, NVFP4, Qwen3.6; NVFP4 is not listed on AMD
  and Intel (the Unsloth line names UD-IQ4_XS there); on NVIDIA its line says native FP4 (RTX 50) or that UD-IQ4_XS is
  faster on this card, with the download and RAM need. The Unsloth family lists its five sizes, images on.
- strata_mcp's built-in tables know the NVFP4 sizes and family. tools/test_setup_menu.py.
2026-10-13 09:34:48 +02:00
Andy Slater 65cf54355f sycl: prompt GDN recurrence with one barrier per token on Alchemist (bit-identical; A770 prompt 8K +3-5%, 32K +7%; STRATA_GDN_XE=0 for the column kernels)
Variant 6 of gdn_recurrence_variant (sycl/include/strata/prefill/gdn_xe.hpp): the column kernel's four row groups
and FMA chains in the same order, a 16-lane sub-group holding four columns x four row groups, so the cross-row-group
sums are sub-group shuffles and a token needs one work-group barrier instead of five (256 GRFs a thread).

Arc A770 16 GB (xe, oneAPI 2026.1 JIT), same day, a0116b27 + #2190 + #2193 with and without this commit:
gdn_parity recurrence 51.0 -> 24.5 ms at 8,192 tokens, 204-236 -> 110-113 ms at 32,768 (48 heads), 0 mismatches
(state, FP32, FP16). Coder IQ1_M, 128K context, int8 KV, --resident-experts: prompt 386 -> 396 / 407 tok/s at 8K,
478 / 480 -> 511 / 518 tok/s at 32K (2 runs each), the same output ids in every run.

On by default on Alchemist only (strata::intel_gpu_gen); every other card keeps the column kernels, unchanged.
STRATA_GDN_XE=0 turns it off, =1 turns it on for any Intel GPU with 16-lane sub-groups. gdn_parity compares
variant 6 with variant 0 bit for bit (state, FP32 and FP16 outputs) on any Intel GPU and skips elsewhere;
ctest -R prefill_gdn_parity runs only that part.
2026-10-13 09:28:00 +02:00
Andy Slater d4b48b3380 sycl/A770: fused GR read with more, smaller work-groups on Arc A-series
On the A770 the multi-token GR read's kernels launch too few work-groups to fill the card. With
STRATA_GR_MORE_GROUPS on:
- sliced down kernel: one work-group per (slice, row block), 80 x 6 = 480 groups instead of 80;
  every row's partial sum is computed exactly as before, only by a different group.
- gr_down_reduce: 8 slice loads in flight per step, then added one at a time, so x accumulates
  slices 0, 1, ... 79 in the same sequence of float adds as the old loop (-fp-model=precise).
- up kernel: 8 columns per group, 320 groups instead of 160 (the kernel is now templated on the
  column count; UPM_COLS stays 16 for every other user), and only group 0 writes the per-token lo
  values, which every group computes identically.

Default on Alchemist only (strata::intel_gpu_gen); Battlemage and other devices launch the same
kernels as before. STRATA_GR_MORE_GROUPS=1 forces it on, =0 off; gr_bench runs either way.

Measured on an Arc A770 16 GB (xe, oneAPI 2026.1 JIT), same day, a0116b27 + #2190 + the previous
commit (GR_PICK, large GRF), with and without this one: gr_bench 1-6 tokens 118.9-183.5 -> 94.6-148.4 us,
checksums identical for every token count; gr_multi_parity 32 cases OK with STRATA_GR_MORE_GROUPS=0 and =1.
Decode, Coder IQ1_M, 128K context, int8 KV, --resident-experts, STRATA_VERIFY_STEPPED=1, 8K prompt, 256
tokens: 22.12 / 22.35 / 22.55 / 22.65 -> 23.24 / 23.61 / 23.62 tok/s (+3-5%), output ids identical.
Not built or run on Battlemage.
2026-10-13 09:27:55 +02:00
Andy Slater e682bded4b sycl: fused GR read without the private GrMulti copy (GR_PICK); large GRF on Arc A-series only
- GR_PICK: a runtime index into the by-value kernel argument (m.a[k]) made IGC copy the whole
  GrMulti (~1.2 KB) into every work-item's private scratch (unitrace on an A770: ~40 KB private
  per SIMD32 thread in gr_up_multi / gr_down_reduce / gr_norm_split). GR_PICK picks one field with
  compile-time indices only (a chain of selects over the uniform kernel argument). Addressing
  only: the same values, read and summed in the same order, so the output is bit-identical. This
  part is on for every device; it is not measured on Battlemage.
- The sliced down kernel and the multi-token up kernel in large-GRF mode (256 registers/thread);
  at 128 the up kernel still spills ~3.4 KB/thread on the A770 (unitrace). Large GRF halves the
  threads resident per Xe core, so it is the default on Alchemist only (strata::intel_gpu_gen);
  Battlemage and other devices launch the same kernels as before. STRATA_GR_LARGE_GRF=1 forces it
  on, =0 off.

Measured on an Arc A770 16 GB (xe, Linux 7.0, oneAPI 2026.1 JIT), a0116b27 + #2190, same day:
gr_bench 1-6 tokens 205-258 us -> 119-184 us (GR_PICK alone, STRATA_GR_LARGE_GRF=0: 186-239 us),
checksums identical for every token count. Coder IQ1_M, 128K context, int8 KV, --resident-experts,
STRATA_VERIFY_STEPPED=1, 8K prompt, 256 tokens: decode 5.37 tok/s (5.88 / 5.93 with
STRATA_DOORBELL_CHECK=1 / STRATA_LEND_MIRROR=0) -> 22.12 / 22.55 / 15.60 (cold-cache run)
tok/s, 20.61 / 20.93 with STRATA_GR_LARGE_GRF=0; the window's pool phase 183-208 -> 34-35 ms
per round; output ids identical where the expert placement matched. Not built or run on
Battlemage or i915.
2026-10-13 09:27:37 +02:00
Andy Slater ddcc5f56dc sycl: size the expert cache from the DRM fdinfo when xe reports 0 free card memory (#1545)
On an Arc A770 16 GB on the xe driver, Level Zero has no ext_intel_free_memory and get_memory_info reports 0 free
with the right total. device_free_bytes() kept the 0 (the #1440 fdinfo path only replaces a figure that is too
high), so --expert-cache auto sized the caches from 0 bytes. A 0 with a known total is now read as unknown: on
Linux the card's total less this process's DRM fdinfo figure is used, the same source the A750/i915 path uses.
Anything that reports a non-zero free figure is unchanged; off Linux the 0 stands.
2026-10-13 09:26:34 +02:00
Niko1221 c106e9629a Merge f-qimage2 into f-144-rel 2026-10-13 09:23:47 +02:00
Niko1221 de4fe43b0c Merge f-qimage into f-144-rel 2026-10-13 09:23:46 +02:00
Niko1221 4cfeaa0f4f Merge f-qwen36 into f-144-rel (geometry: #2179's read_geometry guard kept for qwen4exp, Qwen3.6's family guard on top) 2026-10-13 09:23:46 +02:00
Niko1221 66db34dc40 Merge f-nvfp4 into f-144-rel 2026-10-13 09:21:19 +02:00
Niko1221 0aef10aeae Merge commit 'cf1b0edf' into f-mac-rel 2026-10-13 09:21:10 +02:00
Niko1221 075aaacdff Merge branch 'f-mac-iq3s' into f-mac-rel 2026-10-13 09:21:06 +02:00
Niko1221 322df64128 Merge f-quants into f-144-rel 2026-10-13 09:17:53 +02:00
Niko1221 9adf4e01a7 Merge f-intel-quants into f-144-rel 2026-10-13 09:17:44 +02:00
Niko1221 0b8d31013a Merge f-latency into f-144-rel 2026-10-13 09:17:44 +02:00
Niko1221 800a0904fb strata-q36: every main layer's experts in VRAM when only the MTP block's do not fit
When the main layers' experts fit but the MTP block's do not (UD-Q6_K_XL on
32 GB), the main layers are placed as in the all-in-VRAM mode (same text,
the decode graph kept) and only the MTP block's experts go to slots + the
CPU pool (its drafts). To get there when the fit is close: the packed Q8_0
decode copies are given back, the prompt chunk stays 512, and - said in the
log - the attention scores buffer shrinks to 64 MB and the VRAM reserve to
256 MiB. STRATA_Q36_NO_MTP_CPU=1 keeps the plain hybrid mode.
UD-Q6_K_XL on gfx1201: tg128 73-75 -> 80.6 (llama.cpp HIP 81.2).
2026-10-13 07:53:24 +02:00
Niko1221 9341ebd213 strata-q36 hybrid: the whole-layer prompt copy when fewer than half the experts are resident
Many small copies were slower than the one whole-layer copy on a 12 GB card
(RTX 3060, UD-Q4_K_XL at 37 of 256 resident: pp2048 1540 in runs, 1974
whole). gfx1201 UD-Q6_K_XL (205/256) and Q8_0 (176/256) keep the per-expert
copies: pp4096 3504-3588 and 3015-3079.
2026-10-13 07:39:25 +02:00
Niko1221 cf1b0edf91 Conversation cache: a picture is matched by its bytes' hash (from the server) and its grid, not by its embedding floats (#2228)
The server already keys its encoder cache by the normalized picture bytes (sha256, the .sve file
name); it now sends those keys with the request (img_keys=<hex>,..., in prompt order), and the
engine's conversation cache uses key + grid for a picture when the count matches. Without the key
(an older server) it hashes the embeddings as before. Old engines skip the unknown key.

Measured for #2228 (one 1600x1000 screenshot-like picture, strata-vision, fresh processes and
twice in one process): the CPU encoder gives the same bits on a TR PRO 3975WX at 8/15/16/32 threads
and on a Xeon E5-2643 v3 (e2357d8e...), the GPU encoder on an RTX A4000 is reproducible too
(92cf7818...), but CPU and GPU differ. So a picture's embeddings change when the encoder moves
between CPU and GPU, or after an encoder update, and no longer on its own; with the byte key the
cache's matching does not depend on that at all. CUDA/HIP and SYCL. Test: serve/test_server.py
test_pictures_are_keyed_by_their_bytes_for_the_engine.
2026-10-13 07:33:16 +02:00
Niko1221 577abb0378 strata-q36 hybrid: a streamed prompt layer copies its resident experts from their VRAM slots
The prompt's staging buffer took the whole layer from host memory over PCIe
(~19-28 GB per chunk); now the experts with a VRAM slot are copied device to
device and only the others come over PCIe, in runs of consecutive experts.
Same bytes in the staging buffer (greedy ids identical). UD-Q6_K_XL on
gfx1201 (205 of 256 experts per layer resident): pp4096 1935 -> 3511-3601.
Also: the packed Q8_0 decode copies are given back when that lets every
expert fit (STRATA_Q36_KEEP_PACKED=1 keeps them); STRATA_Q36_STREAM_ALL=1
keeps the whole-layer copy.
2026-10-13 07:30:12 +02:00
Niko1221 ee6d8baccf serve: say when a reusable state is rejected only because a picture's embeddings changed (#2228)
A checkpoint (or the live state) is reused only when the prompt starts with its tokens AND its
pictures, and a picture is its grid plus the hash of its embedding floats. In #2228 (screenshot-heavy
agent, CPU image encoder) the cache fell back to checkpoints a long way back although the client's
messages were byte-identical; one explanation is a picture encoded again with other bits. When the
deepest state whose tokens match is rejected for a picture, the engine now logs which picture and
where, once per request, so the next log tells this apart from a rewritten history. Log only; what
is reused does not change.
2026-10-13 07:20:05 +02:00
Niko1221 a0e577de95 test: a user asking for </tool_response> (or the other tool tags) verbatim gets the text, the outer-pair exception included (#2026)
47Hunter47 measured </tool_response> asked verbatim at 0/15 on 0.1.43.1 (it reached the model as
the real closer). 62185aab already marks it; this pins the cases next to the exception that keeps
a wholly wrapped tool result's outer pair: the bare closer, a request that quotes a whole pair, a
message that only starts with the opener, two closers in one sentence.
2026-10-13 07:16:25 +02:00
Niko1221 1267f093ed docs/MAC.md: the M4 numbers with 7 GB kept and the stepped arena fill 2026-10-13 06:32:51 +02:00
Niko1221 612633c3b0 metal: the streaming arena fills in 1.5 GB steps and stops at macOS's floor; available memory is free + purgeable + the clean file cache (not other programs' inactive memory); the least arena layout goes ahead instead of failing the re-check 2026-10-13 06:32:33 +02:00
Niko1221 78c5cbbc24 q36_llama_dump: STRATA_DUMP_NGL, the GPU layers for a file that does not fit the card 2026-10-13 06:23:47 +02:00
Niko1221 67bedde10e metal: the streaming arena keeps max(7 GB, 15% of the RAM) again - a server soak with 5 GB kept swapped out 93 MB while the arena filled 2026-10-13 06:22:51 +02:00
Niko1221 f9641ff85e strata-q36: Q8_0 / F16 small tensors widened at load (the Q8_0 file's ssm_alpha / ssm_beta)
Qwen3.6-35B-A3B-Q8_0.gguf stores the GDN alpha / beta projections as Q8_0;
the engine read only F32 / BF16 there and refused the file. They are now
widened to F32 at load (fp16 scale x int8, as ggml). q36_llama_dump:
STRATA_DUMP_CTX sets a smaller context / batch for files whose weights leave
no VRAM for 4096-token buffers.
2026-10-13 06:22:45 +02:00
Niko1221 77c65b797b docs/MAC.md: 32K prompts with the experts' copy and the 34-minute server soak on the M4 2026-10-13 06:21:11 +02:00
Niko1221 bb1ed8d4c0 setup: Qwen3.6 sizes recommended by RAM (UD-Q4_K_XL 32, UD-Q6_K_XL 44, Q8_0 52 GB)
The engine keeps every expert in RAM when they do not all fit in VRAM, so a
size's RAM need is its file plus room for the rest. The largest size this
PC's RAM holds is the default; any size can still be chosen (a bigger one is
asked about, as before, never refused).
2026-10-13 06:08:19 +02:00
Niko1221 5a30c2c4e8 Images (I2): NVIDIA mma.sync flash attention for the native runner, fused q/k/v GEMM
- k_attn_mma (sm_80+): m16n8k16 f16 -> f32, 8 warps x 16 queries, 64-key K/V tiles double-buffered with
  cp.async, ldmatrix (.trans for V), fp32 online softmax.  Bench on the RTX 3060 at 4096 x 4150 keys x 32
  heads: 23.7 TFLOPS vs ggml's flash_attn_ext_f16 15.4 (0.376 vs 0.577 s/step); CPU check 7e-5.
  STRATA_QI_GGML_ATTN=1 keeps ggml's.
- q, k and v of a block with one weight type load into one [3C x C] tensor: one GEMM and one activation
  quantize instead of three (STRATA_QI_NO_QKV_FUSE=1 keeps three).
- A float4 LayerNorm path with tanh(gate) once per step, off by default (STRATA_QI_FAST_LN=1): no measured gain.

RTX 3060, Q8_0 1024, seed 42, 3 interleaved rounds: steady step 2.09 s vs stock 2.61 (1.25x), denoise
17.93 s vs 22.13 (1.23x).
2026-10-13 05:53:21 +02:00
Niko1221 92ef26cd57 docs/QUANTS.md: STRATA_PROMPT_DIRECT is mixed on a PC with more file-cache room 2026-10-13 05:50:14 +02:00
Niko1221 4b703a6777 docs/MAC.md: the 4K prompt with the experts' copy on the M4 (72.9 vs 62.5 tok/s) 2026-10-13 05:39:58 +02:00
Niko1221 3d5819df3b serve: start the engine again (2, 4, 8 s apart) when it stopped inside the GPU runtime's set-up
A start right after another engine exited sometimes fails before the engine writes a line: ROCm aborts with
"terminate called after throwing an instance of 'rocr::AMD::hsa_exception'" (or the process exits with an empty
log) while the previous process's GPU queues are still being released (r9700: 1 of ~10 rapid restarts in the A/Bs,
2 of ~40 for the quants runs). Such a start - within 30 s, no engine line other than the runtime's failure, not the
engine's own error exit 1 - is retried with a message saying why; STRATA_START_RETRIES (default 3, 0 = off).
2026-10-13 05:36:26 +02:00
Niko1221 1b210929f6 docs/MAC.md: the experts' streaming copy (M4 numbers), the arena rule, vision and Unsloth UD-Q4 on Mac, the streaming switches 2026-10-13 05:32:28 +02:00
Niko1221 703e63c1e1 docs/NVFP4.md: measured in Strata - the quant yardstick against Q8_0 (beside the Unsloth and GSQ-RCO quants), the tail skip, the decode and prompt expert kernels on the RTX 5070 / 3060 / P100, the RAM-budget mode, images 2026-10-13 05:27:38 +02:00
Niko1221 9269262f13 docs/QUANTS.md: UD-IQ4_XS and the Q6_K MMQ numbers, PROMPT_DIRECT and the budget-mode I/O A/Bs, several cards; MODELS.md UD-IQ4_XS quality 2026-10-13 05:26:35 +02:00
David d8684360b8 CUDA: service CPU doorbells while instrumented graph launches block 2026-10-13 05:25:46 +02:00
Niko1221 9d41e18bd9 layer split auto (CUDA): a stage's link scales its miss cost only against the fastest stage's link and from a 4x gap on (#2007)
0.1.41 (0391e478) scaled each stage's miss term by 20 GB/s / its probed link, for 2x 4090 D
with CUDA1 on a physical x1 link (1.6 GB/s: x12, K 22 -> 38, prompt ~350 -> ~1400 tok/s).
On 2x Tesla T10 the same rule moved auto from K=23 to K=14: CUDA1 runs 34 of 48 layers, and as
the prompt path is a pipeline the slower stage sets the rate - 256K prompt 1035 -> 795 tok/s,
decode 46.8 -> 39.5; --layer-split 23 on 0.1.43 restores both (xiake-1 in #2007).

split_rules::link_miss_scale: the scale is the fastest stage's link over this stage's, applied
only from a 4x gap (a riser or chipset x1/x4 slot beside an x16 one), capped at 64. A symmetric
rig keeps the fitted formula whatever its link speed; the x1 case still gets ~x15.
STRATA_SPLIT_LINK_ABS=1 restores the 0.1.41 rule. SYCL has no link term (unchanged).
Test: split_rules_test (the x1 case, equal gen3 cards, a 2x gap, caps, unprobed stages).
Not run on the reporter's T10s: the reply asks for the 'link ... -> miss cost' lines.
2026-10-13 05:25:31 +02:00
Niko1221 02fadf4447 metal setup: a streaming Mac gets the experts' streaming copy when the disk has room for it and 10 GB more; otherwise a warning and the GGUF 2026-10-13 05:23:14 +02:00
Niko1221 c34ac7093f metal: streaming reads each expert with one request from <gguf>.experts (stream_experts.py: gate, up, down together) - M4 IQ3_XXS after 6K: 8.62 vs 7.37 tok/s, prompts 84 vs 65 tok/s, same tokens 2026-10-13 05:23:14 +02:00
Maxim Mazurok 640ac438be --prefill-head-alias (opt-in, --serve, one GPU): the output head's VRAM holds the prompt buffers while a prompt is read
The native output head is needed only to generate, the batched prompt path's buffers only to read a prompt. With the flag the head is unloaded while a prompt is read (its VRAM counted into the auto cache and the prompt path's hybrid borrowed/owned layout) and read back in before the first decode window. On an 8 GB RTX PRO 2000 Laptop (Windows, IQ3_XXS, 590 MiB reserve) the prompt chunk went from 512 to 2,048 tokens and prefill from 292.5 to 667.1 tok/s on a 29,854-token prompt; reads of up to 512 fresh tokens cost about 0.6-0.75 s more. Off by default: every new path is behind the flag. docs/PREFILL_HEAD_ALIAS.md has the measurements and limits.
2026-10-13 05:00:30 +02:00
Niko1221 903cab02b0 Images (I2): warp-per-head q/k RMSNorm+RoPE kernel (float4, no shared memory; STRATA_QI_OLD_QKV=1 keeps the old one); WMMA attention fragment layer for gfx11 and gfx12 (gfx12 off unless STRATA_QI_GFX12_ATTN=1, not run yet) 2026-10-13 04:56:52 +02:00
Niko1221 8dc89b0f4a metal: the streaming arena keeps max(5 GB, 15% of the RAM) and reserves 0.5 GB for the n-gram table read row by row
The table is read with F_NOCACHE through a bounded row cache (1M rows), so 2 GB was too much to set aside. M4 32 GB,
IQ3_XXS decode after the 6K document, 5 interleaved runs, auto arena (memory_pressure every 3 s, vm_stat swapouts):
keep 7 + 2 GB (54c0de70) 5.93 tok/s median, arena ~8.5 GB; keep 7 + 0.5 6.59 (+11%), ~10 GB; keep 5 + 0.5 7.47
(+26%), ~12 GB; the least free memory 43-57%, no swap-out in any run. The M5 Max still keeps 15% (20.6 GB).
arena_budget_test: 14 checks pass.
2026-10-13 04:56:13 +02:00
Niko1221 728097fb6f HIP: STRATA_COPY_HIP_PRIO=1 (opt-in) creates the verifier's copy stream at high priority
ROCm gives out hardware queues per priority. Sharing one with the window's stream, --pcie-mode dma's expert copies
and the host function that raises flag B queue behind the window graph that spins on that flag: on an R9700 every
dma-mode request timed out at layer 1. With its own queue dma mode runs (copies overlap the VRAM-hit compute).
2026-10-13 04:49:15 +02:00
Vladimir 8488c06bba serve: the elastic encoder's VRAM figure after the driver's counter caught up
The figure in "[strata] images: the image encoder started in N s, M MiB of VRAM" read the free VRAM right after the
cache's shrink, before the driver's counter showed it (amdgpu sysfs), so it came out low (286-362 MiB against the 1,339
measured otherwise). A 0.2 s wait before that reading, only on a start that shrank the cache.
2026-10-13 04:36:30 +02:00
Vladimir ca41a6b957 serve: the image encoder only while there are pictures ("elastic": true in the vision section)
Opt-in, with the config's "vram_elastic": the GPU encoder does not start with the server or a load. A picture's turn
makes the expert cache keep "elastic_mib" (default 2048) free on top of the reserve and starts the encoder; after
"elastic_idle_s" (default 120) without a picture a thread stops it between requests and the cache grows back to the
reserve (0 when no POST /v1/vram was made: the start's reserve left it a segment or more smaller). A POST /v1/vram
while the encoder runs stacks on its share; an unload stops it. Without "vram_elastic" the server says so and starts
the encoder as before. serve/test_vision_elastic.py pins each case; docs in VRAM_ELASTIC.md and DETAILS.md.
2026-10-13 04:36:30 +02:00
Coby Tamayo e8b4551173 rm git dep 2026-10-13 04:36:30 +02:00
Coby Tamayo a55fd1f24e rm confusing comment 2026-10-13 04:36:29 +02:00
Coby Tamayo 68140bbdee document Nix install 2026-10-13 04:36:29 +02:00
Coby Tamayo 6a99330817 rm claude-code 2026-10-13 04:36:29 +02:00
Coby Tamayo 01249588f4 Nix environment 2026-10-13 04:36:28 +02:00
bulhenvaster c046273318 setup --update: an MTP folder shared by several configs keeps the draft subset of the most recently used one (the oldest config's default subset replaced cyrillic: Russian decode 68 -> 54 tok/s on an RTX 5060 Ti + RTX 3060, IQ3_XXS)
update_install refreshed the draft subset once per installed config, newest first, so a folder that two configs
name ended with the subset of the config used longest ago. A config without "draft_vocab" (cjk) then put the
default subset over the cyrillic one of the config in use. Now the first config that names a folder decides;
separate folders and a start (which installs its own config's subset) are as before.

Checked: tools/test_setup_draft_vocab.py (3 new tests, 10 pass), test_setup_config and test_setup_configs (18 pass).
Measured on the affected install, two alternating runs each, greedy 600-token Russian story: 54.0 / 54.0 tok/s
with the default subset installed by the update, 68.7 / 68.6 with the cyrillic one (1.39 -> 1.90 tokens per window).
2026-10-13 04:36:28 +02:00
Niko1221anddvasdekis 2f335a5eab tools/windows/enable_large_pages.ps1: grant "Lock pages in memory" to the current account (#2185)
A helper for the large-page arena (#1412): run as Administrator, it adds the account to
SeLockMemoryPrivilege with secedit, says that a new logon is needed, and prints the secpol.msc
steps when it cannot run. Changed from the PR: secedit /configure is limited to
/areas USER_RIGHTS (the exported policy is not re-applied as a whole), and the description no
longer claims a speed-up we have not measured. docs/DETAILS.md points to it. Not run on our side
(no Windows box for it).

Co-authored-by: dvasdekis <[email protected]>
2026-10-13 04:36:04 +02:00
Niko1221anddvasdekis ea73bc2522 Windows HIP: hipKernelNameRefByPtr is resolved at run time (STRATA_VERIFY_NODES), so a runtime DLL without it still starts (#2183)
Some amdhip64_7.dll builds (AMD TheRock wheels / ROCm 10.2 / 7.2.4 nightly) do not export
hipKernelNameRefByPtr although their import library has it, so strata.exe failed to load with
STATUS_ENTRYPOINT_NOT_FOUND before main. It is only used to name graph nodes under
STRATA_VERIFY_NODES; on Windows it is now looked up with GetProcAddress and a missing one
leaves the node unnamed.

From #2183, its verify.cpp part only. The PR's other part (Windows HIP --pcie-mode auto as
DMA) changes the default path of every Windows AMD user and was not measured on our side; it
is left out.

Co-authored-by: dvasdekis <[email protected]>
2026-10-13 04:35:37 +02:00
Niko1221 5e54a301ef SYCL #2194: drain the other queues before every decode window on every Alchemist card (A770 on xe too), not only on i915
An Arc A770 under xe intermittently never started decoding: the main thread waits in Verifier::run's stepped serve
(queue::wait -> urQueueFinish) for a segment that never drains, xe logs nothing (#2194). The A750's same failure was
fixed by draining the other queues before each window (STRATA_WINDOW_SYNC, default 1 on i915 only). Reproduced on
the A750 by turning that off: STRATA_WINDOW_SYNC=0, STRATA_VERIFY_STEPPED=1, --resident-experts, IQ3_XXS, 4K prompt:
1 of 6 runs hung with #2194's stack; with the default drain 0 of 11 (5 unpinned, 6 with the pipelined helper pinned).
The default is now 1 on i915 or on an Alchemist device (intel_gpu_gen), 0 elsewhere; STRATA_WINDOW_SYNC overrides.
Unchanged on the A750 (i915: already 1) and on Battlemage (0). Not run on an A770 under xe (no such card here).
2026-10-13 04:33:15 +02:00
Niko1221 987548ce2c gfx12 chunk rule: with --resident-experts the RAM copy to come (every uncached expert plus the loan) counts, and the margin is 6 GiB (#2090)
The first version compared only the extra loan with the RAM available at that point, but the
resident RAM copy is pinned after it and holds the lent slots' experts too: in a 30 GB scope on
an R9700 (IQ3_XXS, --resident-experts) 30 GiB were available before the 23.35 GiB copy, so the
rule never fired. The reporter's PC thrashed with 3.4 GB left at 16384 and ran with ~6.4 GB at
8192, hence 6 GiB.
2026-10-13 04:28:32 +02:00
Niko1221 0bfa422899 setup: images for the NVFP4 packs - the original's image encoder (ISTA's mmproj, pinned by size and SHA-256), the dense GGUF as its vocabulary
Both NVFP4 packs keep Qwen's vision tower unquantized, so the image encoder every Flash-Next model uses reads pictures
for them; strata-vision opens the text model vocab-only, which is the pack's dense GGUF here.  Setup asks about images as
it does for every model (default no; --vision gpu|cpu), the encoder is the file other installs already share
(mmproj-Qwen3.8-Flash-Next-BF16.gguf at ISTA's pinned revision, 907,543,008 B, sha256 b1a82259...).

Checked end to end through the server on an RTX 5070 (budget mode, GPU encoder, 1024 image tokens): a picture with
"CODE 4217", a red square on the left and a blue circle on the right - NVFP4 answered "4217" and described both shapes,
their colours and sides correctly, as IQ3_XXS did on the same picture.  VRAM: the expert cache held 393 slots with
images (--vram-reserve-mib 2100 and the encoder) against 982 without (1500).
2026-10-13 04:18:43 +02:00
Niko1221 7d7da88177 Opt-in STRATA_PROMPT_DIRECT=1 (Linux): with the file tier read through the file cache, the prompt path's batched expert reads go unbuffered (merged O_DIRECT requests); the same bytes 2026-10-13 04:14:29 +02:00
Niko1221 1a55fc3788 gfx12: the 16384 auto prompt chunk stays at 8192 when the RAM available does not cover its extra prompt loan (#2090)
The gfx12 default (0.1.43) lets a bare --prefill auto reach 16384, which lends the prompt path
about twice the cache slots of 8192 (R9700, Swift 1.5 IQ3_XXS: 3,907 slots / 6.38 GiB against
2,068 / 3.37 GiB); the lent slots' experts are read from the RAM side during the read. On a
30 GB PC with --resident-experts 38-47K prompts went 20 GB into swap with 1.5M major faults
and the engine was killed; --prefill 8192 ran them at ~95 tok/s (bdwilton in #2090).

At start, when the engine's own 16384 ceiling is in effect, the extra slots of a 16384 loan
over an 8192 one are priced at the largest blob; if the RAM available (the cgroup or
--memory-limit-mib cap included) is below that plus 4 GiB, the ceiling stays at 8192 and a
line says why. STRATA_GFX12_CHUNK=1 keeps 16384; an explicit --prefill N / auto:N is never
touched (the 16384 default only applies to a bare auto).
2026-10-13 04:14:11 +02:00
Niko1221 b5217c8bb7 serve: the VRAM-free check at start covers a layer split's later cards too (#2192)
The 'MiB of VRAM free with everything loaded' line and its LOW warning read CUDA0 alone. In
#2192 (2x RTX 2080 Ti 22 GB, Windows, IQ3_S, --batch 4, --layer-split auto, 0.1.43) CUDA0 had
740 MiB free while CUDA1 began its window captures at 153 MiB and fell to 52; the first batch
window then never finished. Both stall dumps show the host thread inside cudaGraphLaunch called
from Verifier::run_slot_rows (the stage still 'idle': it had not reached its per-layer wait), the
pool asleep and the watchdog firing after 60 s - a page-in under WDDM while the other card's
graph spins on a host flag.

Each later card now gets its own line, and below 512 MiB a LOW warning with the
--vram-reserve-later-mib value that leaves ~768 MiB. Said, never changed.
2026-10-13 04:12:24 +02:00
Niko1221 fb654ba3e8 Windows AMD: the card's name and VRAM from the registry's display class, in the server's hardware_static and in strata_bench (#1587)
On Windows with only an AMD card neither NVML nor the amdgpu sysfs files answer, so
hardware_static.gpu_name was None and strata_bench (no nvidia-smi, no rocm-smi on Windows)
labelled the run "CPU only" although the engine ran on the card (ByteJuggler in #1587: RX 6900 XT).
telemetry.windows_display_adapters reads DriverDesc, ProviderName, DriverVersion and the memory
size from HKLM's display class (no subprocess, no vendor library); windows_amd_cards drops a
Ryzen's integrated Radeon (512 MiB here) when a real card sits beside it. hardware_static gets
the name and gpu_vram_total_mib; strata_bench uses the same list when nvidia-smi and rocm-smi
give nothing. Load, VRAM use, temperature and power stay unread on Windows AMD (that is the
reader of #1224 / #1434 / #1871). Checked against this PC's registry (RTX 5070 + the Ryzen 7600's
iGPU: both listed, the iGPU's 512 MiB read); tools/test_windows_amd_gpu.py with a fake registry.
2026-10-13 04:12:24 +02:00
Niko1221 23db5f9555 setup_intel: a layer split whose experts will likely not fit writes STRATA_VERIFY_NO_HOST=0 and says so (#1980)
The stock split shape (STRATA_STAGE_TRIM=1, 1024 MiB reserve, NO_HOST=1 from strata-sycl.sh) is
refused at the start on a 2x Arc Pro B60 24 GB pair with IQ3_XXS: the second stage is 2230-3731
experts short and the host mirror covers the first stage only. The same config with NO_HOST=0
runs at 39-50 tok/s with 0 faults (jds1-ai in #1980). When the model's experts take more than
80% of the split cards' VRAM, setup now writes NO_HOST=0 into the config (a value the user set
is kept) and prints why and how to turn it back. IQ2_XS on the same pair (74%) keeps NO_HOST=1.
Test: tools/test_setup_sycl.py test_split_short_rule.
2026-10-13 04:12:24 +02:00
Niko1221 1955774783 SYCL layer split: a REFUSED start exits cleanly, and a later stage's misses no longer read the first stage's mirror table (#1980)
- refuse_exit: after the REFUSED message the engine waits for the queued device work, flushes
  and _Exit(2)s instead of returning from main. Returning ran the stages' destructors in an
  order the UR loader did not survive: on a 2x Arc Pro B60 pair every REFUSED start segfaulted
  in libur_loader, and a trimmed one also left an xe page fault (bcs CAT error) on the second
  card that hung the next start (jds1-ai in #1980).
- resident_plan_set_mirror takes the table's length; resident_plan uses the mirror and the
  usage counts only for a residency slice inside the table they were set for. A later stage
  passes its own residency table, so the slice was taken as an offset across allocations and
  read (and counted into) another buffer: the stage-1 xe faults with STRATA_VERIFY_NO_HOST=0
  and no trim (10 of 10 requests, 0 tokens). Such a miss is now a plain miss, served by the
  host. Patch shape as jds1-ai proposed it.

Compile-checked only (no two-card Intel box here).
2026-10-13 04:12:24 +02:00
Niko1221 cb22a53e0e Adaptive overlap: the host-function signal is the Windows default, and the copy stream is flushed after the swaps are queued (#2188)
Under WDDM a stream's work sits in the driver's command buffer until a driver call flushes it.
With the stream-ordered signal the verify window only spins on a pinned int, so the last
layers' copies and their 4-byte signal copies were never submitted: the first request stopped
with "the adaptive tier's copies for layer 45 never landed" (RTX 4070 SUPER, 0.1.43.1, opt-in
STRATA_ADAPT_OVERLAP=1). STRATA_ADAPT_SIGNAL=host worked.

- Windows: the host function is the default signal; STRATA_ADAPT_SIGNAL=copy asks for the copy.
- Every platform: one cudaStreamQuery on the copy stream after the swaps and their signals are
  queued (a no-op for the results; it submits the batch under WDDM). Linux keeps the copy
  signal as before.

Needs a Windows run with STRATA_ADAPT_OVERLAP=1 STRATA_ADAPT_SIGNAL=copy to confirm the flush.
2026-10-13 04:12:24 +02:00
Niko1221 c18d4c2bd5 docs/INTEL_ARC: the auto split's headroom pick on equal cards (#1440) 2026-10-13 04:12:24 +02:00
Niko1221 8af75e1a39 dequant_wide_on: (void) on the cleared CUDA errors (HIP marks hipGetLastError nodiscard: two new warnings) (#2178 port) 2026-10-13 04:12:24 +02:00
Techannel 5025324c2b docs + bench: the Tesla P40 (sm_61) profile, the four changes and their measurements
bench/results/2026-10-12-p40-sm61-kernels: where a prompt and a decode window spend their time on a P40, the kernel
benches and parity runs, every request of the engine A/B (565-token prompt 205 -> 255 tok/s, decode 34-35 -> 37-38
tok/s; with STRATA_DENSE_F32=1 a 7K prompt 360-432 -> 464-474 tok/s), the byte comparison of the answers, and what
was not tested.  docs/OLDER_GPUS.md: the sm_61 section and switches, and two traps when measuring on such a card.
2026-10-13 04:12:24 +02:00
Techannel 2216161362 sm_61: expert mode 8 and its own interleaved-matvec tables by default (bitwise; STRATA_SM61_TABLE=0)
As sm_60 got with STRATA_SM60_TABLE, measured on a Tesla P40 (native dp4a):
- native_expert_bench, 18 groups at 2 tokens on the IQ3_S pack's layers, mode 0 -> mode 8, bitwise: IQ3_S 390 ->
  255 us, IQ3_XXS 345 -> 252, IQ2_S 290 -> 238, IQ4_XS 328 -> 261.
- kIlRows_61 / kIl1Rows_61 from three runs of mmvq_il_parity --bench --emit-table (identical but two Q3_K cells;
  a cell is non-zero only where every run gained 3%).
2026-10-13 04:12:24 +02:00
Techannel 40ba0cf93d prefill gemm: opt-in STRATA_DENSE_F32=1 on Pascal - the large dense GEMMs as cublasSgemm on FP32 copies
For FP16 operands with FP32 sums cuBLAS 12.4 has two Maxwell kernels on a Tesla P40 and picks the slow one for most
chunk sizes: qkv (10240 x 2560) runs at 7.6 TFLOPS for T 7030-7040 and at 4.3 from T 7041 (49 -> 85 ms), so a
7K-token prompt reads in 16.3 or 19.5 s by its token count.  cublasSgemm on FP32 copies runs at 8.4-9.8 TFLOPS from
T 512.  For the five shapes and the sizes the route takes, the result is bitwise the FP16 call's: gemm_f32_parity,
5,201 chunk sizes through Gemm::f16.  That is a property of the cuBLAS build (12.4.5.8 measured), hence opt-in.
2026-10-13 04:12:24 +02:00
Techannel 9c5bbc36d0 dequant: a thread per 8 values for Q4_K / Q5_K / Q6_K / IQ4_XS, sm_61's default (bitwise; Q6_K 7.2 -> 0.33 ms on a Tesla P40)
dequant_kernel's thread reads its 32-value group a byte at a time and writes 32 two-byte values one by one, 64 bytes
from its neighbour's.  On a Tesla P40 that is 10 GB/s for Q6_K, and the IQ3_S pack's dense weights cost every prompt
chunk ~0.5 s of it, whatever the chunk's length (a fifth of a 560-token prompt).  The wide kernel takes 8 values a
thread: 16-bit loads, one 16-byte store, the same expression per value.

dequant_wide_parity: 360 cases bitwise (FP16 / BF16 / FP32 out, a padded stride, row slices, sane and raw random
blocks); --bench on the P40: Q6_K 10240 x 2560 7.25 -> 0.33 ms, Q4_K 5.2x, Q5_K 3.8x, IQ4_XS 3.6x.
On by default on compute capability 6.1 only (where it was measured); STRATA_DEQUANT_WIDE=0 / =1 for any card.
2026-10-13 04:12:24 +02:00
Niko1221 27f2d16725 strata-q36: the prompt's F32 projections on a tiled split-K SGEMM
The router, the GDN alpha / beta and the shared expert's gate (n_out 1-256,
K 2048) ran on hipBLAS/cuBLAS SGEMM with 8x8 / 16x16 tiles (~10% of a
prompt). k_sgemm_nt: (16 TM) x 64 tiles, TM x 4 outputs per thread, K split
so about 128 blocks run, the partials summed in a fixed order
(deterministic). gfx1201 pp4096: 3290 -> 3455 tok/s. STRATA_Q36_BLAS_F32=1
keeps the BLAS SGEMM.
2026-10-13 02:48:23 +02:00
Niko1221 b23869bf16 strata-q36: warm-up at load; hybrid slots independent of --mtp
Warm-up: a throw-away prompt chunk, a decode step and (with drafts) a
speculative step at load make the code objects, BLAS kernels and graphs, so
the first request runs at full speed (pp4096 rep 0: ~1770 -> 3273 tok/s; a
34-token first prompt: 1032 -> 48 ms). --no-warmup skips it.

Hybrid mode with MTP exact by default: the MTP block is loaded whenever the
file has one (so the VRAM layout and the slot count do not depend on --mtp;
the prompt pass through it only runs with drafts); decode and verify rows'
routing is counted only for the rows kept; the slots are rebalanced exactly
at every 32nd position and a verify window never steps over one.
2026-10-13 02:29:01 +02:00
Niko1221 54c0de70ca metal: the streaming arena's size rule - keep max(7 GB, 15% of the RAM), the n-gram table's memory out of it, re-checked just before the arena is committed; buffers past 8 GB when 4 of 8 GB are too few
arena_budget.h: the rule as plain arithmetic, tested by metal/tests/arena_budget_test.cpp (13 checks, run on the
M4). The M5 Max froze (2026-10-13 02:08) with an 89 GB arena planned from 'available - 7 GB' beside a 50 GB n-gram
table: macOS now keeps 15% of the RAM (20.6 GB there), the table's mapped bytes (--ple-io ram) or 2 GB (read row by
row) come out too, and just before the arena is committed the free memory is asked again - less than planned: the
arena is laid out again smaller instead of failing. STRATA_METAL_KEEP_FREE_GB / STRATA_METAL_ARENA_GB override.
stream.cpp: arena buffers grow past 8 GB when 4 of 8 GB cannot hold the arena (STRATA_METAL_ARENA_CHUNK_GB).

The coordinator suggested max(12 GB, 15%); on the 32 GB M4 (Q2_0 streamed, 200 tokens, memory_pressure every 3 s)
12 GB kept cut the arena 9.4 -> 4.4 GB and decode 7.8 -> 4.8 tok/s, while 7 GB kept left macOS 55% free the whole
run (12 GB: 70%); same tokens. Hence 7 GB below ~47 GB of RAM, 15% above. Not yet run on the M5 (down).
2026-10-13 02:17:12 +02:00
Niko1221 5cf2eb2788 metal: Q6_K gate/up passes the host's expert format gate (Unsloth UD-Q5/Q6_K_XL)
iq_host.cpp (Metal's copy of the host helpers): Q6_K among the grouped gate/up formats, as mac-kern's moe_gu_14 /
mm_gu3_14 run them. Before, the engine refused UD-Q5_K_XL ('layer 2's experts are Q6_K/Q8_0 ... no GPU kernels').
2026-10-13 02:17:11 +02:00
Niko1221 14817168b6 One strata binary for both families: the Qwen3.6 engine path as a library
strata reads the --native file's architecture at the top of main() and hands
a qwen35moe model to strata::qwen35::run (src/qwen35, now the strata_qwen35
library; strata-q36 wraps the same entry point) before any Flash-Next state
exists; every other file runs exactly as before. strata --families lists the
architectures a build runs. The Qwen3.6 path takes strata's flag names where
they mean the same (--native, --max-new, --pool-workers, --no-token-graph,
--gpu, --mtp N) and skips Flash-Next-only ones in serve mode. The SYCL build
keeps refusing the family, now saying CUDA and HIP run it.

setup: Qwen3.6-35B-A3B is a family in the menu (OWN_PATH_FAMILIES,
install_own_path): size, context, the engine (checked with --families), the
pinned GGUF, its tokenizer, a config with --native/--max-context/--mtp 2 and
the start script; no pack, profile, draft layer or tuning. The engine step is
a function shared with the Flash-Next path. strata_inspect says the family
runs on CUDA and HIP.
2026-10-13 02:06:01 +02:00
Niko1221 2a0d2f5521 SYCL dense GEMV and HC read: bit-identical variants measured, kept opt-in (none reaches +3% end to end)
- dense_bench: the decode window's native_mmvq calls at the model's shapes and formats, 24 matrices per shape in one
  graph (weights from DRAM), 1-4 columns, us / GB/s and an output hash.
- STRATA_MMVQ_WIDE_K_RPW=2|4: Q4_K / Q5_K / IQ4_XS wide kernel with 2 or 4 rows per sub-group sharing each column's
  activation loads (apply split into act + dot, the same operations); every hash equal. B70, 3 columns: Q4_K
  6144x2560 56.6 -> 27.7 us, Q5_K 2560x12288 89.8 -> 75.5 us, Q6_K unaffected.
- STRATA_MMVQ_WIDE_RPW>1 and STRATA_MMVQ_PF now take the aligned Q6_K loads (they were 2-4x slower than the default
  without them); still slower than the default except Q6_K 6144x2560 at RPW 2 (50.2 -> 45.3 us).
- fused GR read: the up step's columns per block (STRATA_GR_UP_COLS 4/8/16/32/64, default 16 = before) and its T
  butterflies interleaved (STRATA_GR_UP_ILP=1); gr_bench checksums equal for all. 32 columns: 3 tokens 50.7 -> 48.9 us,
  1 token 27.5 -> 29.4; ILP slower.
Engine, B70, IQ3_S with data/expert-profile-sycl.bin, 5 interleaved pairs: STRATA_MMVQ_WIDE_K_RPW=2 STRATA_GR_UP_COLS=32
vs default: decode after a 4K prompt 54.31 -> 54.00 tok/s, story 38.41 -> 38.50, outputs identical. Defaults unchanged.
2026-10-13 02:05:09 +02:00
Niko1221 e48c803e01 native packs: the sized cache's budget counts the 256-byte slots it is filled with - an NVFP4 blob (2,764,816 B with its scale tail) got one slot fewer than asked, none at --expert-cache 1
Found by the quant yardstick (--expert-cache 1) on the NVFP4 GPTQ pack: 'pre-filled 0 of 0 slots', and the verify windows
refused to run.  Every other native format's blob is rows x row bytes with 1280 / 2560 rows, a multiple of 256, so its
budget is unchanged.
2026-10-13 01:49:34 +02:00
Niko1221 1d21b9cd6c MTP drafter on HIP: the draft stream at high priority by default on gfx1200/gfx1201 (STRATA_MTP_HIP_PRIO=0 off, =1 on other HIP cards)
ROCm gives out its hardware queues per priority level; the draft stream shared one with the verifier's, so the
draft round ran only after the commit graph the verifier had just launched. With a queue of its own the commit
runs beside the draft. R9700, UD-Q4_K_XL: draft wall 5.8 -> 3.5 ms a window (STRATA_DRAFT_TIMING).
2026-10-13 01:42:48 +02:00
Niko1221 015039e865 README translations: Unsloth's UD-Q4_K_XL to Q8_0 as in the English README 2026-10-13 01:41:10 +02:00
Niko1221 6f14e90a58 docs/QUANTS.md: every expert in RAM vs the old budget (5-pair A/B), the route tail skip's cost on the K-quant files 2026-10-13 01:38:28 +02:00
Niko1221 6f3e38100e Commit graph: every GDN layer's conv history + state, and every QSA layer's tail restore + indexer append, in one launch each (STRATA_COMMIT_BATCH, default on; =0 keeps the per-layer loop)
The per-layer commit was ~98 graph nodes; it runs between a window and the draft round, and on HIP the round
waits for it (R9700 UD-Q4_K_XL: 2.35 -> 0.65 ms GPU). Each layer's arithmetic is the per-layer call's:
gdn_parity (gdn_commit_layers vs the per-layer commit, n_keep 1-8) and qsa_parity (append_steps_layers vs
copy + append_steps, 3 scalings) are bitwise. STRATA_COMMIT_TIMING=1 prints the commit graph's GPU time.
2026-10-13 01:24:55 +02:00
Niko1221 655145426d MTP drafter: STRATA_MTP_PROFILE (stage stamps in the draft graphs), host split in STRATA_DRAFT_TIMING, draft graph node listing; STRATA_MTP_HIP_PRIO=1 opt-in (the HIP draft stream at high priority, its own hardware queue)
All measurement switches are off unless set; the default path is unchanged.
2026-10-13 01:24:54 +02:00
Niko1221 de9ea19bae setup: NVFP4 below its RAM need takes the RAM-budget mode (--resident-budget-gib, RAM less 24 GB), not the mapped experts
On the RTX 5070 PC (64 GB, Windows) --mmap-experts let the experts it read stay in the engine's working set: free RAM
fell to 0.6-4.9 GB and the memory guard stopped the engine three times (2026-10-13).  --resident-budget-gib 22 with the
same pack kept >= 22.8 GB free.  The budget mode is what setup already writes for Unsloth's files (resident_budget_gib:
40 GiB on 64 GB); the pack's experts.bin is the file the rest are read from.  test_setup_nvfp4: 64 GB -> budget 40.
2026-10-13 01:14:13 +02:00
Niko1221 c711ba071f Images (I2): setup builds the native DiT runner into sd-cli on CUDA / HIP; --image-native on|off (off by default) and the config's "native" add --model-args qwen_image_2_1_native=1 2026-10-13 01:07:28 +02:00
Niko1221 221834c6fc quant_quality: --gpu-experts N (e.g. auto) for a measurement at a real expert-cache size 2026-10-13 00:57:20 +02:00
Niko1221 31f26683c2 metal: the quant yardstick on Metal - window_logprobs from the window's head logits; head logits in every window under STRATA_LOGPOS
Verifier::window_logprobs reads the head rows through window_copy_logits under STRATA_GPU_METAL (the CUDA/HIP path
unchanged: verify.cpp's object is byte-IDENTICAL on P1004 and p620). The prompt's windows sample nothing (head
sampling off), so with STRATA_LOGPOS the Metal window computes the head anyway.

tools/quant_quality.py on the M5 Max against the Q8_0 reference (q8_0.tsv 627558d1, set.ids.json 414634dd), 4,995
positions: UD-Q4_K_XL 92.77% top-1, KL 0.0548 (p99 0.855), PPL 4.043 vs 4.052 (CUDA/HIP: 93.09% / 0.056); IQ3_S
87.85% / 0.1422 (CUDA/HIP: 87.7% / 0.144). The Metal window keeps int8 K/V (the yardstick asks fp16).
2026-10-13 00:54:44 +02:00
Niko1221 5be585a2cf Images (I2): native Qwen-Image-2.1 DiT runner inside stable-diffusion.cpp, opt-in
tools/image/sdcpp_native: a runner that replaces QwenImage21Runner::compute only (sd.cpp keeps the text
encoder, VAE, sampler and RNG), chosen with --model-args qwen_image_2_1_native=1 in CUDA / HIP builds.
- The text prefix runs once per condition (causal, the t = 0 modulation row) and leaves each layer's
  fp16 K/V; each step runs the 32 blocks on the image tokens only.
- GEMMs stay ggml graphs on the runner's backend (sd.cpp's MMQ / BLAS kernels and numerics); fused
  LayerNorm+modulate, gate+residual(+next LayerNorm), q/k RMSNorm+RoPE+fp16 K/V, SwiGLU and, on gfx11, a
  WMMA flash attention (8 waves x 16 queries, 64-key tiles, fp32 softmax) run on the backend's stream.
- apply.py copies the two files into the pinned checkout and makes two small edits (runner choice,
  SD_QI_NATIVE for CUDA / HIP builds); anything not covered falls back to the stock runner.

780M, Q8_0, seed 42: 512 2.11 s/step (stock 2.85), 1024 9.58 s/step (stock 14.2), image 96.5 s (131.8).
2026-10-13 00:52:46 +02:00
zsariboga caf1856bd9 pool: STRATA_POOL_UMWAIT=<cycles> - parked workers wait with UMONITOR/UMWAIT instead of spinning (opt-in, x86 WAITPKG, MSVC x64)
A parked pool worker watches the epoch's cache line with UMONITOR and sleeps in UMWAIT (C0.1) until the host publishes or the cycles run out. On a laptop whose CPU and GPU share one power budget the spin's power goes back to the GPU. Unset, 0, no WAITPKG or a non-MSVC build: the spin, unchanged. STRATA_POOL_UMWAIT_C02=1 picks the deeper C0.2 state, STRATA_POOL_UMWAIT_ESC=<cycles> C0.1 first and C0.2 after.
2026-10-13 00:50:45 +02:00
Shawn Jackson 8825d337c1 generate: the prompt-chunk sizing and the state fingerprint take their QSA shapes from qsa_shapes(g)
Two places upstream builds QsaShapes by hand from the geometry (chunk_bytes, added since the geometry change was
written, and the conversation-state fingerprint, in both trees). They now call qsa_shapes(g) like layer, mtp,
session, verify and prefill; for every file the guard accepts the values are the ones they built before.
2026-10-13 00:49:58 +02:00
Shawn Jackson 51329cff3c artifact: read_geometry refuses another architecture by name before reading any qwen4exp key
generate.cpp reads the geometry before anything else looks at the model, so a model of another architecture was
refused as "missing qwen4exp.block_count" where 0.1.40.2 said "architecture is 'qwen35', this engine requires
'qwen4exp'" (found running Qwen3.8-27B through --native). read_geometry now checks general.architecture first, and
check_architecture is read_geometry plus geometry_unsupported. gguf_geometry_test: a dense qwen35 file (Qwen3.8-27B's
keys) and a file without general.architecture.

Tested on gfx1151 (HIP build of this branch, Windows): the GGUF tests pass; Qwen3.8-27B UD-Q6_K_XL, Q8_0 and
HauhauCS Q4_K_P are refused by architecture in 0.5 s; Qwen3.8-Flash-Next UD-IQ4_XS, greedy, 256 tokens on three
prompts (39, 48 and 5,568 tokens), gives output identical to the installed 0.1.40.2.
2026-10-13 00:49:57 +02:00
Shawn Jackson fc72322bd1 tests: the GGUF fixture's metadata helpers set fields by name
The f32 and array members added for gguf_geometry_test left the braced Kv{...} initialisers short of the struct, which
clang's -Wextra reports as -Wmissing-field-initializers in every test that includes the fixture (MSVC /W4 does not).
make_kv() sets the key and type, and each helper sets its own value. Built with the HIP toolchain (gfx1151):
gguf_geometry_test, gguf_split_test, gguf_reader_test and native_dense_ple_key_test pass, with no warning from the
fixture.
2026-10-13 00:49:57 +02:00
Shawn Jackson f812e9a162 core: the model's geometry is read from its GGUF metadata and checked against the shape the kernels are compiled for
read_geometry (gguf_reader.hpp) fills ModelGeometry from the qwen4exp.* keys, under llama.cpp's names: layers, widths,
GDN, attention, indexer (heads, key width, pooling block from compress_ratios, top-k), rope width, hyper-connections,
experts, RMS epsilon, and the vocabulary from tokenizer.ggml.tokens. The six keys the guard always required stay
required; any other absent key keeps the compiled value. Keys that state one thing twice must agree (the interval and
compress_ratios, key and value head widths, routed and shared expert widths, ssm.inner_size and state x v heads).

check_architecture no longer hard-codes 48 / 2560 / 24 / 2: it reads the geometry and refuses a file the kernels are not
compiled for, naming the key ("qwen4exp.block_count = 40, the kernels are compiled for 48"). Only the expert count,
the experts per token (1..64) and the vocabulary may differ. ModelGeometry moves to core/geometry.hpp (no
dependencies), with kCompiledGeometry; the kernels that fix a dimension at compile time static_assert it there, in the
CUDA/HIP tree and the SYCL copies (plus qsa_prompt_attn_xmx).

generate.cpp takes the geometry and K from read_geometry, and the native embedding takes n_embd / n_vocab from it
instead of a literal 248320. The QSA kernel shapes come from one qsa_shapes(g) (layout.hpp) in layer, mtp, session,
verify and prefill, so n_rot, idx_block and idx_top_k follow the geometry too. Mirrored in sycl/.

Tested: tests/core/gguf_geometry_test.cpp (new, synthetic GGUFs) 28/28, gguf_split_test and gguf_reader_test pass,
built with MSVC /W4 /WX on Windows. `gguf_geometry_test --real` on Unsloth's Qwen3.8-Flash-Next UD-IQ4_XS shard 1
reads exactly the compiled shape and the guard accepts it. Not built here: the engine (no CUDA, HIP or oneAPI compiler
on this PC), so generate/layer/mtp/session/verify/prefill and the kernel asserts are reviewed, not compiled.
2026-10-13 00:49:57 +02:00
Niko1221 20b501743d setup: Ubuntu keeps its checked cuda-toolkit-13-0 install; Debian stops with a message when no 13.x installs (#2172 follow-up)
The Debian fallback loop ran with check=False on Ubuntu too, so a failed 13.0 install (a
network or apt lock error) went on to 13-1..13-4 and then past the loop without a word. Ubuntu
is back to the single checked install of 0.1.43; Debian tries 13-0..13-9 and fails with
guidance when none installs. Checked against NVIDIA's repos today: debian12 and ubuntu2404
carry 13-0..13-3, debian13 starts at 13-1 (up to 13-4). Two tests added.
2026-10-13 00:49:11 +02:00
Gabriel Popa fb7ee72471 setup: install the CUDA Toolkit on Debian 12/13 too, not Ubuntu only
The CUDA apt repo keyring and cuda-toolkit-13-x packages exist for
debian12/debian13 as well as ubuntu2204/ubuntu2404, so derive the repo
name from /etc/os-release instead of hardcoding ubuntu. Debian 13's
repo carries no cuda-toolkit-13-0, so try 13-0..13-5 in order (Ubuntu
still gets 13-0 first, same as before).

Checked with tools/test_setup_cuda_repo.py (4 tests, no GPU or network
needed): repo mapping for ubuntu/debian/other and the minor-version
fallback.
2026-10-13 00:48:38 +02:00
zsariboga d8a49433e3 serve: "env_to_user" (opt-in) moves opencode's <env> block into the first user message, so a new folder or day reuses the cached system prompt (11.9 -> 1.1 s first request)
opencode puts its <env> block (working directory, today's date) near the start of the system message, before ~40K characters that never change. The prefix cache only matches a prompt's start, so each new folder or day read the whole system turn again. With the switch the block goes to the front of the first user message (string or content parts), for OpenAI and Anthropic requests. Off by default; serve/test_env_to_user.py; docs/DETAILS.md.
2026-10-13 00:48:16 +02:00
SuperDistance ac21812c1a serve: a long read gives way to a short request at the next chunk, not a chunk later or never (#656)
The server sends BYIELD for a waiting request under half as long only when a PP line (or the 10 s heartbeat) arrives,
and the engine looks for it at its next boundary. Where a boundary's own PP line is the only line for a while - every
chunk of a --batch read on one GPU, and #2050's pipelined pieces - the BYIELD answers that line and reaches the engine
after it has looked: the read gives way one boundary later, or never when that was the last one it could take. It is in
time only where the engine saves a periodic checkpoint after the PP line (every 16,384 tokens by default).

_control() now also calls on_quiet() whenever the engine has been quiet for half a second (its existing poll), and the
solo and admission reads run the same check there as at a PP line. A BYIELD for a read that has ended is ignored by
the engine, as before. serve/test_parallel.py: the fake engine takes --pp-every K (a PP line and a boundary every K
chunks); a solo read and an admission must give way at their first boundary (both fail on main), and the rule (only
to a much shorter request) holds on the poll too.

Measured on one RTX 5090 (Swift 1.5 IQ3_S, --batch 4, the engine built from main), main's server and this one, two
engine starts each (M F F M): a one-line request sent 0.2-8.6 s into a cold ~33K read, with nothing else decoding and
with one stream decoding. Late (a boundary passed after the request waited): 28 of 40 runs on main, 12 of which never
gave way; 4 of 40 with this change, all requests that started waiting 0.02-0.10 s before that boundary's PP line.
Where main was late the short request's TTFT went 3.56 -> 1.70 s (nothing decoding) and 4.40 -> 1.57 s (one stream);
where main never gave way 4.60 -> 1.51 s and 7.59 -> 2.35 s. Where both gave way at the same boundary, both requests'
TTFTs were the same (medians within 0.02 s). On 2x RTX 5090 with #2050's STRATA_PIPED_READ_PIECE=8192: 6 of 12 late without
it, none with it.
2026-10-13 00:46:48 +02:00
Niko1221 9c5a45f879 SYCL layer split auto: among near-tied placements, the one that leaves its tightest card the most room (#1440)
On two equal cards every K whose caches hold all the profiled pairs predicts the same window
time, and the search kept the first one tried: the smallest K, which gives the last stage (it
also carries the MTP drafter) the most layers and lets its cache fill the card to the reserve.
2x Arc Pro B70 with IQ3_S: auto K=24, CUDA1 at 31.8 of 31.9 GiB, ~11 tok/s; --layer-split 30
~54 tok/s (agnaylor in #1440). A full xe card does not fail an allocation, it moves buffers to
system memory.

The SYCL search now scores every placement it tries with its smallest spare room (a stage's
room after its session minus every expert of its layers) and, among the placements within
STRATA_SPLIT_HEADROOM_TOL (default 0.2%) of the best predicted time, takes the one with the
most (split_rules::pick_with_headroom). A log line names both picks when it moves the split.
STRATA_SPLIT_HEADROOM=0 keeps the old pick. CUDA/HIP placement is unchanged.

Test: split_rules_test (a 48-layer two-card model of the report: plain K=24 with 664 MiB spare
on the last card, headroom K=25 with 1728 MiB; tolerance and tie rules).
2026-10-13 00:44:27 +02:00
Niko1221 62185aab77 serve: a quoted <tool_call>, <tool_response> or </tool_response> is text too (#2026 follow-up)
The three are USER_DEFINED (type 4) tokens like </tool_call> (ids 248058, 248066, 248070), so
the tokenizer matched them inside a message's text as the real call / tool-result markers.
They join TOOL_LITERAL_TAGS: inside a user, system or tool message (and the tool list) they are
encoded as ordinary text; an assistant message replaying raw call text keeps its tokens, as
for </tool_call>.

One exception keeps the template's behaviour: a user message whose whole text is
<tool_response>...</tool_response> (a client that formats tool results itself) keeps that outer
pair as the markers, since the template tells such a turn from a real question by exactly that
pair (last_query_index); the text inside is still marked.

Found by 47Hunter47 in #2026. Tests: serve/test_frontend.py QuotedToolCallCloser (every tag
and role, the assistant replay, the outer pair as string or text parts).
2026-10-13 00:40:31 +02:00
Niko1221 dbf4015069 setup on macOS: the Unsloth sizes offered (UD-Q4_K_XL measured resident on 128 GB; Q5/Q6 stream), the family's pack args (--compat-bf16 for the host's copies)
M5 Max: ./setup.sh --model UD-Q4_K_XL --vision yes: shards checked by SHA-256, repacked and verified, the pack, the
image encoder; the config it writes answers the test picture ('STRATA 42', 'Red circle / Blue square / Green
triangle').
2026-10-13 00:37:29 +02:00
Niko1221 54eb42a3f3 strata-q36: hybrid experts - VRAM slots, the CPU pool, streamed prompt layers
The routed experts are placed after the engine's buffers: all in VRAM when
they fit (unchanged), else every expert in host memory with the same number
of VRAM slots per layer (--expert-vram GiB, --expert-slots N). Batches of up
to --hybrid-rows rows run the resident experts on native_expert_grouped and
the others on the CPU ExpertPool (ggml-cpu arithmetic) side by side; bigger
prompt chunks stream each layer's experts into two staging buffers for MMQ
(prompt chunk 2048 in this mode). Slots follow the routing counts (after a
prompt, every 32 decode positions). ExpertPool's native phases split rows by
the layer's n_ff / n_embd instead of Flash-Next's constants (same values
there). STRATA_MMQ_KQ=1 by default for this family.
2026-10-13 00:36:25 +02:00
Niko1221 2fd1a5b091 metal: Unsloth UD-Q4_K_XL runs - the F32 PLE conv kernel narrowed to F16 at load (as the host loader), no zero-size HC copy buffer
ple_conv_add reads the conv kernel as half: Unsloth's file stores it F32, so it is narrowed (round to nearest even)
into its own buffer; the GSQ packs' F16 tensor is read in place as before. A file whose HC matrices are Q8_0
already makes no copies (the copy buffer was allocated with 0 bytes and the Metal allocation failed).

M5 Max, UD-Q4_K_XL resident (the repacked 82.5 GB file): greedy prompt1 identical to the CUDA reference (z440),
story first differs at token 36; MTP output equal to no MTP; decode no MTP 60.6 / 60.0, MTP --spec 2 93.9 / 74.0
tok/s; prompts 4K 1,374 / 32K 1,288 tok/s, needles found (4K, 32K). llama.cpp Metal on the same shards: 48.4 /
47.3 decode, 821 / 739 prompt.
2026-10-13 00:33:54 +02:00
Niko1221andradoslawsz 277be72a7f Foresight swap: an exchange holds the experts whose bytes it moves, so no slot is served with another expert's bytes (#2134)
With STRATA_FS_SLOTS > 0 and the resident RAM mode's adaptive exchanges, a slot copy issued
from an expert's RAM place (note_miss / the router look-ahead's predict) could land after an
exchange rewrote that place with the outgoing expert's bytes; take() then served expert Y's
bytes for expert X (FS_VERIFY: 19 BAD on clean v0.1.43, reported in #2134).

commit_exchanges, commit_copies and commit_flip now hold every staged expert (in and out) in
the swap space while their bytes move: its slot is dropped (never served again) and no copy
of it is issued until the commit is done (commit_copies holds until commit_flip, so the
asynchronous tier's half-done exchange is covered, and the rotation path too). Unlike the
patch in the issue, a dropped slot is not handed out at once: it waits the same `depth`
finished windows as any slot a window may still read, since a window planned before the
exchange can be reading it. "N dropped by exchanges" joins the swap space's report.

Foresight is opt-in; with STRATA_FS_SLOTS unset nothing here runs. Test:
file_expert_source_test --rotation-gpu (both commit paths, the depth rule, the bytes after).
Based on the analysis and patch in #2134.

Co-authored-by: radoslawsz <[email protected]>
2026-10-13 00:31:31 +02:00
Niko1221 ad8f53fa7a HIP gfx12: the bitwise grouped K-quant decode experts (STRATA_EXPERT_V2K) on by default (R9700 UD-Q4_K_XL +0.2..1.2%, the same text) 2026-10-13 00:30:33 +02:00
Niko1221 dd5f09a50e docs: QUANTS.md speed, bound, CPU pool, dense GEMV, HC and images; MODELS.md one table for Unsloth's five files (old anchors kept); UNSLOTH_Q4/Q6 status; README 2026-10-13 00:30:02 +02:00
Niko1221 6339e8610d serve: the stdin reader shares its state with main instead of borrowing main's locals (QUIT race)
The stdin reader thread is detached and captured main's stop flag, mutex, condition variable,
line queue and EOF flag by reference. The server writes QUIT and closes stdin right after it,
so the reader often wakes with EOF while main is returning: it then locked a destroyed mutex
and wrote to a destroyed deque (undefined behaviour on every platform; the Metal port aborted
on it). The state now lives in a shared_ptr that the thread holds a copy of, so it outlives
main's frame for as long as the reader runs. CUDA/HIP and SYCL engines alike; no change to
what the engine reads or answers.
2026-10-13 00:28:26 +02:00
Niko1221 f5082f41a6 Sources: no raw NUL bytes (expert_source.cpp, pinned.cu) + a test that scans for them
expert_source.cpp's Linux-only STRATA_TEST_UNMAP_GGUF check had a literal 0x00 byte as its
character constant (since 0.1.43; gcc warned "null character(s) preserved in literal"), and the
#2126 buddyinfo parser in pinned.cu got one the same way (a heredoc dropped the backslash).
Both are the escaped character now. Git took pinned.cu for a binary file because of the byte,
so it was committed with CRLF line endings; it is back to LF (no other change).

tools/test_source_nul_bytes.py fails when any tracked source file holds a NUL byte again.
2026-10-13 00:26:51 +02:00
Niko1221 12be1059df setup: images on by default for Unsloth's files where the encoder has room (NVIDIA 16 GB and more: the GPU encoder; AMD Linux with every expert in RAM: the CPU encoder); UD-Q4_K_XL's images no longer untested (run end to end on an RTX A4000 and an R9700); the yardstick keeps a config's --resident-budget-gib 2026-10-13 00:08:31 +02:00
Niko1221 3e11f786af Merge f-quants (945fee23) into f-qwen36: the K-quant CPU pool rows, libstrata_kq, the K-quant GEMV work 2026-10-12 23:48:28 +02:00
Niko1221 f29e74c721 strata-q36: FP16 prompt attention and a column-split GDN chunk kernel
Prompt chunks run the attention scores and P V as FP16 strided-batched GEMMs
with FP32 accumulation (K / V cells and Q converted per chunk, the causal
softmax writing FP16 probabilities), replacing the FP32 SGEMMs. The prompt's
delta rule splits each V head over 4 column groups (16 row groups of 8 rows,
the norm in its own kernel); decode, verify windows and commits keep one
kernel. pp4096 on gfx1201: 2557 -> 3290 tok/s.
2026-10-12 23:48:13 +02:00
Niko1221 3a639f7357 Merge branch 'f-quants' into f-intel-quants 2026-10-12 23:43:12 +02:00
Niko1221 ae02e7db17 metal: vision - image rows in the prompt, M-RoPE (qsa_prep constant 24), setup's --vision
The prefill hook hands the prompt's image rows (sp.embd_rows) to the Metal prompt paths (prompt_chain.inc, the
streaming prompt path, the 8-row windows), which take an image row in place of the token embedding.
mrope_table_set keeps the host view of the server's (t, h, w) table; every window, prompt chunk and draft round copies
the rows it reads into the window's buffer, bound at index 20 of qsa_prep with function constant 24 (mac-kern
29724e5e) - the window, both prompt paths and the draft layer. The cached window's QsaArgs are patched by the last
inline argument (the table is bound after it). setup_mac.py --vision yes: the pinned projector (SHA-256) and
strata-vision built with ggml-metal; the config gets --vision and the encoder.
missing.cpp: a crash prints its call stack and fault address.

M5 Max, Q2_0, ISTA's projector, a test picture (shapes and 'STRATA 42'): 'STRATA 42' and 'Red circle / Blue square /
Green triangle', the same answers as the HIP engine on the R9700 (p620) with the same picture and files, also
through the config setup writes. Text greedy unchanged with and without --vision (prompt1, story, the 1,662-token
chain prompt). prefill.cpp: the CUDA (P1004) and HIP (p620) objects are byte-IDENTICAL (preprocessor-only).
2026-10-12 23:36:13 +02:00
Niko1221 38436f2a44 strata-q36: decode always on the per-token GDN kernels
The greedy divergence seen with the fused single-token kernels is not a
kernel bug: fused_gdn_conv_l2 and k_conv_seq leave the same history and
differ by 1 ulp in h (FMA contraction of the 4-tap sum); the Q8_1
activation rounding turns that into a flipped near-tie (p01 token 4,
margin 0.43). KL with either is the same. Drop the switch.
2026-10-12 23:30:40 +02:00
Niko1221 58515d2b9c SYCL prompt path: the 32-value block dequantizers (Q5_1, Q8_0, Q4_0, Q4_1, Q5_0) write neighbouring values from neighbouring lanes, as vector stores
Lane tid wrote block tid % 8 of its superblock, so a sub-group's 2-byte stores landed 64 bytes apart. Lane tid now
takes block tid / 4 (the same 32 (block, part) pairs, the same values) and writes its runs with store_run. UD-Q4_K_XL's
down projections are Q5_1 and UD-Q5/Q6/Q8_0's are Q8_0, so this is most of their FP16 prompt-path dequant.

B70, dequant_bench (32 experts, gate/up + down): Q5_1 7.18 -> 2.83 ms, Q8_0 6.77 -> 3.53 ms; Q4_0 3.21, Q4_1 2.76,
Q5_0 3.74 ms after; the FP16 output hash of every type unchanged. dequant_bench lists Q4_0 / Q4_1 / Q5_0 too.
2026-10-12 23:29:06 +02:00
Niko1221 13ed803334 metal: the M4's expert tiles for the Unsloth formats (mac-kern 513fc615): decode down Q5_1 4,1,4 and Q8_0 8,1,4 without the M5 layout; v2 prompt K-quant gate/up 16_16_128_2 2026-10-12 23:19:58 +02:00
Niko1221 e65858f84f SYCL: Unsloth's expert formats decode each weight word once per grouped call (KqSplit), bitwise
The lane kernels of native_expert_grouped re-read and re-decoded a Q4_K / Q5_K / Q6_K / Q5_1 / Q8_0 expert row once
per entry of its group (the IQ formats already decode once, Multi). Each of these dots is now split into load (the
weight words and scales) and apply (the activations and the same float expression), and the group's entries go
through the existing multi-entry path in chunks of 4 and 2; every entry keeps its own k order and lane sum, so the
outputs are bitwise the per-entry kernels'. STRATA_KQ_SPLIT=0 keeps the old kernels.

B70 (bmg-g31), 2560 x 640 experts, GB/s of expert bytes, 2- / 4-token windows: Q4_K/Q5_1 362 -> 402 / 369 -> 425,
Q5_K/Q8_0 395 -> 440 / 404 -> 464, Q6_K/Q8_0 259 -> 307 / 256 -> 318, Q8_0 430 -> 479 / 429 -> 502; one token -1..2%.
Experts read from pinned host memory (the NO_HOST mirror) go from 18-19 to 23 GB/s on those windows.

Also: STRATA_KQ_GU_LANES (opt-in; the K-quant / Q8_0 gate/up lane split alone, other bits), native_grouped_parity
--kq (split vs per-entry bitwise, and a CPU float reference with ggml's dequantize_row_* for Q4_K/Q5_K/Q6_K/Q5_1/Q8_0)
and --kqbench (per format pair, device or pinned host memory), Q6_K blobs with a finite scale in that harness,
dequant_bench and kq_cpu_bench for the Unsloth formats in the SYCL build.
2026-10-12 23:19:46 +02:00
Niko1221 b55684708e strata-q36: MTP speculative decode from the GGUF nextn block
--mtp D (0..7) loads blk.<n_layers> (eh_proj, enorm/hnorm, attention + MoE
layer, shared_head_norm) and runs greedy requests as speculative steps:
D chained drafts, one verify window, commit of the accepted rows' GDN state
and conv history. Verify windows and plain decode share the per-token GDN
kernels, so the text is the plain greedy text (10 of 10 prompts identical).
Serve reports accepted/offered drafts in DONE.
2026-10-12 23:19:34 +02:00
Niko1221 e3cd071c21 Merge f-mac-kern 513fc615 (M-RoPE in qsa_prep, faster Unsloth prompt experts, M4 tile notes) 2026-10-12 23:19:10 +02:00
Niko1221 513fc615bb metal: W8<Q6_K> with ushort-pair code loads and vector field math (prompt Q6_K gate/up 14.0 -> 11.0 ms a layer at T 2048; same values) 2026-10-12 23:11:57 +02:00
Niko1221 009dc6c1dc STRATA_DRAFT_TIMING=1: where a draft round's time goes (cudaEvent pairs around the round graph and every step, host wall) 2026-10-12 23:11:00 +02:00
Niko1221 29724e5ea9 metal: K-quant / Q5_1 paired-run prompt decoders (W8) and paired mm_down3; qsa_prep M-RoPE (function constant 24)
- moe_mm.metal: W8<Q4_K / Q5_K / Q5_1 / Q6_K> (the lane's runs k and k + 16 share the sub-block's scale, min and
  high bits; 4-aligned code runs read as one uint); mm_down3 takes the paired-run decode on the COOP_K4 layout (TK
  32).  Prompt per layer at T 2048: Q4_K gate/up 10.6 -> 6.0 ms, Q5_K 15.4 -> 9.0, Q6_K 17.2 -> 14.0, Q5_1 down
  5.0 -> 3.3; the same values (moe_mm_parity v3 incl. Q2_0 and IQ3_XXS / IQ4_NL)
- qsa.metal: QSA_MROPE (function constant 24) - interleaved M-RoPE (ggml IMROPE, sections 11/11/10): buffer 20 of
  qsa_prep = int32 [cells][3] (t, h, w); rotary pair i of the row at pos_base + cell rotates by the cos/sin row
  tab[(pos_base + cell) * 3 + i % 3] (pooled indexer keys at pos_base + 4 b too).  qsa_parity QSA_MROPE=1 (random
  table vs the CPU reference) / QSA_MROPE=id (identity table: the same numbers as the plain kernel)
2026-10-12 23:10:47 +02:00
Niko1221 b1b133e55f metal: Unsloth formats in the window - the file's Q8_0 HC read in place (F32 inject: gr_down_q8f), F32 router/alpha/beta/shexp gate through gemv_0, the v2/v3 expert tile names per format; strata-vision builds on macOS
The HC read takes, per matrix, its load-time Q8_0 copy or the file's own tensor: Unsloth's Q8_0 down/up need no copy,
their F32 inject rows go to gr_down_q8f (mac-kern 945c0d24). router_route_<type> for BF16 and F32 (opt-in). The M4's
v2 prompt kernels: K-quant gate/up TK 128, Q8_0 / IQ4_XS TK 32, Q5_1 / Q8_0 / IQ4_NL down TK 32. GSQ packs unchanged:
Q2_0 prompt1/story, IQ3_XXS prompt1 and the 1,662-token chain prompt give the same tokens as before (M5 Max).
tools/vision/strata_vision.cpp: no posix_fadvise on macOS (it does not have it); strata-vision then builds there with
ggml-metal.
2026-10-12 23:08:35 +02:00
Niko1221 945fee2365 Dense Q8_0 GEMV: one row per block where measured faster (gfx1201: n_in <= 4096 with n_out >= 4096; sm_86: n_out >= 2048), the same bits as two rows 2026-10-12 23:08:31 +02:00
Niko1221 209de26e53 Merge f-mac-kern 945c0d24 (Unsloth formats: Q4_K/Q5_K/Q6_K/Q8_0 experts, Q5_1 down, F32 GEMV, HC down with F32 inject) 2026-10-12 23:02:32 +02:00
Niko1221 5ef9f4c812 q8_gemv_bench: the dense Q8_0 GEMVs at the engine's shapes, 1-6 columns, against the memory bandwidth 2026-10-12 22:59:31 +02:00
Niko1221andsergiywith 23250b5da6 setup: Unsloth's files keep every expert in RAM where they fit with 16 GiB more (the RAM budget below that); UD-Q5_K_XL, UD-Q6_K_XL and Q8_0 in the unsloth family; UD-Q4_K_XL no longer experimental or NVIDIA-only
- unsloth_budget / UNSLOTH_RESIDENT_BESIDE_GB 16, ported from sergqwer's fork (sergqwer/strata-nvfp4 3f3546f6, MIT):
  no --resident-budget-gib where the experts and 16 GiB fit the RAM (UD-IQ4_XS 71.4, UD-Q4_K_XL 87.7, UD-Q5_K_XL 107.6,
  UD-Q6_K_XL 117.7, Q8_0 135.5 GiB), the budget below that, when asked for, and on a unified-memory APU (#1715).
  The menu, --check and the install say which; the menu's default for the family is the largest file whose experts
  all fit (UD-IQ4_XS where none does).
- UD-Q5_K_XL / UD-Q6_K_XL / Q8_0: six shards each, sizes and SHA-256 pinned at 38bb39ee; "ram_gb" is the all-in-RAM
  need, below it setup asks first (default no) and says how many GiB are read from the SSD. Q5/Q6 need engine 0.1.44
  (the Q6_K expert kernels in every build); Q8_0 0.1.38.
- UD-Q4_K_XL: ran on an R9700 (docs/QUANTS.md), so #429's AMD question is gone, and it is not experimental.
- tools/strata_mcp.py mirrors the table and the verdicts. Golden configs re-recorded: only UD-Q4_K_XL on 96 GB
  (65536, --kv-resident) and 128 GB (131072) changed, as in the fork.

Tests: test_setup_unsloth Resident and BigFiles (the budget tests force the budget mode where their PC now fits every
expert), test_strata_mcp, every tools/test_setup_*.py.

Co-authored-by: sergiywith <[email protected]>
2026-10-12 22:57:14 +02:00
Niko1221 945c0d2424 metal: the Unsloth UD-Q4/5/6_K_XL formats - K-quant / Q8_0 / Q5_1 routed experts, F32 GEMV + router, Q8_0 HC read with F32 inject
- Q5_1 (type 7): ref_quant (dequant, random blocks), quant.metal fmt/deq, units.metal Dec (w = d q + m, the min
  through the unit's own x sums) / X, mma.metal DW
- decode experts (moe.metal): moe_gu_{12,13,14,8}_* (Q4_K, Q5_K, Q6_K, Q8_0), moe_down_{7,8}_* (Q5_1, Q8_0)
- prompt experts (moe_mm.metal): W4 decoders Q4_K / Q5_K / Q6_K / Q8_0 / Q5_1; mm_gu3_{12,13,14,8}_*,
  mm_down3_{7,8}_*; mm_gu2 K-quant tiles with TK 128 (16_32_128_2/4, 16_16_128_2, 32_32_128_2), Q8_0 TK >= 32;
  mm_down2_{7,8}_* (TK >= 32)
- F32 (type 0) as a format: gemv_0_*, gemvc_0_*, dq_f16_0, router_route_0
- gr.metal: gr_down_f<SG, KS, F, FI> with the inject rows in their own format: gr_down_q8f_<S>_<K> = Q8_0 down + F32
  inject (gr_down_<S>_<K>, gr_down_q8_<S>_<K> unchanged)
- parity: expert_parity, moe_mm_parity v2/v3 (every gate/up x down pair), gemv_parity / dense_mm_parity F32,
  fuse_parity router_route F32 (bitwise), gr_parity GR_Q8=1 GR_INJ_F32=1
2026-10-12 22:56:44 +02:00
Niko1221 1176eb3c5b strata-q36: the engine path for Qwen3.6-35B-A3B (serve protocol, prompt chunks, decode graph)
A driver of its own beside Flash-Next's (generate/verify/prefill untouched), built from Strata's kernels:
- weights from the GGUF: dense Q8_0 on the native MMVQ (with the packed Q8_0 copy), the routed experts
  repacked at load into native expert blobs (stride in whole blocks for MMQ), F32 norms / projections;
- the plain pre-norm residual layer driver, the residual add fused with the next norm, the MoE end
  (combine + shared-expert gate + add + next norm) in one kernel;
- GDN: the fused conv+L2 and SiLU step for a token, one-launch sequential kernels for a prompt chunk;
- attention without an indexer: split-K decode over 256-cell chunks (positions on the device, so the
  decode step is one captured graph), and for a prompt chunk scores / values as FP32 GEMMs per KV head
  with a causal softmax;
- prompt projections as FP16 GEMMs (prefill::Gemm::native), the F32 router / alpha / beta as SGEMM,
  the prompt's experts on llama.cpp's MMQ (entries sorted by expert on the device);
- decode: native router 256/8, native grouped experts (Q5_K / Q6_K downs added to the grouped
  kernels' down list for FF 512), router + shared gate in one product, alpha/beta in one launch.
- serve: READY / GEN (sampling keys) / T / PP / DONE / STOP, the held state reused when a request
  extends it; --tokens / --force / --dump-full-logits for tools/qwen35/kl.py; --bench-pp / --bench-tg.
- hip_compat cublas_v2.h: cublasSgemm / cublasSgemmStridedBatched.
2026-10-12 22:50:52 +02:00
Niko1221 1269d5fa97 setup on macOS: Unsloth UD-Q4/Q5/Q6_K_XL (not offered until their expert kernels land); the stream rule by size
setup_mac.py: the Unsloth files by name and SHA-256 (setup.py's pinned revision), repacked into one GGUF for the GPU
with every tensor verified, the n-gram table found by name, --native-dense-gguf the repacked file; setup writes
STRATA_METAL_STREAM to the config so the engine follows its choice. UD-* are behind 'missing' (the Metal expert
kernels for Q4_K/Q5_K/Q6_K/Q5_1 are in progress) and only reachable with --model.

stream.cpp: auto streams when the mapped weights + 12 GiB do not fit the GPU working set (was: experts > 60% of
it, which would have streamed UD-Q4_K_XL on the 128 GB M5 Max). M5 Max: Q2_0 stays resident (same tokens and speed);
M4 32 GB: Q2_0 streams as before.
2026-10-12 22:49:27 +02:00
Niko1221 81fefbe953 SYCL native_grouped_parity --kqbench: grouped expert calls per format pair at the model's shapes, us per call and GB/s of expert bytes 2026-10-12 22:48:38 +02:00
Niko1221 a57fa9990e metal/tools/metal_repack.py: a multi-shard GGUF as one 16 KiB-aligned GGUF for the Metal engine
Shard 1's metadata (without split.*) and every tensor but the n-gram table, byte for byte: dense first, then each
layer's routed experts together; --verify hashes every tensor (SHA-256) against its source shard. For Unsloth's
UD-*_K_XL (4-6 shards cutting through layers). Checked on Q2_0 (M5 Max): 35.05 GiB, 1,223 tensors identical,
written and verified in 33 s; the engine on it (--native-dense-gguf the same file, --ple-gguf the original shard 2)
gives the same tokens as on the original shards, prompt1 and story, at the same speed (114.2 / 107.7 tok/s).
2026-10-12 22:45:02 +02:00
Niko1221 74a78869eb nvfp4_decode_bench: NVFP4_BENCH_FILE times real blobs (an experts.bin or its head) instead of random ones 2026-10-12 22:39:47 +02:00
Niko1221 faf6c1c955 Images: Vulkan decodes in 256-pixel tiles at a quarter's overlap (the 780M's 1024x1024 VAE: 23 s instead of 45 s); docs/IMAGES.md speed table 2026-10-12 21:54:20 +02:00
Niko1221 b63e38b1ae docs/QUANTS.md: Unsloth's K-quant files and Q8_0 per role, the kernels per format and backend, the quality yardstick; UNSLOTH_Q4/Q6 notes for the Phase 1 build 2026-10-12 21:48:34 +02:00
Niko1221andsergiywith 5629161b14 mmq_expert_bench: the prompt path's routed-expert products timed at the engine's shapes, NVFP4 beside UD-IQ4_XS's IQ3_S / IQ4_NL
From sergqwer/strata-nvfp4 1688a344 (src/prefill/mmq_expert_bench.cpp), with UD-IQ4_XS's gate/up and down formats
added as comparison cases (an f16 d leads their blocks, as Q8_0's).

Co-authored-by: sergiywith <[email protected]>
2026-10-12 21:26:37 +02:00
Niko1221 6e6d4faf5b decode: NVFP4 down on 8-lane rows (4 a warp, 32 a block) - bitwise warp_sum's tree, the whole expert at 89% of the 3060's DRAM peak
row_dot_20_sub8: lane t takes k = t (+ k = t + 16 for t < 4, else + 0) and k = t + 8 (+ 0), then xor 4, 2, 1 - the
order warp_sum adds a K = 20 row in (lane i + 16, then + 8), so the result is the same bits; native_down_multi_kernel
routes TD 40 there (as IQ4_NL's K = 40 row_dot_40_sub8) and the down grid takes 32 rows a block for d_type 40.

nvfp4_decode_bench, RTX 3060, 64 distinct experts: down alone 0.260 -> 0.189 ms at 1 token (226 -> 312 GB/s, 87% of
360); the whole expert 0.626 -> 0.554 ms (89%), 2 tokens 0.686 -> 0.596, 3 0.786 -> 0.661, 4 0.910 -> 0.766 ms (from
2.261 before Split<40>).  Output hashes unchanged, nvfp4_expert_gpu_parity unchanged.
2026-10-12 21:25:17 +02:00
Niko1221 bcc2a84454 metal engine: the routed gate/up's i-quant grids from constant memory (function constant 23 on moe_gu_*; STRATA_METAL_CTAB=0 off) - IQ3_S spec 2 decode +0.7-1.0%, greedy tokens unchanged 2026-10-12 21:17:53 +02:00
Niko1221 af6738eb74 metal: MM_PROBE 7 - a repacked i-quant layout's cost (one aligned 16-byte load a lane a row a 32-group): IQ3_S prompt gate/up 9.5-10.9 -> 6.8-6.9 ms a layer 2026-10-12 21:17:53 +02:00
Niko1221 c7070a72cf kq_cpu_bench: the CPU pool's expert rows per format on T threads in GB/s of weights; tools/quant_bound.py: decode bounds from the per-layer formats; setup copies libstrata_kq next to a compiled engine 2026-10-12 21:16:33 +02:00
Niko1221 546d767543 decode: NVFP4 experts decode each weight part once per call for every token of the window (Split<40>), its UE4M3 scales from bits, down on 16-lane rows - bitwise the same, 1.14-2.5x
The ported NVFP4 decode experts had no Split, so they took the per-entry kernels: every weight was decoded again for
every token of a verify window, and the kernel slowed almost linearly with the tokens per expert.  Measured with
nvfp4_decode_bench on the RTX 3060 (64 distinct experts, sm_86, CUDA 13.4; best of 50):

| tokens per expert | before | after | |
|---|---:|---:|---:|
| 1 | 0.714 ms (248 GB/s) | 0.626 ms (283 GB/s) | 1.14x |
| 2 | 1.200 ms | 0.686 ms | 1.75x |
| 3 | 1.730 ms | 0.786 ms | 2.20x |
| 4 | 2.261 ms | 0.910 ms | 2.48x |

(UD-IQ4_XS's IQ3_S / IQ4_NL experts, 15% fewer bytes: 0.512 / 0.578 / - / 0.814 ms.)

- Split<40>: vec_dot_nvfp4_q8_1 split into load (the E2M1 codes through kvalues_fp4, the two sub-block scales) and
  apply (the dp4a chain and the float expression in the same order), so the multi-entry kernels apply one decode to
  up to four tokens.  The multi kernels now apply the NVFP4 tail scales on their outputs as the per-entry kernels do
  (s_gate / s_up on gate/up, s_down on down; x * 1.0f for every other format, which is exact).
- ue4m3_to_f32_half builds the value from its bits instead of two ldexpf and branches: equal for all 256 bytes
  (nvfp4_decode_bench checks them).
- down: NVFP4's rows are K = 20 calls at n_ff 640, as Q2_0's, so they take row_dot_20_sub16 (2 rows a warp, 16 rows a
  block) instead of one 20-of-32-lane row a warp: down 0.361 -> 0.260 ms at 1 token.  Its addition tree is
  warp_sum's (lane i + lane i+16 first), bitwise.

Bit for bit: the output hash of every case above equals the per-entry kernels' (STRATA_OLD_IQ_MMVQ=1);
nvfp4_expert_gpu_parity on real GPTQ blobs unchanged (1.088-1.168%).  Other formats: only the store multiplies by a
1.0f scale, and the down launch grid changes for d_type 40 alone.
2026-10-12 21:13:50 +02:00
Niko1221 8cc41f3ac6 nvfp4_decode_bench: the decode experts' weight bandwidth on random blobs of any format pair, with an output hash
nvfp4_decode_bench [gu_type d_type] [groups] [tokens per group] [iters]: native_expert_grouped on distinct random
blobs (64 x 2.76 MB, far past L2), best/mean time, weight GB/s against the card's DRAM peak, and an FNV hash of the
output so a kernel change can be checked bit for bit.  NVFP4_BENCH_PHASE=1|2 times gate/up or down alone.  Also an
exhaustive check of the UE4M3 decode built from bits against the ldexpf form (all 256 bytes).
2026-10-12 21:13:50 +02:00
Niko1221 de81ac91f4 Qwen3.6 T1: SiLU-gated GDN step and a 256/8 router in the native kernels; q36ref runs on them
- fused_gdn_step_norm_silu: the fused GDN step with the SiLU output gate (Qwen3.6), a compile-time
  template flag; fused_gdn_step_norm (Flash-Next, sigmoid) is the same kernel instantiated as before.
- native_router_topk(n_expert, k): 512/10 goes to the existing pinned top-10 kernels unchanged; 256/8
  is a templated copy of the same topk-moe contract.
- strata-q36ref --kernels strata (default): native MMVQ (q8_1) for every quantized matmul, dense and
  per routed expert (Q4_K/Q5_K/Q6_K/Q8_0), the fused GDN conv+L2 and SiLU step, the native router;
  --kernels plain keeps the reference kernels.
- kl.py: answer-only statistics beside all steps, llama.cpp's logits cached between runs.
2026-10-12 21:11:03 +02:00
Niko1221 c43c9ac948 Merge commit '450c82f5' into f-mac-iq3s 2026-10-12 21:07:47 +02:00
Niko1221 465cd61dfb Formats cost nothing unless used: Q6_K gate/up experts in the normal build; the K-quant MMQ prompt kernels on by default, on CUDA as their own library (strata_kq) opened only for a model with these experts and STRATA_MMQ_KQ=1 2026-10-12 21:04:17 +02:00
Niko1221 450c82f5e2 metal: IQ3_S pack work - QUANT_CTAB (constant-memory i-quant grids, function constant 23), mm_gu4 / mm_gu5 probes
- quant.metal: QUANT_CTAB (function constant 23): the IQ3_XXS / IQ3_S / IQ2_XXS / IQ2_XS / IQ2_S grids and the sign
  table read from ggml-common's constant arrays, no threadgroup copy (same values).  Decode moe_gu: 3-7% faster
  (IQ3_S-pack T2 window 14.96 -> 14.79 ms, 3 of 3); token_chain default (CHAIN_CTAB=0 off).  Prompt mm_gu3: slower
  (kept off)
- moe_mm.metal: mm_gu4 (decoded operand shared by MS 16-row token tiles): IQ3_S 9.5 -> 8.6 ms, IQ3_XXS equal, Q2_0
  slower; mm_gu5 (weight blocks staged in threadgroup memory, coalesced): slower (12.9-20.6 ms); W8 templated on the
  address space for IQ3_S / IQ3_XXS / IQ2_S; probes MM_PROBE 5 (loads only: as slow as the full decode)
- moe.metal MOE_PROBE (function constant 22; no table load)
- finding: the prompt i-quant decode is bound by its scattered small device loads (2-7 a 4-value run, 8 rows a load
  instruction), not by the grid lookups or the ALU
2026-10-12 21:03:58 +02:00
Niko1221andsergiywith fab8c10547 setup: the NVFP4 family - sergqwer's ready-made packs (GPTQ default, NVIDIA's checkpoint second), pinned and SHA-256-checked
--family nvfp4 --model NVFP4 | NVFP4-NVIDIA (MODEL-UPDATE Phase 4).  The files come from the packs' Hugging Face
repositories at the commits the fork's setup pins (NVFP4_REPOS: Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata
640a1db9, ...-NVIDIA-NVFP4-Strata cca8f7fe), each checked against its size and SHA-256; nothing is converted on the
PC and no file is mirrored.  The FP8 n-gram table and the BF16 embedding are the same file in both: a second install
links them.  The MTP draft layer is setup's own, as for every model.  The engine gets the fork's model flags
(--native and --native-dense-gguf on the dense GGUF, --ple-gguf, --embd-gguf); the rest of setup's choices (context,
KV, the low-RAM mode on the pack's experts.bin) are the usual ones: 96 GB keeps every expert in RAM, 64 GB with a
32 GB card the resident low-RAM mode.  NVIDIA only: on AMD setup stops and names UD-IQ4_XS (the HIP engine has no
NVFP4 kernels).  Images: off for these packs.  A ready-made engine older than NVFP4_ENGINE (0.1.47, to be set at
release) is replaced by one compiled from this source.

Tests: tools/test_setup_nvfp4.py (fetch from a file:// mirror with SHA-256 checks, the shared file linked, a wrong
file refused, the config name read back, mocked installs on 128/96/64 GB, AMD refused); test_setup_pins covers the
new table.  Every tools/test_setup_*.py passes.

Co-authored-by: sergiywith <[email protected]>
2026-10-12 20:53:31 +02:00
Niko1221 0d3553fc8f Quant yardstick: tools/quant_quality.py (teacher-forced top-1 agreement and KL against Q8_0 on a fixed 5x1000-token set) and tools/quant_formats.py (per-role tensor formats from GGUF headers, local or HF range requests) 2026-10-12 20:48:37 +02:00
Niko1221 5c6bdf6175 NVFP4 port: the SYCL tree takes the new defaulted args (no NVFP4 there); the 12x -> 12xa rewrite works on CMake < 3.29
- sycl: iq_dequant_f16 / iq_dequant_gu_f16 / moe_combine accept the NVFP4 scale arguments the shared headers now
  declare (refused if ever given: the SYCL port has no NVFP4 experts), and the no-MMQ stubs match the new mmq
  signatures (the calls were ambiguous with the defaulted ones).
- CMake: list(TRANSFORM ... REPLACE) with \2 of an optional group failed on CMake 3.28 for a plain "120"; two
  rewrites, one per form.
2026-10-12 20:43:44 +02:00
Niko1221 3a020d58c7 Qwen3.6 T1: strata-q36ref, the reference path, and its llama.cpp parity tools
- src/qwen35/q36ref.cu (target strata-q36ref, CUDA and HIP): Qwen3.6-35B-A3B on one GPU, the whole
  model resident, one token at a time with fp32 activations: plain pre-norm residual, GDN with the
  SiLU output gate (32 V heads over 16 K heads, conv 8192), dense gated attention (16 Q / 2 KV x 256,
  NEOX RoPE over 64 dims), softmax top-8 router over 256 experts with the sigmoid shared-expert gate,
  every weight read in its GGUF form (F32/F16/BF16/Q8_0/Q4_K/Q5_K/Q6_K). --act q8 (default) rounds the
  activations of quantized matmuls as llama.cpp's GPU path does; --force / --dump-full-logits for
  teacher-forced KL; --dump-layers / --dump-logits for parity.
- tools/qwen35/: q36_llama_dump.cpp (llama.cpp's side through the eval callback), parity.py
  (greedy answers + per-layer error), compare_dumps.py (two runs, e.g. the GPU-vs-CPU floor), kl.py
  (teacher-forced KL against llama.cpp HIP with the CPU backend as the floor), parity_prompts.json.
2026-10-12 20:41:17 +02:00
Niko1221 e000fe5d9c Expert arena: say at once that huge pages were skipped (#2126) 2026-10-12 20:37:22 +02:00
Niko1221 6b395578a2 Expert arena: MADV_HUGEPAGE is not requested when free 2 MiB blocks cover less than half of it (direct compaction stalled a 62 GB box's load at 100% system time); STRATA_ARENA_THP=1 asks anyway (#2126) 2026-10-12 20:37:21 +02:00
Niko1221 8e2c57be78 NVFP4 port: the SYCL build compiles the NVFP4 CPU rows (native_expert.cpp calls them); mmq_nvfp4_parity gets -mf16c on GCC/Clang
The fork was built with MSVC only: on Linux its parity tool's F16C rounding helper failed to inline without the
target flag, and the SYCL tree's own list of CPU expert sources lacked nvfp4_avx512.cpp / nvfp4_avx2.cpp.
2026-10-12 20:35:14 +02:00
Niko1221andsergiywith 55e2072c99 serve: the route tail skip is not a default with NVFP4 experts (STRATA_ROUTE_TAIL_SKIP=7 opts in)
Port of sergqwer/strata-nvfp4 c83625c3 (the rule for NVFP4 experts in any layer) onto 0.1.43's default, with its
#1884 nearly-all-cached exception kept.  On the fork's qwen-nvfp4-gptq pack the skip moved the answers by
teacher-forced KL 0.035 against it off, about the GPTQ quantization's own error against an all-Q8_0 reference
(0.038), for +11.5% decode (RTX 5090, the fork's measurement).  Packs without NVFP4 experts keep the default
unchanged.  The final default is the owner's, from our own 3060 A/B and KL against Q8_0.

Co-authored-by: sergiywith <[email protected]>
2026-10-12 20:35:14 +02:00
sergiywith b766cde631 cpu: NVFP4 expert rows in 256-bit lanes for CPUs without AVX-512
Without AVX-512 (Zen 2/3, Intel 12th-14th gen) the pool's NVFP4 rows went
through ggml-cpu's vec_dot, which re-decodes a row's blocks for every token of
a window.  nvfp4_256_* decode a 64-value block once and apply it to up to 8
tokens.  Each token's arithmetic is ggml_vec_dot_nvfp4_q8_0's AVX2 path step
for step (vpsignb, vpmaddubsw + vpmaddwd, the same FMA order, hsum_float_8),
so every row is bit-equal to it: an AVX2 CPU computes what it did before.

nvfp4_avx512_parity checks both kernels, whichever this CPU runs (the AVX2
rows must also be bit-equal; 0 differing rows of 1..9 tokens against the
portable and the native ggml-cpu builds), and is now a ctest
(nvfp4_cpu_parity).  One expert's gate+up, Zen 5 core: 1.12x ggml at one
token, 2.00x at 4, 2.35x at 8; under DRAM load (--bw) with 6 threads 47.3
against 24.0 GB/s at 4 tokens, 26.6 against 12.0 at 8.  Chat decode on the
RTX 5090 with STRATA_FORCE_AVX2=1 and 6 pool workers: 16.5-16.9 ms a round
against 17.3-17.5 (pool 6.6-6.8 against 7.2-7.3 ms).

STRATA_NO_NVFP4_256=1 falls back to ggml-cpu.

(cherry picked from commit 0c859e15f89054a376db7458bfb298e3e6c81f22 of sergqwer/strata-nvfp4)
2026-10-12 20:30:13 +02:00
sergiywith f72f203d91 prefill: the CPU rows' row_sd ones sized on their own - 0.1.42 takes the rows buffer once at its largest, which left them unallocated
Upstream's 04dbe561 allocates cpu_rows at its largest before the walk; the fork's allocation of cpu_ones (row_sd 1 for
the CPU's rows on an NVFP4 pack, b59f5dda) sat inside the reallocation that then never ran, so a staged chunk with the
CPU share copied from a null host pointer: "prefill moe_combine: invalid argument" (2K prompt, qwen-nvfp4-gptq).

(cherry picked from commit 52b272ebb92c9ef49029314da1a695d38f570d88 of sergqwer/strata-nvfp4)
2026-10-12 20:30:13 +02:00
sergiywith 77c6991d18 prefill: the CPU share on NVFP4 experts - the CPU's rows take row_sd 1 (upstream's CPU share is the base)
Was 640df4f0, the fork's CPU share.  Upstream 0.1.40.2 took it (978d3558, with 667f2eca's blobs taken on the
prefill thread and 4322e241's STRATA_DBG_NAN rows; opt-in there), so the port keeps upstream's code and adds only
what an NVFP4 pack needs: the CPU's rows carry s_down already, so their row_sd entries are 1 (cpu_ones, pinned
beside cpu_rows).

Port note: the release loop keeps 0.1.43's steps_up / tok_up / up_ring releases.

(cherry picked from commit a0c2ca233ac8fe8ec9d342bd319c93404791fa15 of sergqwer/strata-nvfp4)
2026-10-12 20:30:13 +02:00
sergiywith f235e394c1 Mixed expert formats: an NVFP4 projection beside another format in the same expert (e.g. NVFP4 gate/up + Q8_0 down)
- The blob keeps its {s_gate, s_up, s_down, 0} tail whenever any projection is NVFP4; the other projection's scale is
  exactly 1.0, and every path (GPU decode, CPU pool, prompt path) already multiplies by the tail, so 1.0 is exact.
  native_expert_layout (GPU) and native_fmt (CPU) no longer require NVFP4 in all three projections.
- tools/iq_pack.py writes such blobs (1.0 for a non-NVFP4 projection, .scale tensors only for the NVFP4 ones).
- native_expert_parity: the synthetic NVFP4 blob gets its tail (it was left zero, so every NVFP4 synthetic expert
  computed 0 and failed), non-unit for NVFP4, 1.0 for the other projection, and the reference applies it.
  nvfp4/q8_0, q8_0/nvfp4, nvfp4/nvfp4 and q8_0/q8_0 pass: CPU and GPU within 1e-7, ~1.2% from the float reference
  (activation rounding).

(cherry picked from commit 473c0959141b36d28e7a2eb16005f8c8a4e3c8d0 of sergqwer/strata-nvfp4)
2026-10-12 20:29:54 +02:00
sergiywith 37216b6e52 Prompt path: the MMQ group gather (upstream since 0.1.38) carries NVFP4 tails and zeroes the MMQ tail
0.1.38 took the fork's group gather without the NVFP4 parts: each expert's 16-byte scale tail now goes to its
group slot and the group's last flush zeroes the MMQ tail, in the same launch (as #353 on 0.1.38).

(cherry picked from commit 8f2193f028940b9a6d270203b640562435937a7a of sergqwer/strata-nvfp4)
2026-10-12 20:29:54 +02:00
sergiywith e51b4379dc NVFP4 routed experts: converter, pack, decode, prompt path, AVX-512 CPU rows
A ModelOpt NVFP4 checkpoint of Qwen3.8-Flash-Next (its routed experts NVFP4, group 16) runs on the native path.
Rebased on 0.1.31: NVFP4 is type 40 in the per-role format lists (STRATA_GU_FMTS / STRATA_D_FMTS / STRATA_MMVQ_FMTS)
beside Q4_K/Q5_K/Q5_1/Q8_0, and 0.1.31's dq_q8_0 serves both.

- tools/nvfp4_convert.py: llama.cpp's converter with the engine's type policy (BF16 control projections, Q8_0 native
  tensors and token embedding, no PLE table); tools/nvfp4_verify.py / nvfp4_verify_gguf.py prove the experts bit for
  bit.
- tools/iq_pack.py: expert_layout() gives NVFP4 blobs a 16-byte tail {s_gate, s_up, s_down, 0} (the per-expert
  weight_scale_2), so every consumer that moves whole blobs moves the scales; an NVFP4 pack always writes experts.bin
  (the GGUF has no tails, so the in-place GGUF read does not apply).
- decode: NVFP4 in the native expert kernels (q8_1, UE4M3/E2M1 decoded in software, bit-identical to ggml); each scale
  on its own projection's FP32 output (folding s_down into up put the hidden into FP16's subnormals).
- CPU pool: AVX-512 NVFP4 rows (1.8-3.7x ggml-cpu's), ggml-cpu elsewhere.
- prompt path: llama.cpp MMQ - W4A8 by default (int8 MMQ with Blackwell's FP4 MMA hidden, mmq_nvfp4_w4a8.cu), W4A4
  and FP16 through STRATA_PREFILL_NVFP4; the scales applied where they are read.
- CMake: the architectures stay as given. Only mmq_nvfp4_w4a4.cu (the FP4 x FP4 kernel and its FP4 activation
  quantizer, symbols renamed) is built with 12x -> 12xa in strata_mmq_w4a4; without its image W4A4 falls back to W4A8.
  strata.exe: 55 sm_120 cubins + 1 sm_120a.
- 0.25 of the missed experts over PCIe for NVFP4 packs (their blobs are 2.76 MB).
- Parity tools (mmq_nvfp4_parity, nvfp4_avx512_parity, nvfp4_expert_gpu_parity); HIP guards; docs/NVFP4.md.

Checked on this rebase (RTX 5090): the parity tools as before (decode experts 1.09-1.16%, MMQ W4A8 0.53%, W4A4 8.6%
- the same numbers as the 120a-everywhere build); IQ2_XS 128 greedy tokens identical to 0.1.31 main; the NVFP4 pack
against the 0.1.29-based build at the same expert-cache size: identical tokens on a 24-token and an 8K prompt, in W4A8
and in W4A4 (prefill 4,231 / 4,583 tok/s on 8K).

Port note: the PCIe share base for NVFP4 packs (0.25) sits beside 0.1.43's PcieAuto start values.

(cherry picked from commit 1f8a12858cc4ed80a977767afef6b6c3a754ac63 of sergqwer/strata-nvfp4)
2026-10-12 20:29:45 +02:00
Niko1221 0601072dfc test_setup_strix_halo: gfx1103 is in the gfx11 WMMA list since the 780M change 2026-10-12 20:19:49 +02:00
Niko1221 a5b3b6c27b Image generation: Qwen-Image-2.1-Turbo through stable-diffusion.cpp (POST /v1/images/generations)
- serve/images.py: OpenAI's images API (prompt, size, n, response_format b64_json|url) plus seed and steps; one
  sd-cli process per request with the model card's arguments (8 steps, CFG 1, euler, Turbo's sigmas,
  --diffusion-fa), the PNGs returned unchanged.  The job takes the service's turn (one job at a time on the GPU)
  and the config's gpu_policy says what the text engine does meanwhile: keep, shrink (--vram-elastic), unload
  (its conversation saved to a session file and restored after the next start), auto.
- setup.py --images qwen-image (--image-quant, --image-backend, --image-gpu-policy): sd.cpp 2988060 + its ggml
  as zips, built for CUDA / HIP / Vulkan (LunarG SDK into third_party when the PC has no Vulkan dev files) / CPU;
  DiT quant by GPU memory (Q4_K_M <= 8 GB, Q6_K <= 12, Q8_0 above), VAE and the Qwen3-VL-8B UD-Q4_K_XL text
  encoder at pinned revisions, each checked against the Hub's SHA-256.  tools/image/install.py adds it to an
  existing install.
- Web page: an Images tab when the server has image generation.
- docs/IMAGES.md; tests: serve/test_images.py, tools/test_qimage.py.
2026-10-12 20:18:33 +02:00
Niko1221 852e0657c6 setup on macOS: rebuild the engine when its sources change (a digest of the source files, not the git commit: an unzipped copy has none); the Mac's own words for the sizes 2026-10-12 20:06:02 +02:00
Niko1221 ceab2ce2b4 docs/MAC.md: Strata on a Mac - what it needs, memory per pack, setup, the measured numbers (M5 Max resident, M4 32 GB streaming), quality, limits, switches 2026-10-12 20:03:23 +02:00
Niko1221 93d7ceb91d setup on macOS: IQ3_S offered (recommended from 110 GB), streaming Macs configured without MTP
IQ3_S runs on Metal since mac-kern's IQ4_XS kernels: recommended from 110 GB (Apple's 128 GB Macs), IQ3_XXS where it
fits resident, then Q2_0. A pack that does not fit Metal's share of the memory streams its experts from the SSD (the
engine decides, f-mac-ssd): setup says so with the M4 numbers, writes --suffix-draft 0 and no --mtp (one token per
step, measured faster there) and does not fetch the draft layer.
2026-10-12 20:02:39 +02:00
Niko1221 ff0546db26 metal: serve - a new conversation resets the decode window; conversation checkpoints hold the window's state
session_zero and conversation_checkpoint_save/restore go to the Metal window under STRATA_GPU_METAL (the running
state lives there, not in the session's buffers): reset zeroes its GDN recurrences and conv histories, PLE history
and indexer tails (and the draft layer's); a checkpoint holds the same set, after the pending fold. Before this a
new request carried the last conversation's state (M5 Max, Q2_0: an unrelated request after a story answered
differently from a fresh engine; now the same), and on a streaming Mac the session buffers it zeroed or copied had
been given back (the engine stopped).

Checked on the M5 Max (resident) and the M4 (streaming): an unrelated request after another equals a fresh
engine's answer; a three-turn conversation reuses 35 / 73 of its tokens and its second answer equals a fresh
engine's. Shared files: session.cpp and conversation_state.cpp preprocess IDENTICAL and their objects are byte-
IDENTICAL on CUDA (P1004) and HIP (p620). Also: IQ4_XS's v2 prompt tile (_16_64_32_2), and a call stack when a
given-back allocation is touched.
2026-10-12 20:02:39 +02:00
Niko1221 14ba7a627a Model families: Qwen3.6-35B-A3B (qwen35moe) recognised, geometry and layout from the GGUF, no engine path yet
- ModelFamily from general.architecture (qwen4exp, qwen35moe); the architecture guard accepts both, the
  engine's guard (native dense/head, --native) refuses Qwen3.6 with "recognised, not run yet".
- ModelGeometry moved to core/geometry.hpp (+ family, shared-expert FFN, in-file MTP layers, vocab);
  geometry_from_gguf reads every shape key, checked against the compiled kernels (Flash-Next files give
  exactly the defaults; expert count and top-k stay run-time sizes).
- Per-family tensor layout tables (GDN / attention / nextn MTP block, hyper-connections or plain
  pre-norm, indexer or none) and a GGUF directory check; MXFP4 block size in the reader.
- setup: --family qwen36 downloads unsloth/Qwen3.6-35B-A3B-MTP-GGUF (pinned 5bc3e238, size + SHA-256)
  UD-Q4_K_XL / UD-Q6_K_XL / Q8_0 (+ mmproj) and says the engine does not run it yet; serve's default
  model name per family; strata_inspect recognises qwen35moe.
- Tests: model_family_test (synthetic, --real for headers), test_setup_qwen36, serve/test_family_name.
2026-10-12 19:58:28 +02:00
Niko1221 f5914a1818 metal: IQ3_XXS prompt decode probes (MM_PROBE 3: no table lookups, 4: computed signs) - the grid/sign threadgroup lookups are 2.5 of 6.6 ms a layer; computed signs are slower (7.0-7.6 -> 7.7-8.3) 2026-10-12 19:45:09 +02:00
Niko1221 b6a2dacd86 Merge f-mac-kern b6480437 (IQ4_XS expert kernels: IQ3_S on Metal) 2026-10-12 19:42:48 +02:00
Niko1221 93e582d087 Merge f-mac-ssd f0609f40 (expert streaming from the SSD for Macs whose memory cannot hold the pack)
Conflicts in metal/src/engine/window.cpp resolved: the stream's next-layer routing guess beside router_route
(opt-in); the cached window through the stream's segmented run, the kernel profile off while streaming; host reads of
the weights through W_ (the mapped file, or the dense weights in memory when streaming), also in prompt_chain.inc;
the drafter's prompt rows batched, from either prompt path. The prompt: resident -> prompt_chain.inc, streaming ->
the stream's prompt path (it streams each layer's experts for the chunk); STRATA_METAL_PROMPT=window: 8-row windows.

M5 Max: Q2_0 prompt1/story and IQ3_XXS prompt1 with MTP equal to the no-MTP references; the 1,662-token prompt
through prompt_chain.inc equal to before; Q2_0 streamed (STRATA_METAL_STREAM=1) equal to resident on prompt1. The
1,662-token prompt streamed (the stream's prompt kernels: chunks of 1024, flash QSA) first differs at token 21 of 60
from the resident prompt_chain.inc path (different kernels and chunking).
2026-10-12 19:42:48 +02:00
Niko1221 b6480437d2 metal: IQ4_XS routed-expert gate/up for decode (moe_gu_23_*) and prompt (mm_gu3 W4<IQ4_XS>, mm_gu2 TK >= 32 tiles) - the IQ3_S pack's one IQ4_XS layer; parity on the M5 and the M4 2026-10-12 19:41:13 +02:00
Niko1221 5565a347de metal: M4 window GEMV probes - gemv_x (x staged in threadgroup memory), GEMV_PROBE (function constant 9)
- gemv.metal: gemv_x<F,LPR,NR,SG> = gemv (KS 1) with the window's prepared x staged in threadgroup memory (bitwise
  the same arithmetic); GEMV_PROBE 1/2/3 bench probes (skip later tokens' x loads / combine; rows-outer order)
- units.metal: X<F>::load templated on the pointer's address space (device or threadgroup), same arithmetic
- gemv_parity: GEMV_X=1 sweeps gemvx_*, GEMV_PROBE passes the probe
- M4 finding: a window token's cost is the fp32 combine (1 FMA a weight a token), at ~0.85 T FMA/s (near the M4's
  peak); the x loads are ~6 us of the ~30-60 us a token, and staging x in threadgroup memory does not help
2026-10-12 19:38:29 +02:00
Niko1221 37c3b38c26 setup on macOS: --mmap-experts in the config (what the Metal engine does; the server's start text then says the experts are mapped, not loaded) 2026-10-12 19:38:09 +02:00
Niko1221 a9edff4f80 setup on macOS: metal/setup_mac.py (Apple Silicon, the Metal engine built from source)
setup.py hands a darwin run to metal/setup_mac.py (nothing else changes on a PC); setup.sh fetches a pinned
standalone Python 3.12 into .python on a Mac without Python 3.10+ (no sudo, no Homebrew). The Mac setup checks the
chip, unified memory, macOS (15+) and the Command Line Tools; recommends the pack by memory (IQ3_XXS from ~96 GB,
Q2_0 below; IQ3_S only with --model: the Metal expert kernels lack its IQ4_XS experts); builds the engine with pip's
cmake/ninja, Apple's metal-cpp (pinned SHA-256) and llama.cpp's ggml at the pinned commit; prepares the native pack
and the MTP draft layer; writes strata-<model>.json (backend metal, --spec 2) and run-<model>.sh; starts
serve/server.py. tools/test_setup_mac.py: 10 tests with sysctl/sw_vers/xcode-select mocked.

On the M5 Max (m1002): ./setup.sh --model Q2_0 --gguf-dir ... --yes from a fresh copy: Python fetched, engine built,
pack and config written in 29 s (models already downloaded); the server answered /v1/models, the chat page,
streamed /v1/chat/completions and /v1/messages (Q2_0, 112-121 tok/s). python -m pytest serve -q on the Mac: 705
passed, 11 skipped.
2026-10-12 19:37:54 +02:00
Niko1221 a5d17ab0e3 metal: the draft vocabulary beside the draft layer (<mtp>/rt/draft_vocab.bin, setup's --draft-vocab) 2026-10-12 19:37:54 +02:00
Niko1221 b66c61984b metal: serve ends cleanly after QUIT (Metal-only _Exit)
The stdin reader (a detached thread) can take in_mu after the serve scope destroyed it, when the server closes the
pipe after QUIT; on macOS that lock throws and the engine ended with an abort. Inside #if STRATA_GPU_METAL: flush
and _Exit(0) after the QUIT profile save. HIP (p620): generate.cpp preprocesses IDENTICAL and its object is
byte-IDENTICAL against the gated tree (P2003 was unreachable for the CUDA compare; the change is preprocessor-only).
2026-10-12 19:37:53 +02:00
Niko1221 818845ea10 Merge commit 'a0116b27' into f-144-int 2026-10-12 19:34:25 +02:00
Niko1221 e30cffb8f8 metal: stubs.cpp regenerated without cvec() (now in missing.cpp) 2026-10-12 19:15:57 +02:00
Niko1221 f33434836a metal: serve - the empty control vector on Metal; a call stack on terminate
The serve loop asks cvec() every request: Metal returns the empty one (no --control-vector-scaled on Metal yet)
instead of the stub that stopped the engine; stubs.cpp regenerated without it. An uncaught exception now prints its
call stack (backtrace_symbols_fd) before the abort.
2026-10-12 19:15:16 +02:00
Niko1221 389c24eebd metal: router_route (mac-kern 3f9ca996) in the window, opt-in STRATA_METAL_ROUTE=1
The BF16 router GEMV and moe_route as one dispatch beside the shared expert's GEMVs. Greedy output unchanged (Q2_0,
IQ3_XXS, both prompts, on and off). Not the default: in this window the router GEMV already runs beside the
shared GEMVs, and the T2 window measured 13.035 ms GPU with it vs 13.027 without (the fused kernel alone 0.615 ms
vs 0.566 for the two); end to end, --spec 2, 3 interleaved pairs, off -> on: Q2_0 113.61 -> 113.28 / 107.50 ->
106.97, IQ3_XXS 103.51 -> 103.07 / 96.19 -> 96.02 tok/s.
2026-10-12 19:15:11 +02:00
Niko1221 be49de3e34 Merge f-mac-kern 3f9ca996 (router_route, gemv_cat, gr_norm_comb, Q3_K packed loads) 2026-10-12 18:56:48 +02:00
Niko1221 3f9ca99632 metal: decode T=2 items - router GEMV + moe_route in one dispatch (router_route), gemv_cat, gr_norm_comb, Q3_K packed loads
- router.metal (new): router_route<F> - the router GEMV rows as gemv's (LPR 32, KS 1) with 512-thread threadgroups,
  the last threadgroup to finish (device counter, device-scope fences) runs moe_route's body and resets the counter;
  logits / ids / w / plan bitwise gemv + moe_route.  token_chain default (CHAIN_ROUTE=0: separate): T2 13.08 -> 12.93 ms
- gemv.metal gemv_cat: two same-format matrices over one x in one dispatch (ssm_alpha + ssm_beta), bitwise; no gain
  in graph mode (CHAIN_ALBE opt-in)
- gr2.metal gr_norm_comb: the MoE combine folded into gr_norm, bitwise; +0.6 ms at T2 (CHAIN_COMB opt-in, not kept)
- units.metal Dec<Q3_K>: qs / hmask as packed_ushort4 loads (same values): T2 2560->10240 33.8 -> 32.2 us
- tests/fuse_parity: the three fusions bitwise for 1..4 tokens
2026-10-12 18:51:19 +02:00
Niko1221 38cb85a8c3 metal: gdn_step's partial commit in the window (opt-in STRATA_METAL_PARTIAL=1)
With it a verify window of T > 1 commits its first row in place (commit = 0x100 | 1, mac-kern 56526aed): commit(1)
needs no fold, commit(n) folds rows 1..n-1; a prompt window commits all its rows in the step. Greedy output equal
to no MTP (Q2_0, IQ3_XXS, both prompts, --spec 2 and 4, the window prompt path).

Not the default: M5 Max, --spec 2, 3 interleaved pairs, off -> on: Q2_0 113.35 -> 113.24 / 106.98 -> 106.75,
IQ3_XXS 103.53 -> 102.71 / 96.18 -> 95.88 tok/s. The in-place state write (36 x 3 MB) costs every window; the
fold it saves is only the rounds with no accepted draft (24-36% at --spec 2).
2026-10-12 18:45:58 +02:00
Niko1221 3bbfa9e6e1 Merge f-mac-kern 56526aed (gdn_step partial commit) 2026-10-12 18:40:05 +02:00
Niko1221 56526aedc2 metal: gdn_step partial commit (commit = 0x100 | k): the state and conv history after the first k window tokens
- gdn.metal: with commit = 0x100 | k (1 <= k <= GDN_NTOK) the state is written in place and the conv history to
  hist_out right after token k - 1, while every token's y is still computed; commit = 1 / 0 unchanged (the full
  commit now writes at the last token inside the loop - same values, same speed)
- gdn_parity: the partial commits are bitwise a k-token commit's state and history, and y bitwise the verify window's
  (k = 1 .. ntok - 1, ntok 2..4); gr_parity GR_DOWN=SG_KS picks gr_down's variant (sweep: 10.1-11.8 us at T 2, none
  better than 8_8 by more than 7%)
2026-10-12 18:39:03 +02:00
Niko1221 39acb889cd metal: test tooling - STRATA_METAL_KPROF=T (the GPU time of each kernel class of a T-token window, alone)
Once, after the 20th cached window of T tokens with the head: the window's dispatches grouped by pipeline and weight
role, each group run by itself (7 runs, median). Profiling only: the reruns move the state, so the text after it
is not meant to be read.
2026-10-12 18:31:58 +02:00
Niko1221 4eca56f46a metal: --spec defaults to 2 on Metal builds (an explicit --spec N wins)
Owner decision. Only the Options default under STRATA_GPU_METAL; CUDA/HIP/SYCL keep 0: on P2003 the CUDA
generate.cpp translation unit preprocesses IDENTICAL and its object is byte-IDENTICAL.

M5 Max, MTP, 200 tokens, greedy output equal to no MTP in every run: Q2_0 114.0 / 107.9 tok/s (prompt1 / story,
--suffix-draft 0; was 112.0 / 86.2 at --spec 4), 119.5 / 108.2 with the default suffix drafter; IQ3_XXS 103.5 /
96.9, 108.9 / 97.0 (was 110.2 / 87.4).
2026-10-12 18:30:51 +02:00
Niko1221 f0609f40d4 Merge f-mac-kern 272bbbbd 2026-10-12 18:26:09 +02:00
Niko1221 54405425e0 metal: token_chain's --fuse in the decode window (GDN y prep in the step, no QSA selection within the budget)
The GDN step writes the prepared y planes (no xprep dispatch per GDN layer); a window whose context is within the
2,051-cell budget skips the QSA score and top-k (qids_ holds the identity, restored after a selecting window; the
cached encode is keyed by it). STRATA_METAL_FUSE=0: off.

M5 Max, Q2_0, 200 tokens, 3 interleaved pairs: no MTP 85.1 -> 87.4 (prompt1) and 85.2 -> 87.3 (story); MTP 110.7 ->
112.0 and 85.3 -> 86.2 tok/s; tokens identical on and off and equal to the no-MTP reference.
2026-10-12 18:24:20 +02:00
Niko1221 a26122339f Merge f-mac-kern 272bbbbd (M4 QSA prompt attention, mm_gu3 paired-run decode, chunked GDN scan) 2026-10-12 18:20:49 +02:00
Niko1221 f159858146 metal: the prompt through mac-kern's prompt kernels (prefill hook)
prompt_chain.inc: tests/prompt_chain.cpp's chain (6f6c555b) into the decode window's own state - fp16 GEMMs on the
matrix units, grouped expert GEMMs, the GDN prompt scan, QSA K/V and indexer keys into the window's caches - plus
the PLE block after layer 0, every row's final residual for the drafter (its prompt rows in one command buffer),
the history parity, RoPE table and pending fold. Chunks of 2048 rows (STRATA_METAL_PROMPT_CHUNK); COOP_K4 picks
the register-operand kernels on the M5 and the older ones elsewhere. The 8-row-window path stays as the fallback
(STRATA_METAL_PROMPT=window, and prompts under STRATA_METAL_PROMPT_MIN = 32 rows).

M5 Max: Q2_0 4K/32K prompts 1,434/1,462 tok/s (window path 190/174), IQ3_XXS 1,260/1,240. Teacher-forced KL of
the chain's prompt vs the window path's, 28 prompts: 0.0004 (top-1 99.4-99.6%); needles 4K-32K x 10/50/90: the
same answers as the window path (12/12, both models).
2026-10-12 18:20:40 +02:00
Niko1221 b174c71f86 metal stream: a prompt's buffers live in a region of the expert arena (held only while a prompt runs; the arena is anonymous memory wrapped no-copy); 7 GB kept for macOS 2026-10-12 18:04:19 +02:00
Niko1221 8b91c64483 metal stream: size classes merged (<= 15% slot waste; 3 for the IQ3_XXS mix), each holding a prompt layer; misses never evict a wanted slot (pinned first); read-ahead depth / LRU place by expert size (measured) 2026-10-12 17:46:30 +02:00
Niko1221 272bbbbd4a metal: chunked GDN scan (WY form, chunks of 32) - correct, slower in scalar fp32 than gdn_scan_r; kept, not used
- gdn_prompt.metal: gdn_cprep (per head and chunk: gates in log space, A, forward substitution for U and W),
  gdn_cstate (per head and 32 state columns: the chunks in order), gdn_cout (the intra-chunk output)
- gdn_prompt_parity: CB -100 = the chunked form; y 3e-7, state 2e-7 vs the decode step; 7.4 ms vs 1.9 ms at T 2048
  (cprep 1.5, cstate 4.7, cout 1.4) - it needs the matrix units to pay off
2026-10-12 17:32:53 +02:00
Niko1221 072c19f60f metal: mm_gu3 paired-run decode without the runtime layout check (Q2_0 gate/up 4.9 -> 3.5 ms a layer); SLC prefetch probe
- moe_mm.metal: the paired-run (k, k + 16) decode is taken on the COOP_K4 layout unconditionally (coop_probe now
  checks that pairing on both tiles used); the runtime per-element check cost ~30%.  The MMA-only ceiling of mm_gu3
  (MM_PROBE=1: no decode) is 2.45 ms a layer = 55 TFLOP/s useful, so the decode is what is left.  Measured and not
  kept: fp16-math Q2_0 decode (slower), a W8<Q2_0> sharing the block scale (slower than two W4)
- glue.metal slc_touch + token_chain CHAIN_PREFETCH: the next HC read's weights touched concurrently with the first
  projection GEMV after a read; measured +0.83 ms a token (10.46 -> 11.29, 3 of 3 interleaved) - not kept, opt-in
2026-10-12 17:23:39 +02:00
Niko1221 41ffe3eef6 metal: prompts on the matrix units in the engine (prompt_chain's chunk on the window's state), SSD-streamed
A prompt of 64+ tokens runs in chunks (STRATA_METAL_PROMPT_CHUNK, default 1024 streaming / 2048 resident) through
mac-kern's prompt kernels (fp16 GEMMs, the GDN scan, QSA over the chunk, grouped expert GEMMs v2 on the M4, v3 on the
M5), the PLE block for T rows, on the decode window's GDN / K/V / PLE state; the draft layer's prompt K/V read the
chunk's final rows.  Streaming: the expert GEMMs address arena slots (function constant 20 in moe_mm.metal too), a
chunk's handshake puts the layer's experts in place and streams the next layer's non-resident experts into the LRU
tail behind the compute (the decode-hot experts stay); slots in use by the running layer are pinned.  The host
code's own device weight copies are given back on a streaming Mac (purge_device_allocations).
STRATA_METAL_PROMPT_GEMM=0 / _MIN, STRATA_METAL_PROMPT_EXPERTS=v2|v3, STRATA_METAL_PROMPT_QSA=v1|flash.
2026-10-12 17:18:46 +02:00
Niko1221 d9f4b16006 Merge f-mac-kern dbc2e2fd (M4 QSA prompt fixes) 2026-10-12 16:58:49 +02:00
Niko1221 e00b29862b metal stream: read-ahead two layers ahead (STRATA_METAL_PREFETCH2), most likely first; leave-one-class-out timing (STRATA_METAL_SKIP), window timings per T, STRATA_METAL_GEMV_NR, runtime allocation log 2026-10-12 16:58:49 +02:00
Niko1221 dbc2e2fd8f metal: QSA prompt attention correct on the M4 (K = 32 runs, threadgroup-staged store, fresh per-chunk accumulator)
- qsa_attn_mm: the P V matmul in K = 32 cell runs (K = 64 runs computed garbage on the M4), the output through
  threadgroup memory (a direct matmul2d store into 258-float rows lost the threadgroup on the M4); also faster on the
  M5 (8.1 -> 7.4 ms at pos 0)
- qsa_flash: the same K = 32 runs (was a hang on the M4) and a fresh per-chunk accumulator folded into the running
  one (on the M4 a run does not accumulate into elements rescaled in place); now about equal to attn_mm + merge on
  the M5 (9.4 vs 9.5 ms at pos 0, 20.1 vs 20.3 at 30000) and the prompt_chain default there (PC_QSA=flash|mm|v1);
  the M4 defaults to v1 attention (faster there) with the GEMM score
- qsa_attn_mm bisection probe (function constant 8, QSA_STOP), qsa_prompt_parity QSA_DEBUG / pipeline report;
  coop_layout prints the destination layout too
2026-10-12 16:57:55 +02:00
Niko1221 6294f3468f Merge f-mac-compat 68387f1a 2026-10-12 16:51:03 +02:00
Niko1221 a6e55d062e metal: SSD-streaming expert arena for Macs whose memory cannot hold the model (the M4 32 GB)
The dense weights are read into one resident buffer; the routed experts live in an arena of slots filled from the
GGUF with F_NOCACHE preads (several pieces of an expert in flight); the window runs in 48 segments with a shared-event
handshake after each layer's router (mtl_rt run_graph_seg), where the host puts the missing experts in place (LRU)
and reads ahead the experts the next layer's router predicts from this layer's FFN input.  The expert kernels address
slots through smap (function constant 20); the resident path is unchanged.  STRATA_METAL_STREAM=auto|1|0.
2026-10-12 16:50:18 +02:00
Niko1221 68387f1a08 Merge f-mac-kern 6f6c555b (prompt experts / QSA on matrix-unit operands, COOP_K4) 2026-10-12 16:45:43 +02:00
Niko1221 6f6c555b17 metal: COOP_K4 (function constant 7) - per-element right-input decode for GPUs without the M5's layout (the M4)
- moe_mm.metal mm_gu3/mm_down3 and qsa_prompt.metal qsa_attn_mm/qsa_flash: with COOP_K4 = false every element is
  decoded on its own (the M4 holds runs of 2 n at one k); the experts are correct on the M4 that way
- tests/coop_probe.h: probes the layout (COOP_K4=0/1 overrides); moe_mm_parity, qsa_prompt_parity, prompt_chain
  set the constant; prompt_chain defaults to experts v2 and QSA v1 without it (faster / correct on the M4)
- qsa_prompt_parity: NaN-fills the partials before the matrix-unit run (a kernel that writes nothing shows),
  line-buffered output, QSA_NOFLASH
2026-10-12 16:36:08 +02:00
Niko1221 994ccdf451 metal: test tooling - STRATA_METAL_LOGPOS (teacher-forced rows for the KL harness) and STRATA_METAL_PROFILE
STRATA_METAL_LOGPOS=<path>: per decode position of one-token windows, the log-probability of the token that came
next (with --spec-follow, the forced text) and the top 64, in phaseA-tests cuda_kl.py's row format.
STRATA_METAL_PROFILE=1: per window size, host time before the GPU, run_graph wall vs GPU time, commit and draft
round times, printed when the window goes away. Both off unless set.
2026-10-12 16:35:15 +02:00
Niko1221 1f4b6b3615 metal: the GDN fold of a commit runs at the start of the next window
commit(n) after a window of several tokens no longer submits its own command buffer: the fold of the kept prefix is
encoded first in the next window's (cached per pending count). Nothing reads the GDN state in between.
STRATA_METAL_LAZY_FOLD=0 folds at commit as before.

M5 Max, Q2_0, MTP --spec 4, 200 tokens, 3 interleaved pairs: commit 0.87 -> 0.05 ms a round; prompt1 109.25 ->
110.21 tok/s, story 83.99 -> 85.14 (medians); greedy output equal to no MTP, lazy and not.
2026-10-12 16:35:03 +02:00
Niko1221 a0116b2782 Layer split + --batch: the default pipeline wins over --batch-mtp (STRATA_BATCH_MTP_SERIAL=1 or --batch-groups 1 keeps the serial MTP windows) (#2038)
0.1.42 let --batch-mtp run on a layer split and made the --batch-groups auto default give way to it: one serial group. On
2x RTX A4000 (IQ3_XXS, 3 pairs) four streams aggregated 62.2 tok/s against 92.7 on 0.1.41's pipelined groups (-33%, 0/3 pairs
faster), two streams 55.7 against 71.4 (-22%). With --batch-groups 2 (which turns --batch-mtp off) 88.3 and 71.1. The start-up
line says why --batch-mtp is off and how to keep it. --adapt-async 1 still gives one group. Solo decode and the answers are
untouched.
2026-10-12 16:00:57 +02:00
Niko1221 9bdfcd7166 metal: QSA score as an fp32 GEMM, paired-run i-quant decoders (W8), qsa_flash (measured slower), one gate/up tile
- qsa_prompt.metal: qsa_score_mm (fp32 MPP, relu-sum epilogue, the tail block scalar; 6.2 -> 1.1 ms at pos 30000,
  T 2048), qsa_flash (whole attention a threadgroup with an online softmax, fp16 out; correct but 27.6 ms vs 20.9 for
  attn_mm + merge - not used)
- moe_mm.metal: W8<F> decodes a lane's two runs of a 32-group with one read of the group's scale/sign fields;
  IQ3_XXS gate/up 9.3 -> 6.7 ms a layer
- prompt_chain: PC_QSA=mm uses qsa_score_mm too; gate/up 16,32,32,4 for every format; breakdown sees the far
  chunk's classes
2026-10-12 15:47:30 +02:00
Niko1221 026d7cd4a1 metal: the MTP draft round in one command buffer; catch-up embeddings fixed
msl/mtp_chain.metal: mtp_pick (the pick through the draft vocabulary and its softmax probability) and mtp_embed_F
(the picked token's embedding row through mac-kern's unit decoders), so every draft step of a round runs in one
command buffer. At load the GPU embedding is checked against the host's dequant (max |diff| 0 on Q3_K and IQ3_S;
any difference falls back to a command buffer per step). STRATA_METAL_MTP_CHAIN=0 keeps the per-step path.

Fix: with accepted drafts (a > 0) draft 0's embedding overwrote the catch-up's row 0 before the GPU read it; draft
0's row now goes to memb_ row a. The dense attention's cell list is per step.

M5 Max, 200 tokens, --spec 4, greedy output equal to no MTP in every run: Q2_0 109.5 / 84.4 tok/s (was 101.4 /
80.0), IQ3_XXS 107.9 / 86.0 (was 101.5 / 82.8).
2026-10-12 15:07:25 +02:00
Niko1221 8bf827b940 metal: the MTP draft layer's BF16 matrices in Q8_0 (STRATA_METAL_MTP_Q8=0 keeps BF16)
Drafts only; the verify window decides every token. M5 Max, 200 tokens, --spec 4: greedy output equal to the
no-MTP runs on Q2_0 and IQ3_XXS, both prompts. Q2_0 101.4 / 80.0 tok/s (was 101.2 / 79.4), IQ3_XXS 101.5 / 82.8
(was 96.4 / 81.2).
2026-10-12 15:01:21 +02:00
Niko1221 b507277646 metal: the MTP drafter, the prompt path through the window, the PLE block and the Q8_0 HC read on the GPU
- MTP (src/core/mtp.cpp hook under STRATA_GPU_METAL: load / bind / prefill / draft / draft_first): the draft layer
  of mtp.hpp on mac-kern's kernels, from the MTP GGUF (tools/mtp_pack.py, beside --mtp's rt dir; the F32 norms
  folded to 1 + w as tools/mtp_rt.py does): input branches (msl/mtp.metal: rms, strided xprep, stream combine),
  the three HC reads, K/V and indexer keys through qsa_prep into its own cache, dense attention over the last
  2,051 cells, the MoE (fused gate_up experts), the final mixer and the head over data/draft_vocab.bin's subset
  (STRATA_METAL_DRAFT_VOCAB=full: the whole head).  One command buffer per draft step (the next step's embedding
  is dequantized on the host).  Greedy output with MTP equals the output without it, token for token (Q2_0, two
  prompts, BF16 and Q8_0 HC); acceptance equals the HIP engine's on the same prompts (124 of 228; 104 of 291 vs 285).
- Prefill (src/prefill/prefill.cpp hook): init brings up the decode window, run reads the prompt through windows
  of 8 rows (no head) and feeds on_chunk (the MTP's K/V); draft_kv declines.  Prompt windows of 8 rows give the
  same tokens as one-token windows.  generate.cpp's own Metal prompt loop is gone again (--prefill auto as on the
  PC).  One window per process (the prompt path may bring it up before the Verifier).
- msl/ple.metal: the PLE block's norms, gate, conv and residual update on the GPU (same tokens as the CPU version).
- The HC read in Q8_0 by default (the owner's decision; STRATA_HC_Q8=0: BF16): the down / up / inject matrices
  quantized at load (290 matrices, 680 MB, 70 ms), read by gr_down_q8 / gr_up_q8.
- The encoded window is cached per (T, head, history parity) and only its positions are patched per window.
- f-mac-kern merged up to 4775d233.

M5 Max, Q2_0, greedy, 200 tokens: Q8_0 HC 85 tok/s without MTP; with MTP 101 tok/s (acceptance 0.54) and 79 tok/s
(acceptance 0.36, a prompt the drafter does badly on everywhere: HIP 71 tok/s there).
2026-10-12 14:55:46 +02:00
Niko1221 aea8d64bcc metal: prompt experts and QSA attention on matrix-unit register operands (v3), prompt_chain defaults to them
- moe_mm.metal: mm_gu3 / mm_down3 - weights decoded per lane straight into matmul2d's right-input cooperative tensor
  (single-simdgroup scope, no threadgroup staging or barriers in the K loop); W4<F> 4-value decoders for Q2_0,
  IQ3_XXS, IQ2_XXS, IQ2_XS, IQ2_S, IQ3_S, IQ4_NL (quant.metal's deq arithmetic)
- qsa_prompt.metal: qsa_attn_mm (S = Q K^T and P V as single-simdgroup matmuls, int8 K/V decoded into the right
  input; same partials as qsa_attn, so qsa_merge is unchanged), qsa_q_f16
- tests: qsa_prompt_parity, coop_layout (the cooperative tensors' element layout), moe_mm_parity --v3;
  prompt_chain PC_GU / PC_DN / PC_QSA switches (defaults v3 / v3 / mm)
2026-10-12 14:54:56 +02:00
Oleksandr Dovhoselets 85f839d80c hip: gfx103x creates the MTP drafter's side stream again (#2055)
#1240 stopped creating the drafter's branch stream and its two events on HIP: nothing uses them there, and an R9700
lost prompt speed with them. On gfx103x the stream's existence alone makes prompts read faster. RX 6800, Windows,
STRATA_HIP_PROMPT_F16=1, IQ3_XXS, two runs each: a 7.9K-token prompt at 798-808 tok/s with the stream and 584-596
without, a 30.4K-token one at 855-863 against 553-600; all of it is in the prompt path's "wait copy" phase. The two
events change nothing, so they stay out.

gfx103x creates the stream; other HIP cards do not, as before. STRATA_MTP_SIDE_STREAM=1 / =0 forces either. CUDA is
unchanged. Measured on one card under Windows only.
2026-10-12 14:34:18 +02:00
Niko1221 017a672ae1 bench: the server the bench starts runs without --expert-profile-save, so it never saves a profile learned from its short prompts over the user's (#2088) 2026-10-12 14:21:19 +02:00
Niko1221 682ae446a2 Merge f-mac-kern (Q8_0 HC default in the chain, prompt kernels) 2026-10-12 13:35:43 +02:00
Niko1221 4775d23372 metal: Q8_0 HC read the default in token_chain (STRATA_HC_Q8=0 = exact BF16); prompt chain, GDN prompt scan, fused HC prompt read
- token_chain: the HC read defaults to the Q8_0 copies through the 3-dispatch read (q8_3); STRATA_HC_Q8=0 selects
  the exact BF16 path (--hc still overrides)
- prompt.metal: route_topk, gr_epi_down/up, swiglu_h, moe_out, gr_norm_h (fp16 xn), gemm_hc_up (the HC up GEMM with
  the mixing epilogue; w_up rows interleaved by dq_f16's new perm field)
- gdn_prompt.metal: gdn_pre (parallel conv/norms/gates) -> gdn_scan_r (state scan, one barrier a token) -> gdn_post
- dense_mm.metal: DqArgs gains `perm` (4th field)
- moe_mm.metal: mm_scan zero-fills the unused simdgroup slots; indirect grids for gather/gate-up/down in meta;
  MM_PROBE bench constant (function constant 5: pipelines of mm_gu2 need it set)
- tests: prompt_chain (whole prompt in chunks, class breakdown), prompt_parity, gdn_prompt_parity; mm_bench tiles
2026-10-12 13:33:12 +02:00
Niko1221 cf00b0d031 metal: windows of several tokens (verify + commit of the kept prefix), experts read in place
- window.cpp: a window of 2..8 tokens runs the GDN steps without committing; the projections it feeds them are
  kept per layer, and commit(n) folds the kept prefix into the state (gdn_step over n tokens with commit, the same
  kernel: bitwise the window's trajectory) and turns the conv-history ping-pong.  K/V and indexer cells past the
  prefix are rewritten when those positions run again, before anything reads them; the PLE history keeps the n
  rows.  Checked: greedy output with suffix drafts (windows up to 4, 50 of 59 drafts kept) is the same token for
  token as one-token windows.
- generate.cpp (Metal only): --mmap-experts by default (the GGUF shards in place; the window reads them itself):
  the 31.6 GB expert arena is not loaded any more (RSS 74 -> 40 GB, the rest is the mapped shard)
- stubs.cpp regenerated (256 entry points left as stubs)

IQ3_XXS vs the CUDA engine (P1004, f-mac, same flags, --kv int8): 200 of 200 tokens equal on the capital
prompt, 124 on a story; following CUDA's tokens 3 of 200 picks differ, at margins 0.07-0.23 (near ties).
2026-10-12 13:25:08 +02:00
Niko1221 4468e0e62c Prompt loan: say when a chunk's loan is larger than the auto chunk's (#2090) 2026-10-12 13:20:17 +02:00
Niko1221 af6affb871 Prompt loan: a request's smaller chunk can need more slots than the auto chunk's region (a chunk at or below small_max gets 0.1.39's bigger ring), so the loan starts where the chunk needs it and a chunk that still cannot be lent is halved, instead of ending the engine (#2090) 2026-10-12 13:20:17 +02:00
Niko1221 a86aa74763 Tests: STRATA_TEST_PINNED_SHARE sets the pinned share the prompt ring rule reads (#2090 repro) 2026-10-12 13:20:17 +02:00
Niko1221 5d76056a4a metal: the engine runs on the M5 Max - the shared host loop over a Metal decode window (C3, first greedy answers)
The Metal build of `strata` is the shared host code (src/, unchanged except preprocessor-only Metal/arm64 branches)
over two Metal-only parts in metal/src/engine:
- window.cpp: the decode window, the one backend hook (Verifier::init/run/commit/copy_logits under
  STRATA_GPU_METAL).  It is mac-kern's chain on the model GGUF itself (shard 1 mmapped and wrapped no-copy, every
  weight read in place: no upload, no expert cache): HC reads, GDN/QSA mixers with their own state (recurrence,
  conv history ping-pong, int8 K/V, indexer), router + plan, routed and shared experts, combine, the final mixer,
  the LM head and the greedy pick, encoded as one dependency graph.  The embedding is dequantized on the CPU into
  the shared buffer; the PLE rows come from the shared table reader, their projections run on the GPU and the
  block's norms / gate / conv on the CPU between two command buffers.  One-token windows (--mtp-max-t 1) for now.
- host helpers copied from the CUDA kernel files the host code reads while it starts (format tables, sizes, path
  switches, the RoPE table/config); every other kernel entry point is a generated stub that stops with its name
  (stubs.cpp, tools/gen_stubs.py, from clang's AST of our headers).

Shared host code, all inside #if STRATA_GPU_METAL / __APPLE__ / __aarch64__ (the PC objects do not change):
- verify.hpp/.cpp: the hook (an opaque window pointer; init/run/commit/wait_commit/copy_logits hand over)
- generate.cpp: no x86 ISA refusal; a native pack without --prefill reads its prompt through one-token windows;
  the model shard handed to the window
- device.cu: the poison fill through the allocation's CPU view; pool_affinity_linux.hpp: a macOS branch (no
  pinning); expert_layout.cpp: no xgetbv asm on arm64
metal/include/metal_compat: cpuid.h (all zero on arm64), linux/mman.h (no hugetlb/THP: the fallback taken).

M5 Max, Q2_0, greedy, "What is the capital of France?": "The capital of France is Paris." - the same 36 first
tokens as the HIP engine (R9700, same flags) on that prompt and 39 on a story prompt, then near-tie flips
(margins 0.08-0.41 when following the HIP tokens); decode 74-75 tok/s.
2026-10-12 13:18:11 +02:00
Niko1221 7f3bed34ea Merge f-mac-kern (prompt-path probes, grouped expert GEMM, dense prompt GEMM) 2026-10-12 13:05:00 +02:00
Niko1221 773b44b922 metal: prompt path - MPP matrix throughput probe, grouped routed-expert GEMM, dense prompt GEMM
- kernels/mm_bench.metal + tests/mm_bench: M5 Max MetalPerformancePrimitives matmul2d fp16/bf16 62-65 TFLOP/s
  (64x64 tiles), fp32 15, simdgroup_matrix half 14.
- kernels/moe_mm.metal: deterministic GPU plan (block histograms, scan, stable placement; padded per-expert tiles),
  activation gather to fp16, mm_gu2 (gate+up+SwiGLU) / mm_down2 with weights dequantized (ref_quant arithmetic)
  into threadgroup memory and matmul2d in multiply_accumulate mode, mm_combine. tests/moe_mm_parity: Q2_0 and the
  IQ3_XXS-pack formats vs a reference with the same fp16 operands (1e-6..4e-5; vs exact float 2e-4..5e-4).
  T=2048, 512 experts, per layer: Q2_0 8.0 ms = 25 TFLOP/s = 3.9 us a token (~5,300 tok/s experts only);
  IQ3_XXS gate/up + IQ4_NL down 14.5 ms. The tile idea (dequantized weights from threadgroup memory under matmul2d)
  follows yibie/strata-mlx's tile_matmul (MIT); the code is ours.
- kernels/dense_mm.metal: dq_f16 (any dense format to fp16 once a chunk), act_f16, gemm_f16 (MPP): 64-65 TFLOP/s
  on the GDN/QSA shapes for 2048 tokens, dequantization ~4% of it; tests/dense_mm_parity (12 formats).
2026-10-12 13:03:25 +02:00
Niko1221 e4f80af46d Merge f-mac-kern (the Metal kernel tree) into f-mac-compat
metal/CMakeLists.txt holds both: mac-kern's runtime face, kernels and parity programs as they are, and the device
runtime + the engine build (taken from ../src through include/metal_compat) when METAL_CPP_DIR is given.
runtime_hello is kept out of the kernel tree's test glob.
2026-10-12 12:47:00 +02:00
Niko1221 ac36e3d4c0 metal: Q8_0 HC through the 3-dispatch read (gr_down/gr_up templated on the format) - the best HC variant
- gr.metal: gr_down_f<SG, KS, F> / gr_up_f<F> for BF16 (the old names) and Q8_0 (gr_down_q8_*, gr_up_q8);
  tests/gr_parity GR_Q8=1: vs double on the dequantized Q8_0 weights, 1/3/8 tokens, M5 Max and M4 pass.
- token_chain --hc q8_3. M5 Max, --fuse --graph, interleaved, 2 rounds: Q2_0 1 token 11.62 (BF16) / 10.59 (Q8_0,
  2 dispatches) / 10.38 ms (Q8_0, 3); 3 tokens 16.78 / 17.21 / 16.24-16.31. IQ3_XXS 1 token 12.85 / 11.87 / 11.68;
  3 tokens 17.88 / 18.59 / 17.47-17.51. The 2-dispatch read's gain was all Q8_0; its kernels are slower than the
  3-dispatch ones at windows. M4 HC read: Q8_0 85 us vs BF16 141-144 us.
2026-10-12 12:44:42 +02:00
Niko1221 5f18e8dcf8 metal: gemv_pair (shared expert gate + up + SwiGLU in one dispatch), raw-activation Q2_0 GEMV; pair measured slower
- gemv.metal: gemv_pair<F1, F2> for the packs' gate/up format pairs (tests/pair_parity: 8 pairs x 1/3/8 tokens vs
  double, 1.4e-7..2.6e-7) and GV_SET(T_Q2_0R).
- token_chain CHAIN_PAIR=1: interleaved A/B, 3 pairs, Q8_0 HC --fuse --graph: 10.60-10.61 ms without vs 10.71-10.72
  with (3 tokens 17.23-17.25 vs 17.31-17.33): two concurrent GEMVs and the SwiGLU win; opt-in only.
2026-10-12 12:33:59 +02:00
Niko1221 4e5ea96d23 metal: persistent-kernel probe (grid barrier between phases): correct only with device-scope fences, not faster
kernels/gridbar.metal + tests/gridbar_bench: phases of a streaming read separated by an atomic grid barrier vs by
dispatch boundaries (M5 Max). Without atomic_thread_fence(device, seq_cst) the phases read other threadgroups'
stale writes; with it 0 stale reads. Speed: 4.5 MiB phases 10.3 us (10 threadgroups) vs 9.9 us a dispatch, 30-34 us
with 20-40 threadgroups; 0.5 MiB phases 1.5 vs 2.2 us. Not pursued.
2026-10-12 12:30:33 +02:00
Niko1221 f1c5a6772c macOS platform guards in the host files (C2): every host source under src/ now compiles for Metal
All under __APPLE__ / STRATA_GPU_METAL, so the Linux and Windows code is unchanged:
- direct_file.cpp: no O_DIRECT on macOS; F_NOCACHE after open keeps the reads out of the buffer cache
- memory.cpp: F_RDADVISE is the read-ahead hint where posix_fadvise(WILLNEED) is used
- expert_source.cpp: available memory from host_statistics64 (free + inactive + purgeable); the posix_fadvise of a
  released expert layer is skipped (the madvise before it already released the mapping)
- generate.cpp: <sys/sysmacros.h> is not on macOS (major/minor come from <sys/types.h>); the CUDA toolkit mismatch
  check is CUDA's
- vmm.cpp: the cuMem* virtual memory stays CUDA's (Metal takes the HIP branch: elastic K/V off)
- prefill.cpp: std::max<size_t> where size_t and uint64_t meet (the same type on Linux, distinct on macOS)
2026-10-12 12:29:27 +02:00
Niko1221 ae84bcdc14 metal: GDN step templated on the row blocks (8: no gain), token_chain expert tiles by environment
- gdn.metal: gdn_step_t<IB>; gdn_step8 (1024 threads, 16 state rows a thread) passes and is not faster than
  gdn_step (16.3 vs 16.4 us commit, n=1): the state write after the compute is what a commit adds.
- token_chain: CHAIN_GU / CHAIN_DN / CHAIN_DN4 select the routed-expert tiles; measured in the chain (Q8_0 HC,
  --fuse, --graph): gate/up 32,1,4 stays; the Q2_0 down 8,1,4 over 8,2,4 (10.63 vs 10.67 ms, 3 tokens 17.23 vs
  17.50 ms), now the default.
2026-10-12 12:27:11 +02:00
Niko1221 a691e31b27 metal: QSA top-k v2 (parallel digit scan, range emission); attention variants measured
- kernels/qsa2.metal: qsa_topk2 - the same selection (ids exact at contexts 700..33000): the radix digit found by a
  parallel suffix scan of the 256 bins, cells emitted from contiguous block ranges after one scan. 23 -> 8.2 us at
  8K, 54 -> 14.5 us at 32K; token_chain (Q8_0 HC, --fuse, --graph, interleaved): 8K 11.40 -> 11.00 ms,
  32K 11.90 -> 11.27 ms.
- qsa_attn2 (chunks of 32, head groups) and qsa_attn3 (lane per cell for q.k): both pass, neither beats qsa_attn at
  the full budget (21-24 us, nearly independent of the cell count: a latency chain); QSA_ATTN selects, default 1.
- Read yibie/strata-mlx's OPTIMIZATION_RECORD/ROADMAP (MIT) before this round: its finding that merging kernels cut
  the kernel count 43% for 15% of the time matches ours.
2026-10-12 12:21:22 +02:00
Niko1221 79b98c82b7 metal: simdgroup-matrix GEMV for windows (exact, measured slower; kept as a reference path)
kernels/mma.metal: 8-row tiles dequantized with ref_quant's arithmetic into threadgroup memory, x loaded transposed
from device memory, simdgroup_multiply_accumulate; tests/mma_parity: 12 formats pass vs double, every token of a
window bitwise a one-token call. Speed (M5 Max): 60-190 GB/s for 1, 3 and 8 tokens alike, against the per-lane
GEMV's 400-595 GB/s at 1 token and 70-230 at 8: the dequantization through threadgroup memory (~8 B a weight) is
the limit. Only Q2_0 at 8 tokens is faster (115 vs 126 us). Not used by token_chain.
2026-10-12 12:15:03 +02:00
Niko1221 dca7c38405 metal: GDN step writes the prepared activations of its output; token_chain --fuse / --fold-comb
- gdn.metal: GdnArgs.plane: the step writes y as the raw plane plus the other planes and the per-16 sums, so the
  out projection needs no prep dispatch.
- gr2.metal: gr_down3 can form bo as the MoE combine itself (GrArgs3.comb).
- token_chain --fuse: the GDN prep in the step, no QSA score/top-k dispatch while the context fits the 2051 budget.
  Q8_0 HC, --graph: 10.71 -> 10.49 ms (1 token), 17.13 -> 16.86 ms (3); BF16 2-dispatch 11.91 -> 11.65 ms.
  --fold-comb (the combine in the next HC read) is slower: 10.49 -> 11.22 ms; off.
2026-10-12 12:08:33 +02:00
Niko1221 3c6eeb0049 metal: two-dispatch hyper-connection read (per-stream norm in the down kernel), Q8_0 HC option
- kernels/gr2.metal: gr_down3 (grid row blocks x streams: each threadgroup forms its stream's R', norm and xn in
  threadgroup memory, writes per-stream partial dots; R' ping-pong, never in place) and gr_up3 (lo from the
  partials, xn on the fly); BF16 or Q8_0 weights. tests/gr2_parity: both formats vs double on their own weights,
  prepared planes bitwise = xprep; Q8_0 moves the read's output by 1.2e-3 relative.
- ref_quant: quantize_q8_0 (ggml's reference rounding), used for the load-time Q8_0 copies (STRATA_HC_Q8's form).
- token_chain --hc 3|2|q8. M5 Max, Q2_0, --graph: 1 token 12.00 / 11.91 / 10.71 ms; 3 tokens 17.21 / 18.02 /
  17.13 ms.
2026-10-12 12:04:34 +02:00
Niko1221 bf4b79abda metal: the Metal runtime behind a CUDA-name compat layer (the HIP model), so the host files build for Metal unchanged
MAC-PLAN Phase 1, compat design (the alternative to renaming the host calls, f-mac-design S1-S4).
metal/include/metal_compat goes first on the Metal build's include path, as include/strata/hip_compat does on HIP:
its cuda_runtime.h gives the ~70 CUDA runtime names of the Phase 1 inventory (types, constants, calls) on Metal,
each forwarding to metal/src/gpu/runtime.cpp (metal-cpp: gpuAddress device pointers in one residency set, streams =
queue + command buffer + shared-event timeline, graphs = recorded dispatch lists), and immintrin.h gives _mm_pause /
_mm_sfence on arm64.  The CUDA / HIP / SYCL builds never see metal/.

Shared files: four preprocessor conditions also exclude STRATA_GPU_METAL from CUDA-only blocks (the cuMem*
segmented expert cache, the STRATA_VERIFY_NODES graph listing, the MTP stream priority, cudaInitDevice of the remote
experts); nothing else in src/ changes, so the PC objects are byte-identical.

On the M5 Max: runtime_hello passes with the unchanged src/core/graph.cpp, and graph, expert_cache, weights,
native_head, native_dense, layer, verify, mtp and session compile for Metal as they are (-fsyntax-only); prefill,
expert_source and generate stop only at macOS platform items (std::max of size_t vs uint64_t, posix_fadvise /
_SC_AVPHYS_PAGES, sys/sysmacros.h).
2026-10-12 11:53:32 +02:00
Niko1221 99614041ba metal: GDN commit writes the conv history to a second buffer; GEMV parity on real GGUF tensors
- gdn.metal: a q/k conv channel is read at the start of three threadgroups (value heads h, h + 16, h + 32) while
  its owner wrote it back in place at its end: on the 10-core M4 the threadgroups run in waves and the later ones
  read the new history (gdn_parity failed there, passed on the M5 Max where all 48 start together). The committed
  history now goes to hist_out (the caller swaps); gdn_parity checks determinism too. Passes on both Macs.
- gemv_parity --gguf: one real tensor per format of a GGUF (experts: expert 0) vs double; Q2_0 file: 11 formats
  pass, rel L2 8e-8..2.7e-7.
2026-10-12 11:51:56 +02:00
Niko1221 e03c0fa43b metal: whole-model decode step on real GGUF weights (token_chain), dependency-graph encoding
- src/gguf_lite: a minimal GGUF v3 reader; the file is mmapped and wrapped no-copy as one MTLBuffer.
- moe.metal addresses experts by gate/up/down offsets and strides (a pack blob or the GGUF's three tensors).
- kernels/glue.metal: the shared expert's SwiGLU (prepared activations) and the combine with its sigmoid gate.
- tests/token_chain: every layer of the model with its own formats, all kernels in one command buffer.
  M5 Max, Q2_0, context 1000: 12.83 ms a token (serial encoder), 12.01 ms with --graph (a concurrent encoder,
  barriers only at true dependencies); a 3-token window 18.42 / 17.23 ms. Bound: 7.27 ms.
- mtl_rt: run_graph (concurrent encoder, Dispatch::barrier).
2026-10-12 11:48:27 +02:00
Niko1221 6d94e2e409 metal: QSA decode (prep, block scores, top-k, split-K attention, gated merge) and the greedy sampler
- kernels/qsa.metal: q/k RMSNorm + NEOX RoPE (host table), int8 KV append (fp16 scale per 64), the indexer's raw
  tail / pooled block keys / dead key, ReLU-sum block scores with the +1e9 tail, a cell-level radix top-2051
  (ties to the lowest cell, ascending), split-K GQA attention over the selection, log-sum-exp merge * sigmoid(gate)
  written as prepared activations; tests/qsa_parity stage by stage (ids exact; contexts 700..9001).
- kernels/sample.metal: two-stage argmax, ties to the lowest id; tests/sample_parity.
2026-10-12 11:38:40 +02:00
Niko1221 89edfd7c3b metal: fused GDN layer step (conv, L2 norms, gates, delta rule, output norm) for a window, commit/verify
kernels/gdn.metal: one threadgroup per value head, the head's state in registers across the window's tokens;
commit writes state + conv history back, verify does not; tests/gdn_parity vs double (1.7e-7), verify leaves the
state untouched, the first m outputs of a longer window are bitwise a window of m (the fold).
2026-10-12 11:31:39 +02:00
Niko1221 dffa167fcd metal: hyper-connection read, router tail + plan, expert kernels specialised per group size
- kernels/gr.metal: gr_norm (folds the previous write, per-stream RMSNorm), gr_down (down + inject rows, silu/hc
  epilogue, K split), gr_up (gated mean over the streams, writes the prepared activations); tests/gr_parity vs double.
- moe.metal: moe_route = native_router.cu's softmax/top-10/renormalisation and the plan in one threadgroup
  (tests/route_parity: ids exact incl. ties, weights 2e-7); moe_gu/moe_down switch once per threadgroup into a body
  specialised for the group's entry count (the predicated 8-way loop cost 2x); Q2_0 down reads raw activations
  (T_Q2_0R) so no prep dispatch is needed.
- mtl_rt: warm() before timings (the GPU starts at idle clocks).
2026-10-12 11:27:33 +02:00
Niko1221 7b022f1fb8 metal: dense GEMV for every pack format, prepared activations, window-size specialisation
- kernels/units.metal: per-format unit decoders (Q2_0, Q3_K..Q6_K with the mask trick, IQ4 via a byte->half2
  codebook table, BF16, generic for the rest) and one combine for all; xprep writes the pre-scaled activations
  and per-16 sums once per input vector.
- kernels/gemv.metal: y = W x for 1..8 tokens in one pass over the weights, NTOK a function constant, optional
  K split across simdgroups; tests/gemv_parity: parity vs double for 12 formats x 20 tile shapes, every token of
  a window bitwise equal to a one-token call, and the bench on the packs' projection shapes.
- moe.metal on the same decoders; mtl_rt compiles with safe math by default (no reassociation; measured free).
- tests/alu_bench: instruction rates (fp32/fp16 FMA, int, convert, threadgroup/device/constant gathers).
2026-10-12 11:17:02 +02:00
Niko1221 8b271f24b5 STRATA_TRACE=2: wait for the device at each startup mark and name the first step after which it reports an error (#2013) 2026-10-12 11:05:03 +02:00
Niko1221 665d484236 SYCL: resident_stage_swaps copies the evicted expert back to its exchange buffer again (#2014)
The dpct migration had dropped the cudaMemcpyAsync D2H and the stream sync, so the resident RAM mode with adaptive swaps committed
whatever the exchange buffer held as the evicted expert (lost expert, wrong weights on a later CPU read). STRATA_CHECK_SWAPS=1 is a
test aid: it compares each staged buffer with the model file and audits the RAM copy at the end of a run.
2026-10-12 11:05:03 +02:00
Niko1221 ad9eb89079 metal: standalone Metal kernel tree - runtime face, bandwidth/dispatch probes, grouped routed-expert kernels
metal/ is its own CMake project (macOS only), nothing in src/ or include/ changes. MSL is compiled at run time
(the Command Line Tools have no offline metal compiler) with ggml-common.h prepended for the i-quant grids.

- src/mtl_rt.*: a C++ face over Metal (buffers, pipelines with function constants, serial/concurrent encoders,
  indirect dispatch, indirect command buffers, GPU timing).
- src/ref_quant.*: CPU reference dequantizers for Q2_0, Q4_0, Q5_0, Q8_0, Q3_K..Q6_K, IQ2_XXS/XS/S, IQ3_XXS/S,
  IQ4_NL/XS, BF16, F16; checked bitwise against gguf-py (tests/ref_dump + tools/check_ref.py: 17/17 equal).
- kernels/quant.metal: the formats as per-lane units (decode once, dot with every token of the window).
- kernels/moe.metal: moe_plan (top-k -> groups by expert, deterministic, writes the indirect grids), moe_gu
  (gate+up+SwiGLU, grouped), moe_down, moe_combine; tests/expert_parity: parity vs double and the bench.
- tests/rt_bench: M5 Max streaming read ~585 GB/s, ~0.85 us per dependent dispatch, ~170 us commit+wait.
2026-10-12 10:42:59 +02:00
Niko1221 0613cc00bc Stall report: an MTP draft wait is named (it read as the verify window before it); docs: the free VRAM moves after startup, and a stall in "request" can be the OS paging the parked conversation (#2049 #2065) 2026-10-12 10:39:47 +02:00
Niko1221 3804b43f19 Parking a conversation beats the watchdog: its checkpoint and K/V copies report progress, the stage is named, Windows counts hard page faults as file-tier activity, and the stall report and each request say how little VRAM is free (#2049 #2065) 2026-10-12 10:39:47 +02:00
Niko1221 798f3006bb Tests: STRATA_TEST_SLOW_COPY_US makes a parked conversation's checkpoint copy as slow as paged-out memory (for the #2049 repro) 2026-10-12 10:39:47 +02:00
Niko1221 905a60588b bench and monitor: name the GPUs the engine uses - mixed cards one by one in the report, the server's own card when it is not an NVIDIA one, the Intel reader first on the SYCL server (#2062 #2001) 2026-10-12 10:39:47 +02:00
Niko1221 17d8a171da CMake: STRATA_NATIVE_EXPERTS is defined with the other options, before the GPU section reads it, so the first configure registers the four native-expert tests (#2027) 2026-10-12 10:39:47 +02:00
Niko1221 6956f6052c setup: --calibrate with --model/--family tunes that installed model, an uninstalled one is an error, --calibrate with --setup/--check or nothing installed says it does nothing (#2036) 2026-10-12 10:39:47 +02:00
Niko1221 873a0d66b6 serve: a client that leaves while its pictures are downloaded, waiting for the turn or encoded stops the preparation: cancel exists before prepare() (#1999) 2026-10-12 10:39:47 +02:00
Niko1221 de603bb8bd serve: tool_call_recovery reads a declared tool's JSON call in a Markdown fence inside <tool_call> (#2037) 2026-10-12 10:39:47 +02:00
Niko1221 a683000875 serve: a quoted </tool_call> in a message is text, not the end-of-call token (#2026) 2026-10-12 10:39:47 +02:00
Niko1221 d320fecc85 serve: the adaptive overlap's off line printed (null) after the hint (a null reason went to %s) 2026-10-12 10:10:26 +02:00
Niko1221 d1ab65a18b setup: a BUILD.json is trusted only beside an engine binary of the source's version; a build that leaves the old binary is neither stamped nor installed (#2054) 2026-10-12 09:46:02 +02:00
Niko1221 0cf3d51030 ple: the slow-direct probe only warns on every platform (#2040: a SATA SSD tops out under the NVMe-calibrated 15K rows/s bar and mmap was 7x slower there); STRATA_PLE_AUTO_MMAP=1 keeps the 0.1.42 switch 2026-10-12 09:44:07 +02:00
Krypto-Whitehat 661ff29339 STRATA_ADAPT_OVERLAP: opt-in on every backend (RTX 5090 Laptop, Linux CUDA: 84.6 vs 106.6 tok/s decode median with it, 5 generations of 256 tokens from one build); log the state in both directions
Linux CUDA had it on by default since 0.1.43, measured on an RTX A4000 (+5.5..8.3% decode) and a Tesla P100 (+4.4%). An RTX 5090 Laptop with a native IQ3_XXS pack and --expert-cache-per-layer decodes 21% slower with it, and the overlap is the default only there. The switch is now =1 turns it on everywhere, and the startup log names the state (with the reason when it cannot run), so a decode-rate drop is not the only way to see it.

Unchanged: the copy path, the gate predicates and the tokens (same tokens with or without it).

Fixes #2010
2026-10-12 09:43:30 +02:00
Niko1221 b59e69c7bb gfx12 (RDNA4): STRATA_HC_Q8 is on by default (STRATA_HC_Q8=0 or STRATA_GFX12_DEFAULTS=0 turns it off; other cards unchanged). Droppable: R9700 decode +2.5 to +4%, KL 0.0003-0.0010 over the noise floor 2026-10-12 09:40:08 +02:00
Niko1221 ee9263ee38 sycl: NativeDense::hc_q8_requested() is declared by the shared header since STRATA_HC_Q8; the port's file defines it (the anonymous helper of the same name was hidden by the member, link error) 2026-10-12 08:01:06 +02:00
Niko1221 24d972e542 Multi-GPU: draft layer on the first card in the pipelined decode (default on gfx12), pipeline default from two stages, 8192 prompt chunks on a split, PcieAuto in the pipelined loop
On 2x/4x R9700 the drafter's chain and the last stage's window shared a card and slowed each other (window 13 -> 27 ms) in most engine starts; the draft layer now loads on the first card (STRATA_MTP_STAGE0=0 keeps the old place). Exact flags, tokens identical: 2 cards code +21%, 3 cards +19%, 4 cards +22% over the serial loop. A split reads prompts in 8192 chunks (STRATA_SPLIT_CHUNK16K=1 keeps 16384): 16K prompt +36..53%, 32K +18..29%, same tokens. STRATA_PCIE_AUTO_SPLIT=1 now also runs in the pipelined loop (opt-in).
2026-10-12 07:40:49 +02:00
Niko1221 2d47caa44a STRATA_HC_Q8: the prompt path's scratch is ordered by an event across streams (batch serving), the session read has its own pair 2026-10-12 07:40:49 +02:00
Niko1221 446bec512d STRATA_HC_Q8: the dequantization scratch of hc_bf16_view is allocated at load (per device), never on first use, so a graph capture cannot meet an allocation 2026-10-12 07:40:49 +02:00
Niko1221 b60aab1ba2 STRATA_HC_Q8=1: no bf16 copy of the layers' hyper-connection down/up projections (single GPU): they are quantized to Q8_0 from the pack file at load and left out of the arena; the prompt path and the single-token read dequantize a half's two matrices into a per-stream scratch 2026-10-12 07:40:49 +02:00
Niko1221 485984fdb8 fused_gr_bench: hash of the Q8 read's outputs and q8_1 image 2026-10-12 07:40:49 +02:00
Niko1221 0a379a1f62 STRATA_HC_Q8=1 on a bf16 pack: the hyper-connection down/up projections are quantized to Q8_0 on the GPU at load and read by the Q8 multi read; Q8 up kernel with 32-column blocks that quantize their own q8_1 group (HIP); bench: cold Q8 arm and Q8-vs-bf16 error check 2026-10-12 07:40:49 +02:00
Niko1221 34458600ab gfx1103 (Radeon 780M) joins the gfx11 WMMA kernels: decode x1.53, same answers
gfx1103 was missing from the gfx11 matrix-core lists, so the 780M ran the portable verify-window and prompt kernels. Added to the
compile-time lists and gfx_arch_is_gfx11_wmma(). Radeon 780M, Q2_0, 5 interleaved pairs: decode (code) 11.96 -> 18.25 tok/s,
same token ids; prompts unchanged. hip_prefill_wmma_gemm_parity passes on gfx1103 (3520 cases). Identity gate (defaults off,
mg_norepeat 10/10 against the portable build, exact_pp 4K/20K residual dumps identical).
2026-10-12 07:40:49 +02:00
Niko1221 24efc6e333 sycl/src/kernels/xmx_int8_bench.cpp: DPAS int8 vs FP16 expert GEMM microbench (standalone, not in the build) 2026-10-12 07:40:49 +02:00
Niko1221 3d6f1718ff SYCL: --expert-profile-save from the device routing counts under STRATA_VERIFY_NO_HOST (opt-in)
The serve path counts every routed expert on the device when a save path and a mirror exist; the saved ranking mixes the counts with the start ranking (prior worth 800k routed entries, STRATA_PROFILE_PRIOR_ENTRIES). Exact: placement is fixed under NO_HOST, the file applies at the next start. B70, IQ3_S: code +13..19% vs the xe default profile, prose -18%.
2026-10-12 07:40:49 +02:00
Niko1221 88892dbde4 Version 0.1.43.1: project version and the setup minimum engine 2026-10-12 02:12:36 +02:00
Niko1221 7bab660bf8 Prompt path: every pageable async host-to-device copy goes up through pinned bounce slots (#1996)
0.1.43 moved the QSA step records and the batched token ids to a pinned buffer (#1712). A 4-hour multi-slot load on 2x RX 7900 XTX (parallel 4, vision plus long
prompts, layer split) still faulted after 31 minutes, with the picture rows, the draft tokens and bounds and the non-mapped routing tables still going up from
caller memory. PinnedRing (include/strata/core/pinned_ring.hpp) is a ring of four pinned slots with one event each: the bytes are copied into the next slot after
the copy that last read it is done, the DMA runs from the slot, and the caller may change its source on return. No pinned RAM (or STRATA_TEST_PAGEABLE=1): the
copy is made from the caller's memory and the stream is waited for. A failed upload is returned and reported.

Moved onto it: the picture's embedding rows (gathered per run into one slot), the draft K/V step's tokens and bounds, the PLE rows' pageable fallback, the
non-mapped branch's routed-id download and slot/src/bounds uploads, the peer device's src and bounds tables, and the drafter's prompt records (mtp.cpp).
Copies from pinned arenas and cudaHostAlloc buffers are unchanged. Same bytes reach the GPU, so answers are identical.

The analysis of which copies are still pageable, and the bounce-slot-per-event idea, are mantovaniluca91's (patch C on #1712). This is our own form of it.
2026-10-12 02:11:56 +02:00
Niko1221 38032f546e Watchdog: a file tier busy within the last 10 s keeps the I/O allowance (a thrashing host has silent seconds), and Windows measures it too (bytes read + page faults); before, a 46 GB box's 20K prompt was stopped after ~90 s of page-ins (#1407 #1984 #1971) 2026-10-12 01:26:31 +02:00
Niko1221 ccc4be9165 bench: report only the GPUs the engine uses (#1985)
collect_machine listed every nvidia-smi card. It now takes the bench's --gpu, else the config's gpu (or --gpu in its args); with none given it names the one card the server reports instead of all of them.
2026-10-11 23:44:07 +02:00
Niko1221 c7838c529f serve: honour a failing before_load hook (503 + Retry-After), configurable timeout, /unload?wait=1 (#1997)
ensure_loaded logged 'loading anyway' for a before_load command that failed and co-loaded two models onto one GPU. A non-zero exit or a timeout now leaves the model unloaded and answers 503 with Retry-After (before_load_ignore_failure restores the old behaviour); before_load_timeout_s sets the 120 s limit; POST /unload?wait=1 (or wait=<s>) waits for the running turn instead of answering busy.
2026-10-11 23:44:07 +02:00
Niko1221 c5fa1b58a7 serve: an engine refusal of a BGEN slot (#1994) tries another slot instead of a permanent 400
The engine answered 'ERR BGEN: no such free slot' when its view of a slot differed from the server's; the server turned the ERR into an HTTP 400 and the conversation, drawn to the same slot by its held prefix, got it again on every turn. Now the refused slot is parked (busy, no owner) for SLOT_PARK_S, the request waits for / takes another slot, and after BGEN_REFUSALS_MAX refusals it ends with a retryable 503.
2026-10-11 23:44:07 +02:00
Niko1221 00e7d0c4cc SYCL: refuse STRATA_VERIFY_NO_HOST when a later layer-split stage is short of VRAM (#1980); setup_intel writes STRATA_STAGE_TRIM=1 for a split
The host mirror and the unmirrored count cover the first stage only; a later stage whose cache lacks experts has no mirror entry and, under NO_HOST, no host answers for them (device lost at the first verify window on 2x Arc Pro B60). The start now refuses with the existing wording, or logs the count when NO_HOST is off.
2026-10-11 23:26:45 +02:00
Niko1221 62497bff92 ple: --ple-io mmap prompt gather prefetches its rows a window ahead; on Windows the slow-direct probe warns and keeps direct unless STRATA_PLE_AUTO_MMAP=1 (#1984 #1971)
The mapped gather_batch took 16 serial demand faults per token from a cold table with no progress mark; the first chunk of a prompt only has layer 0 to hide it behind, so the stall watchdog stopped the engine. Rows are now hinted 4096 at a time (PrefetchVirtualMemory / MADV_WILLNEED). Answers are unchanged. The 0.1.42 auto-switch to mmap on Windows is opt-in until it is measured there.
2026-10-11 23:26:44 +02:00
Niko1221 9ba1d8deb6 docs: STRATA_ADAPT_SIGNAL 2026-10-11 23:00:49 +02:00
Niko1221 25e0a2ef79 Adaptive overlap: tell each layer's landing with a stream-ordered copy into pinned memory, not a host function
The per-layer host function made the overlap depend on the driver's callback thread being scheduled: with the pool's
workers spinning on every other CPU it still cost 7 ms per window on a 20-core desktop even with the thread on all CPUs
(RTX 3060, Core Ultra 7 265: 46.5 tok/s with the warmed callbacks against 56.5 with the overlap off). The copy stream now
ends each layer's copies with a 4-byte device-to-host copy of the layer number into a pinned int the gate polls; same stream
order, no host thread. STRATA_ADAPT_SIGNAL=host brings the host functions back. Swaps, residency and answers are unchanged.
2026-10-11 23:00:41 +02:00
Niko1221 0e20d9aee9 Start the GPU host-callback thread before the host is pinned, by default (STRATA_ADAPT_CALLBACK_WARMUP=0 turns it off)
On Linux the CUDA (and HIP) runtime creates its host-callback thread lazily, at the first cudaLaunchHostFunc, and a new
thread inherits the creator's CPU mask. The decoding host pins itself to one CPU (SessionLoopScratch::init), so a thread
created after that - which the adaptive overlap's per-layer callbacks did - ran on the host's CPU and fought the host's
spin-wait: ~1000 us instead of ~4 us per callback, decode 2-4x slower with STRATA_ADAPT_OVERLAP on (#1964, #1975).
oal's warm-up (one no-op callback + sync before ExpertPool) is now unconditional on Linux; the answers, swaps and
residency are untouched. Windows CPU Sets are per thread and not inherited, so Windows is unchanged.
2026-10-11 21:05:23 +02:00
Olav Lindekleiv c08fd90262 generate: warm CUDA callbacks before host-core pinning (opt-in) 2026-10-11 20:59:23 +02:00
Niko1221 7695859aa1 Canonical Q2_0 pack: the measured PCIe share is opt-in on Windows (STRATA_PCIE_AUTO_CANON=1); RTX 5070: code +2.5%, a short answer -3.3%, 5 pairs 2026-10-11 16:19:59 +02:00
Niko1221 7c035d729f STRATA_ADAPT_OVERLAP: opt-in on Windows (RTX 5070, Q2_0: story -2.6%, code -2.1% with it, 5 pairs; IQ3_XXS neutral); Linux CUDA keeps it on 2026-10-11 14:34:41 +02:00
Niko1221 0d0f106d5a docs/MULTI_GPU.md: the default pipeline's tokens are the serial loop's up to CPU rounding (4x R9700, default flags: 10 of 10 prompts matched over 160 tokens) 2026-10-11 14:08:31 +02:00
Niko1221 60abf63edc HIP layer split: the route tail skip's default stays off from 60% of the experts cached (90% elsewhere); measured on 2x R9700: Q2_0 story -6.2% at 70%, -1.8% at 60%, IQ3_S about 0 at 60% 2026-10-11 12:35:58 +02:00
Niko1221 003ea8c678 Merge PR #1892 (lask3802): the --serve NaN guard (non-finite head logits drop the caches and re-read from token 0; STRATA_NAN_GUARD=0 off); bitwise identical otherwise, cost under 0.2% 2026-10-11 12:01:30 +02:00
lask3802 f332a85570 Non-finite logits (#879): a guard with self-healing
Rarely a window's logits become NaN and the reply degenerates into token 0
('!') repeated (#606); the poison sits in the conversation's K/V and
recurrent state, so with the prompt cache every later turn stays poisoned
until it is read again from token 0. The q8_1 clamps (2b726ba, f6601bf,
7a0e302) remove the overflows found so far; this guard catches whatever
cause is left, and drops the poisoned state instead of serving it again.

On by default, as a fix: STRATA_NAN_GUARD=0 turns it off. It only reads
flags, so an answer without a NaN is the one the engine gives with the
guard off (compared byte for byte on a GPU, below).

Engine:
- every verify-window graph checks its head logits beside the greedy pick
  (logits_nonfinite_rows: 16 blocks a row, one flag word each, into mapped
  memory); Verifier::note_nonfinite reads them after the sync the window
  already does (run, run_slot_rows, batch_poll, pl_finish). No extra sync.
  With STRATA_NAN_GUARD=0 no kernel is recorded.
- --serve: a flagged window emits none of its tokens, one stderr line names
  the position and row, and the request ends with `DONE ... nonfinite` (a
  batch slot: `BDONE <slot> <n> nonfinite`; an admission: BADM <slot> 0).
  The next request first drops the live session, every checkpoint, every
  parked conversation (ConversationCache::clear) and every idle slot, and
  reads its prompt from token 0; a slot still running then finishes its
  reply and is not kept as a cache after it. A request that ends on it does
  not keep its conversation live (#1620's cancelled-read path is skipped
  for it).
  Covered: the serial decode loop (one GPU or a layer split), the
  --pipeline-windows decode loop of a layer split, the prompt's short tail
  (read_windows, and read_windows_pl with --pipeline-windows), and both
  batch-slot loops (run_slot_rows, and the --batch-groups pipeline's
  batch_poll). Not covered: the one-shot command line (no --serve) and the
  SYCL port in sycl/, which has its own verify.cpp and generate.cpp.
- STRATA_NAN_INJECT=<n>[,<n>...] (debug): the n-th window the guard checks
  reports row 0 as non-finite.

Server: EngineNonFinite on that finish. A request that has sent nothing yet
is sent again once (its usage is the retry's: the failed pass's DONE is
dropped); a reply that had started ends as "length"; a second fire on the
retry ends with an error (503 before an answer, an error event in a
stream). With replicas the router forgets what that replica held.
repeat_stop_tokens stays as the backstop.

Checks: serve tests (python -m unittest discover -s serve -t . -p
"test_*.py": 697 tests, OK, 11 skipped; NonFiniteLogits,
test_nonfinite_in_batch_mode and test_nonfinite_forgets_the_replica are
new). ctest with CUDA_VISIBLE_DEVICES=-1: 24 pass (conversation_cache_test
with clear() among them), 3 skip for want of a GPU, and the 43 GPU parity
tests were not run on a GPU. Built for CUDA 13.0 on Linux (GCC, sm_75 +
sm_86); generate.cpp, verify.cpp and verify_kernels.cu compile with
MSVC 2022 + CUDA 13.0, and conversation_cache_test and conv_cache_test
pass there; built for HIP (ROCm 7.0, gfx1100, compile only) and SYCL
(oneAPI 2026.1.1; it takes only the new declaration in verify_kernels.hpp
and the new inline ConversationCache::clear(), and calls neither).

On a GPU (RTX 3080, RVN IQ3_S, strata.service's one-GPU config):
- Answers, guard on vs STRATA_NAN_GUARD=0, at fixed expert sets
  (--pcie-frac 0 --adapt-every 0 --adapt-swaps 0 --mtp-max-t 1
  --suffix-draft 0, STRATA_IQ_MT_MIN=1 STRATA_PREFILL_CPU_SHARE=0), a chat
  replay of 2 x 4 turns on a 16K-token system prompt: byte-identical in 3
  pairs (one with STRATA_ROUTE_TAIL_SKIP=0, one with STRATA_SH_STREAM=0),
  and two guard-on runs 8 of 8. One more guard-off run differed from every
  other run from its first answer, and a second guard-off run matched the
  guard-on runs: a difference between runs of this config, not the guard.
- Cost, Nsight Systems: logits_nonfinite_kernel 424 launches, 4.77 us on
  average (7.78 us at most), 2.0 ms against 21.6 s of decode: 0.009%.
- STRATA_NAN_INJECT=1,60: the first window of a new chat flagged, the
  server sent the request again from token 0 and the client saw one normal
  answer; the 60th, mid-reply in the next turn, ended that reply as
  "length" after 58 tokens; the turn after it reused 0 cached tokens and
  answered normally. The engine logged exactly the two injected windows.
2026-10-11 12:01:22 +02:00
Niko1221 248dbee5b8 Merge PR #1737 (Ethan): route-resident swap one warp per token; bitwise equal to the scalar kernel, opt-in STRATA_ROUTE_RESIDENT decode +1.6-2.4% (R9700 32 GB) / +5.7-7.8% (16 GB-eq) 2026-10-11 11:22:28 +02:00
Ethan 115f62b00a route-resident parity: the swap against a scalar reference
route_resident_k is one warp per token and the form it replaced walked all 512 experts on a single
thread.  Nothing in the tree checked that the two land on the same expert and leave the same ids,
weights and counters, and the option's whole justification is that it moves misses across PCIe and not
the model.

src/kernels/route_resident_parity.cu: 240 random cases over 1 to 8 tokens, a 40-84% resident table,
random rank windows, margins from -0.5 to 1.5, and half the cases rounding the logits onto a coarse
grid so the tie rule (the smallest expert index wins) is exercised.  Half the cases get the router's
own top-10 and half get ids with repeats, which is what the already-picked test is for.  The reference
is the spec written out plainly - a serial walk with the same comparisons - and not a copy of the
kernel this branch replaced; it runs on the device so that expf, and so the softmax bits, are the same
function on both sides.

ids and the four counters have to match bit for bit.  The weights are a softmax ratio and this
reference is a second compilation of it, so they are compared within 8 ulp and the worst seen is
reported; the kernel itself is bit-identical to the serial form it replaced, which was measured with
both kernels side by side in one translation unit.  A case that swaps nothing still has to leave the
weights untouched, which the sentinel they start from checks, and the test fails if no case swapped at
all.

    route_resident_parity: 240 cases, 205 with swaps (2004 swaps), 0 mismatches, weights within 6 ulp

Registered in ctest next to the other router parity tests, TIMEOUT 120.
2026-10-11 10:52:44 +02:00
Ethan 33fd01b413 route-resident: one warp per token in the swap kernel
The swap kernel is the only work STRATA_ROUTE_RESIDENT adds to a window, and it was written as one
thread per token: with a verify window of 1 to 8 rows that is up to 8 threads of a block, each doing,
per rank in [lo, hi], a 512-expert scan with a ten-way "already picked" test inside it.  On this
machine that is about 4 x 512 x 12 scalar operations running on six threads, on the critical stream,
once per layer, and it shows up as its own cost in the window.

STRATA_VERIFY_PROFILE, three legs in one session, same prompts, same expert cache (8083 slots, one
learned profile), medians of the per-window stage table:

    configuration                        hc-read1+router   waitB    ms/window   pcie_e
    option off                                2.25          9.30      21.85      1.87
    option on, one thread per token           5.00          5.01      21.97      1.00
    option on, this commit                    2.58          4.87      19.75      0.95

The option redirects about a miss per layer-window away from PCIe and saves 4.3 ms of waiting for it;
the serial swap spends 2.75 ms back, which is why the option measured as a wash.  In the warp form the
swap costs 0.33 ms, so the waiting it saved stays saved.

End to end, twelve matched pairs of engine restarts, nine decode samples a leg, three greedy 150-token
prompts, the learned expert profile copied back before every leg so both arms start from the same
8083-slot cache.  The first six pairs ran the unpatched build first, the last six the patched build
first, so a within-pair order effect would show up as the two halves disagreeing:

    pair  order   decode unpatched   patched    delta    ms/window unpatched   patched    delta
     1    A->B         104.1          115.7     +11.1%         18.64          17.01      -8.7%
     2    A->B         103.5          107.4      +3.8%         18.47          18.12      -1.9%
     3    A->B         111.4          107.7      -3.3%         17.71          18.26      +3.1%
     4    A->B         101.4          113.0     +11.4%         19.11          17.03     -10.9%
     5    A->B         105.8          107.6      +1.7%         19.05          18.05      -5.2%
     6    A->B         104.8          105.6      +0.8%         18.60          17.83      -4.1%
     7    B->A         106.6          113.4      +6.4%         18.70          17.16      -8.2%
     8    B->A         103.1          110.2      +6.9%         18.14          17.29      -4.7%
     9    B->A         106.9          111.5      +4.3%         17.91          17.26      -3.6%
    10    B->A         104.3          114.7     +10.0%         17.81          17.11      -3.9%
    11    B->A         108.3          115.9      +7.0%         18.47          16.61     -10.1%
    12    B->A         104.5          113.6      +8.7%         18.43          17.76      -3.6%

    twelve legs each:   unpatched  decode median 104.7 (101.4-111.4)   ms/window median 18.47 (17.71-19.11)
                        this commit decode median 112.2 (105.6-115.9)  ms/window median 17.27 (16.61-18.26)

    decode      median +6.6%   mean +5.7%   faster in 11 of 12 pairs   (sign test p = 0.006)
    ms/window   median -4.4%   mean -5.2%   shorter in 11 of 12 pairs  (sign test p = 0.006)
    first six pairs: +2.7% decode, -4.7% window     last six: +7.0% decode, -4.3% window

pcie_e, the engine's own count of experts per layer-window that cross PCIe, is the same in both arms
(leg medians 0.63 against 0.64), so the same experts are routed and the difference is the kernel.

The swap's cost, and so its removal, scales with the tail experts the option actually redirects.  Four
more pairs with a 4000-slot cache instead of the automatic 8083 (pcie_e 1.83 against 1.78, i.e. about
three times the traffic of the series above), two pairs each way:

    pair  order   decode unpatched   patched    delta    ms/window unpatched   patched    delta
     1    X->Y          83.8           92.0      +9.8%         21.94          20.60      -6.1%
     2    Y->X          84.6           92.5      +9.3%         23.97          20.53     -14.4%
     3    X->Y          82.2           85.5      +4.0%         23.69          22.65      -4.4%
     4    Y->X          81.0           87.7      +8.3%         24.54          22.21      -9.5%

    decode median +8.8%, ms/window median -7.8%, 4 of 4 pairs, both orders.

What this changes is the shape, not the choice.  One warp per token: lane j reads experts j, j+32, ...
(coalesced instead of a serial walk), the warp reduces to the best logit in a shuffle tree, and the
ranks are still visited in order, because a swap changes the set already picked.  The choice is the
same bits: a strictly greater logit wins, the smallest expert index breaks ties, the already-picked
test runs against the ids as they are at that rank, a token with no swap keeps the router's weights
untouched, and the softmax is the same expression.

Parity.  400 random cases - half of them built with many equal logits, random lo/hi, margins from
-0.5 to 1.5, 1 to 8 tokens, about 60% of the experts resident, routed ids with repeats - compare the
two implementations bit for bit on the ids, on the weights (their raw bits) and on the four counters.
The harness carries both kernels side by side; 0 mismatches.

For context, the option's price is the margin's and not the kernel's, and this commit does not touch
it.  Teacher-forced log-probabilities over 7677 positions of one 7.6k-token text (technical prose,
source code and Chinese), top-64, against the same text with the option off:

    margin   argmax same   KL (top-64)   teacher-forced loss   perplexity
    off      -             0.000         3.2033                -
    0.25     84.9%         0.235         3.2238                +0.6%
    0.50     83.0%         0.293         3.4360                +7.3%
    1.00     63.5%         1.001         4.7751                +49%

Two runs with the option off are bit-identical, so that spread is the option and not the machine.
0.25 keeps nearly all of the waitB saving for a fraction of the distortion; the default 1.0 is not
worth its price.

Tests: the parity harness is a standalone .cu (both kernels, the random cases, the bit comparison).
Happy to wire it into the test suite next to the other kernel tests if you want it in the tree.
2026-10-11 10:52:44 +02:00
Niko1221 5ef500d48f Merge f-1712: the prompt chunk's step records and token ids go up from a pinned buffer (#1712) 2026-10-11 10:51:53 +02:00
Niko1221 46d22108ec HIP layer split: the route tail skip's default stays off from 70% of the experts cached (90% elsewhere); 2x R9700 16 GB-eq Q2_0 story -6.1% with it at 73% 2026-10-11 10:46:14 +02:00
Niko1221 a5f82ab77d Merge f-143-dense (via the trial merge 94f6d18d): CUDA sm_86 Q3_K interleaved kernel and one column; bitwise exact, A4000 decode +2.0-2.7% 2026-10-11 10:22:39 +02:00
Niko1221 2eba71548d Merge f-143-intel: B70 prompt and decode speed (copy engine, wide kv_append, no GEMM staging on bmg, lend mirror, fused dequant, lean verify graph, the xe expert ranking); same output tokens, B70 4K prompt 8.5 -> 3.3 s (128K config), decode +14-21% on prompts outside the ranking's corpus 2026-10-11 09:27:15 +02:00
Niko1221 b97a36a49a prefill: the chunk's step records and token ids go up from a pinned buffer, not pageable memory (#1712)
The QSA step records of a chunk were uploaded with an unchecked cudaMemcpyAsync from a
std::vector.  On gfx1100 the first text prompt after a picture one read garbage records
(5 of 5 faults on a two-request repro, 0 of 5 with a pinned, event-guarded source).  The
token id upload has the same shape and now takes the same path; both check the result.

Reported and traced by mantovaniluca91, #1712.
2026-10-11 09:16:52 +02:00
Niko1221 4d4f8bd50b docs/INTEL.md: server latency numbers 2026-10-11 09:10:59 +02:00
Niko1221 94f6d18d54 Merge f-143-dense: CUDA sm_86 Q3_K on the interleaved path and one column (STRATA_MMVQ_IL_Q3K=0 / STRATA_MMVQ_IL_T1=0 off) 2026-10-11 09:06:29 +02:00
Niko1221 3d5c16efe0 docs/INTEL.md: the 0.1.43 prompt and decode speed changes on the B70 2026-10-11 08:27:12 +02:00
Niko1221 8cb26e6095 SYCL: data/expert-profile-sycl.bin, the experts' ranking for a card on xe, built with make_profile_mix.py from the routing of 45 varied prompts and eight document summaries; setup_intel writes it instead of the shipped profile there (i915 keeps the shipped one). B70: +5..64% decode (mean +21%) on 12 unseen prompts, the same tokens. The copy-engine queue is Battlemage-only by default (it hung the A750's prompt) 2026-10-11 08:25:30 +02:00
Niko1221 ddd84269e4 Merge f-143-amdmg: --pipeline-windows 2 by default on gfx12 layer splits of 3+ stages (exact; 4x R9700 code +7-12%, 4K +9-11%, 32K +8-19%); STRATA_PCIE_AUTO_SPLIT=1 opt-in 2026-10-11 08:18:26 +02:00
Niko1221 b03304d934 HIP: --pipeline-windows 2 is on by default for gfx12 layer splits of 3+ stages (STRATA_PIPELINE_AUTO=0 / --pipeline-windows 0 off); STRATA_PCIE_AUTO_SPLIT=1 (opt-in) refines a two-stage split's PCIe shares per stage 2026-10-11 08:16:49 +02:00
Niko1221 218df14283 SYCL copy engine: one queue per device (a layer split asks once per card) 2026-10-11 07:58:41 +02:00
Niko1221 9e3ba42755 tools: make_profile_mix.py (an expert profile from several routing traces, each trace counting the same, with weights), collect_profile_traces.py (the traces of bench/profile_corpus.json's 45 varied prompts through the engine) 2026-10-11 07:58:34 +02:00
Niko1221 46a77907bb SYCL decode window: under STRATA_VERIFY_NO_HOST with the device plan the graph drops the nodes nobody answers (doorbell publish, three flag waits, the plan copy, the PCIe-share group and its staging); -2% decode on a B70, same tokens (STRATA_VERIFY_LEAN=0 turns it off) 2026-10-11 06:51:51 +02:00
Niko1221 331b591b36 Merge f-143-amdk: gfx12 decode node cuts (HC norm from registers, host-flag wait folded into the copy, HC up kernel owning its q8_1 groups, hit counts in the combine); bitwise exact, +1.6-2.0% decode 2026-10-11 06:01:01 +02:00
Niko1221 a9917a5524 gfx12 decode: the HC up kernel owns whole q8_1 groups and writes its consumer's image itself (STRATA_QFUSE_HC, no atomics), moe_hit_add folded into the combine (STRATA_FUSE_HITS); STRATA_FUSE_CONVAB (conv + a/b in one launch, opt-in, measured slower) 2026-10-11 05:59:48 +02:00
Niko1221 beba872b86 SYCL: the experts in the prompt loan's slots (the cache's last 10%, at most 1,200) are mirrored in pinned memory too, so the lent slots are refilled by DMA, not by a single-threaded GGUF read: 930 slots 916 -> 84 ms (STRATA_LEND_MIRROR=0 turns it off) 2026-10-11 05:53:27 +02:00
Niko1221 7a07f2811a SYCL prompt path: no GEMM operand staging on Battlemage (STRATA_SYCL_GEMM_STAGE=1 brings it back): the A770 oneMKL bug (operands past 4 GiB) did not show on a B70, first-token logits are byte-identical at 4K and 49K, and the 10 MB copy per expert was ~10% of a prompt 2026-10-11 05:44:57 +02:00
Niko1221 5a0df89817 SYCL prompt path: kv_append writes the host KV copy and the staging pool 4 bytes a lane (STRATA_KV_WIDE_HOST=0: the 1-byte stores): with --kv-resident the 1-byte stores into pinned host memory ran at ~30 MB/s, 2 s of a 4K prompt on a B70 2026-10-11 05:30:32 +02:00
Niko1221 cd5660593e SYCL copy engine: the stager's issued-flag is a copy, not a fill kernel (a copy-only command list runs no kernel: the driver aborted the lent-slot prompt of the 128K config) 2026-10-11 05:30:32 +02:00
Niko1221 751095962c SYCL prompt path: an expert's gate/up and down dequantized in one launch (iq_dequant_expert_f16, the same values; STRATA_PF_DQ_FUSED=0: two launches), -2% on a B70 4K prompt 2026-10-11 04:45:51 +02:00
Niko1221 7fc02cb5c2 gfx12 decode: the split HC norm from registers (-1.8 us a read), the host-flag waits folded into the copies that follow them (-2 nodes a layer); STRATA_VERIFY_NODES lists the window graph on HIP; the stage profile splits the FFN-side HC read; STRATA_DUMMY_NODES=k measures what an empty graph node costs 2026-10-11 04:04:07 +02:00
Niko1221 bb519f5bca bench: fused_gr_bench cold-weight mode with the norm/down/up split and an output hash; router_chain_bench (router GEMV, top-10 and doorbell per layer in a graph) 2026-10-11 04:04:07 +02:00
Niko1221 01fe68ddab SYCL: the prompt path's expert copies run on the GPU's copy engine (an immediate command list on the copy-only queue group, wrapped as a SYCL queue); the Level Zero v2 adapter ran them on the compute engine, ~13% of a B70 prompt. STRATA_COPY_ENGINE=0 turns it off 2026-10-11 04:00:52 +02:00
Niko1221 70f9c41772 SYCL: the grouped XMX expert kernels on Battlemage (STRATA_PF_XMX=1/2, opt-in): sub-group 16, joint_matrix 8x16x16, built for AOT bmg and lnl (STRATA_XMX_SG16); Alchemist keeps 8x8x16 2026-10-11 04:00:40 +02:00
Niko1221 5008962499 SYCL: STRATA_USAGE_DUMP=PATH counts every routed expert in the device plan (an atomic per entry) and writes the counts in --dump-routing format; a measurement of how much residency matters under STRATA_VERIFY_NO_HOST 2026-10-11 04:00:40 +02:00
Niko1221 195d379125 CUDA sm_86: Q3_K on the interleaved 2-4 column path and one column on the same kernel, sm_86 table (STRATA_MMVQ_IL_Q3K=0 / STRATA_MMVQ_IL_T1=0 turn them off)
IlQ3K runs q3_q8_dot_impl's arithmetic with the scales and signed values done once per chunk; bitwise native_mmvq's output
for T 1-4 and rows 1/2/4 (mmvq_il_parity). Q2_0 pack: the Q3_K multi-column kernels were 14% of GPU kernel time; the new kernel
takes 2560x10240 at 4 columns from 120 to 70 us on an RTX A4000. The shared table (sm_80+/sm_120) is unchanged.
2026-10-11 03:03:31 +02:00
Niko1221 da3fe1f119 mmvq_il_parity: the engine's real shapes, one column, Q3_K, MMVQ_IL_ONLY filter, table emit by worst ratio; fused_gr_bench: engine's staged read, L2-cold weights, GB/s 2026-10-11 03:03:31 +02:00
Niko1221 7c3fc48e86 Merge f-143-q2pcie: measured PCIe share for the canonical Q2_0 pack on CUDA (conflict with the gfx12 PcieAuto default resolved: one pf_old, canonical pack CUDA only) 2026-10-11 02:24:23 +02:00
Niko1221 ab97f17415 PcieAuto for the canonical pack: probe the link and refine the share from the CPU pool's speed (CUDA; start value stays 0.2; STRATA_PCIE_FRAC_DEFAULT=old and --pcie-frac keep it fixed) 2026-10-11 01:49:53 +02:00
Niko1221 c69adca61f Version 0.1.43: project version and the setup minimum engine; DETAILS lines for the gfx12 defaults 2026-10-11 01:49:39 +02:00
Niko1221 f5a0126762 Merge f-143-amd: gfx12 decode defaults 2026-10-11 01:49:18 +02:00
Niko1221 bf1e083f8a Merge f-143-amdpp: gfx12 chunk rule and PF_FUSED default 2026-10-11 01:49:17 +02:00
Niko1221 4ad4a25877 Merge f-143-kernels: P100 sm_60 tables 2026-10-11 01:49:17 +02:00
Niko1221 b5e040da40 Merge f-143-prefetch: adapt overlap default-on CUDA 2026-10-11 01:49:13 +02:00
Niko1221 85d90002b6 serve: a slot request that ends at its stop token reads its BDONE before it leaves, so its decode time and tokens are the slot leg's (#1869) 2026-10-11 00:56:31 +02:00
Niko1221 7475df931a HIP: the measured PCIe share (PcieAuto) is the default on gfx1200/gfx1201; STRATA_PCIE_FRAC_DEFAULT=old opts out; other AMD parts unchanged 2026-10-11 00:42:35 +02:00
Niko1221 ad2aef7648 HIP: STRATA_ROUTE_TAIL_SKIP=7 is the default on gfx1200/gfx1201 under the CUDA rules (#1884 cached-share line included); other AMD parts unchanged 2026-10-11 00:42:25 +02:00
Niko1221 606eb1080c generate: decode_defaults_arch_ok() (gfx1200/gfx1201 on HIP, always on CUDA) for the HIP decode defaults 2026-10-11 00:42:09 +02:00
Niko1221 5f9bbf61c2 serve: a request moved from the solo path to a batch slot keeps the solo leg's prompt figures, decode time and tokens (#1869, #1870) 2026-10-11 00:34:36 +02:00
Niko1221 5a66924d8d gfx12: the fused int8 prompt experts are on by default for IQ3_S and IQ3_XXS packs; STRATA_PF_FUSED=0 turns them off 2026-10-11 00:19:22 +02:00
Niko1221 cf09057a86 gfx12 (Linux): a bare --prefill auto may take 16384-token chunks; explicit --prefill N, auto:N and STRATA_PREFILL_AUTO_MAX are untouched; STRATA_GFX12_CHUNK=0 or STRATA_GFX12_DEFAULTS=0 keeps 8192 2026-10-10 23:50:49 +02:00
Niko1221 0d761d99aa Resident RAM mode overlap: opt-in (STRATA_ADAPT_OVERLAP_RESIDENT=1); exact, +7.0% on an A4000, -2.6% on a RAM-bound 3060 box 2026-10-10 23:00:37 +02:00
Niko1221 6ab541f625 CUDA: the tail-skip default stays off when 90% or more of the experts are in the GPU cache (#1884); STRATA_TAIL_SKIP_MAX_CACHED moves the line 2026-10-10 22:16:48 +02:00
Niko1221 2eaeda2e4b STRATA_ADAPT_OVERLAP in the resident RAM mode: soft_commit_exchanges makes a staged evicted expert look committed (pinned, device alias into its exchange buffer) while in's RAM place stays the copy source 2026-10-10 22:14:33 +02:00
Niko1221 8c34c475da STRATA_ADAPT_OVERLAP_SLOW_US: test hook that slows the adaptive copies so the windows wait for them 2026-10-10 22:12:27 +02:00
Niko1221 0636c4642f serve: a request handed from a batch slot back to the solo path keeps the slot leg's decode time and tokens (#1869); timings.prompt_n is what the engine read, the rest is cache_n (#1870) 2026-10-10 22:05:52 +02:00
Niko1221 cd78501359 strata_bench: skip the agent session when the context is below ~10K instead of sending a request that fails with HTTP 400 2026-10-10 22:00:17 +02:00
Niko1221 f95f2eabbd STRATA_ADAPT_OVERLAP on by default for CUDA (=0 turns it off), docs; the h2d copy-engine microbenchmark 2026-10-10 20:37:02 +02:00
Niko1221 e7558f8c5f CUDA sm_60: expert mode 8 (grid and activations in shared memory, SwiGLU fused) is the Tesla P100's default, STRATA_SM60_TABLE=0 or STRATA_EXP_MODE=0 turns it off
Bitwise equal to the CUDA layout on all 48 layers at 1-4 tokens a group (native_expert_bench); a 28-group VRAM call 17.6 -> 15.5 ms over the layers at T=1, 24.1 -> 22.6 at T=2, even at T=3, +2.6% at T=4.
2026-10-10 20:01:31 +02:00
Niko1221 9ef4484248 STRATA_ADAPT_OVERLAP=1 (opt-in): the adaptive tier's copies land while the next decode window runs, a layer waits for its own copies 2026-10-10 18:49:32 +02:00
Niko1221 e5387d41ec CUDA sm_60: interleaved 2-4 column GEMV with a Pascal rows table (P100), STRATA_SM60_TABLE=0 turns it off
Bitwise equal to native_mmvq (mmvq_il_parity, T 2-4, rows 1/2/4); table read off mmvq_il_parity --bench --emit-table on a Tesla P100 (three identical runs).
2026-10-10 18:39:38 +02:00
Niko1221 86070e7845 STRATA_TRACE: VRAM marks after the context, the pack weights, the session and the head (the 0.1.43 VRAM audit) 2026-10-10 18:30:35 +02:00
Niko1221 885f9f7016 Version 0.1.42.1: project version and the setup minimum engine 2026-10-10 18:24:47 +02:00
Niko1221 00cc68e77a Merge f-142d-bench into f-1421 2026-10-10 18:24:33 +02:00
Niko1221 bccef1cf81 Merge f-142d-numa into f-1421 2026-10-10 18:24:33 +02:00
Niko1221 02a4216fea Merge f-142d-unmap into f-1421 2026-10-10 18:24:33 +02:00
Niko1221 61b3fb5dd3 Merge f-142-pciefrac into f-142 2026-10-10 14:49:37 +02:00
Niko1221 0e9af37041 docs/DETAILS: the PCIe share set from the CPU pool's speed (CUDA, single GPU); measurements and opt-outs 2026-10-10 14:48:49 +02:00
Niko1221 c2cbf8c112 Merge f-142c-triage into f-142 2026-10-10 13:54:32 +02:00
Niko1221 407ea8192a Merge f-142c-small into f-142 2026-10-10 13:54:28 +02:00
Niko1221 879096d66c Benchmark: backend wording, run time text 2026-10-10 13:46:55 +02:00
Niko1221 ed5536ea88 file_expert_source_test: fix the cache-hint test string 2026-10-10 13:35:47 +02:00
Niko1221 6b8c2e74d7 Benchmark: prompt sizes calibrated by slope, agent re-read counts only earlier turns, no doubled quant in the folder name 2026-10-10 13:25:09 +02:00
Niko1221 dfaebd9d9f Benchmark mode: BENCH.bat / bench.sh and tools/strata_bench.py measure an install through the HTTP API and write a community report 2026-10-10 13:17:12 +02:00
Niko1221 52a2dd10ac Windows GGUF-in-place: close every GGUF shard view once the file tier reads unbuffered (NTFS serializes unbuffered reads of a mapped file) 2026-10-10 13:13:17 +02:00
Niko1221 8c80a3ba12 Start-up after another model: judge pinning by the emptiest NUMA node, say how much is file cache when RAM is short, opt-in STRATA_DROP_CACHE_ON_EXIT 2026-10-10 13:12:47 +02:00
Niko1221 aa7b0cb13e CUDA: the default --pcie-frac is refined in the first decode windows from the CPU pool's measured speed (single GPU, no --pcie-frac); STRATA_PCIE_FRAC_DEFAULT=old keeps the start-up rule 2026-10-10 13:01:46 +02:00
Niko1221 d3e9106e1d tools tests: follow #1792 (no version range without CUDA, STRATA_VCVARS), and the effort-end test no longer collides on a case-insensitive disk 2026-10-10 11:11:20 +02:00
Niko1221 0ae808f37d tools tests: the llama.cpp archive test passes a plain path (a file:// URI with a drive letter is not found on Windows) 2026-10-10 11:04:37 +02:00
Niko1221 a02cacbb3d docs: card order keeps the config's order for cards within 5% (#1760) 2026-10-10 11:02:19 +02:00
Niko1221 f63a57e726 prefill gemm: BF16 on Pascal/Volta converts the weight in tiles when the conversion buffers do not fit, instead of falling through to the raw BF16 cuBLAS call (#1757) 2026-10-10 11:02:14 +02:00
Niko1221 184012cac4 serve: cards whose speed scores are within 5% keep the config's card order (#1760) 2026-10-10 11:02:14 +02:00
Niko1221 47cb731026 serve: telemetry test for the AMD PCIe link (#1733): the reader gains the pcie_* keys, and the hop walk is tested 2026-10-10 10:51:32 +02:00
Niko1221 4d0d11f72d bench: COMMUNITY.md rows and the docs list entries for #1739 #1751 #1784 #1793 #1799 #1802 #1811 2026-10-10 10:47:13 +02:00
duncanmcqueen 27528ca53d docs: Intel Arc A770 decode speed investigation record (PR #1765; local paths removed) 2026-10-10 10:47:13 +02:00
Mitsuasa513 83b2b676dd bench: community report, Radeon Pro VII 16 GB (gfx906), ROCm 6.2 (PR #1793; build and server logs left out) 2026-10-10 10:47:13 +02:00
Hcl192088 d5400b54f5 bench: community report, RTX 4070 12 GB, IQ2_XS, modified engine (PR #1739; hostname redacted) 2026-10-10 10:47:13 +02:00
kgmkm 3bfc708d07 bench: community report, RTX 5060 Ti (WSL2), IQ3_S, engine 0.1.41 (PR #1751) 2026-10-10 10:47:13 +02:00
VykosX 432a45c708 bench: community report, RTX 5060 Ti / Xeon E5-2680 v4 (PR #1784) 2026-10-10 10:47:13 +02:00
gun-powder 80ea56af12 bench: community report, 2x Tesla P40, IQ3_S, 131K (PR #1799) 2026-10-10 10:47:13 +02:00
isoux35 7f980b5bd2 bench: community report, RX 9060 XT 16 GB, IQ2_XS and IQ3_XXS (PR #1802) 2026-10-10 10:47:12 +02:00
Daniel Spitzer c50a217a99 bench: community report, RX 9070 16 GB, IQ3_XXS, engine 0.1.41 (PR #1811) 2026-10-10 10:47:12 +02:00
Niko1221 70ac6dd41f serve: repeat_penalty / repeat_last_n (llama.cpp's names) for the config's sampling block and for requests (#1819) 2026-10-10 10:45:03 +02:00
Niko1221 2ea10c2e9a sycl: iq_parity and ple_parity are registered as the top-level build does (#1795) 2026-10-10 10:44:09 +02:00
architectds 6b392641af verify: service returns before any CUDA call while nothing has rung
The pipelined decode loop asks both stages' windows for service millions of times a second while they wait on their
doorbells, and every call switched the device (OnDevice: cudaGetDevice and two cudaSetDevice) before finding nothing
to do. On a 0.1.41 split (RTX 3060 + RTX 5070 Ti, Windows) the host stalled inside one of those cudaSetDevice calls
after 919 million loop passes, with both stages' windows waiting on it (stall dump: the serving thread in cudart from
Verifier::service, every other engine thread idle). Now a call with nothing rung, no WDDM flush due (2 ms) and no
timeout due returns 0 first - the same answer the loop below gives - so the driver is called only to serve a layer,
to flush, or to time out.
2026-10-10 10:43:10 +02:00
Niko1221 093edf2474 build: guard the CUDA::cuda_driver link for the HIP stand-in targets 2026-10-10 10:43:10 +02:00
Kurokosama b0eb396528 serve/web: add a favicon to the web app and the API Monitor
The pages had no <link rel="icon"> and serve/web had no icon file, so browsers asked for
/favicon.ico, got a 404 and showed the default blank icon in the tab.

serve/web/favicon.svg is the layer stack the app already uses (serve/web/sprite.svg, i-layers),
redrawn on a 32-unit viewBox so it stays legible at 16 px, in the existing accent token #10b981.

Checked: curl /web/favicon.svg -> 200 image/svg+xml, both pages come back with the icon link,
and the icon rendered at 16/32/48/64 px in Chrome.
2026-10-10 10:42:55 +02:00
phoenixclyde 7f3c0da241 telemetry: the Monitor's PCIe tile had no link for AMD cards on Linux
_Nvml.read() reports pcie_gen, pcie_gen_max, pcie_width and the two
throughput counters; _Amd.read() (the amdgpu sysfs reader from #301)
reports none of them, so on Linux the Monitor's PCIe tile shows only
dashes while the NVIDIA tile fills in.

This adds the link the card actually gets: the narrowest and slowest hop
between the root port and the card, from each hop's max_link_speed and
max_link_width.  A card that negotiates Gen4 on its own hop but sits
behind a Gen3 root port really runs at Gen3, and that is the number a
bandwidth budget needs.  pcie_own_gen keeps the card's own hop aside so
the gap is visible, and pcie_path reports how many hops could be read --
the bottleneck is only claimed when all of them could, because a kernel
or a container that hides part of /sys/devices would otherwise report the
card's own hop as the whole path.

Measured on one MI50 32 GB in a Gen3 x16 slot (lspci: the root port at
00:02.0 runs at 8 GT/s x16, the bridge above the card is downgraded, and
the card's own hop negotiates 16 GT/s), /metrics reports pcie_gen 3,
pcie_gen_max 3, pcie_own_gen 4, pcie_width 16 and pcie_path 4/4.

Throughput is deliberately left out: amdgpu's pcie_bw is three counters
whose documented use is a total, not a direction, while _Nvml's
pcie_rx_mb is receive-only, so filling that name from pcie_bw would
label a combined figure as one direction.  Happy to follow up separately
if you want the tile to show a rate.

Hardware the numbers come from: one Radeon Instinct MI50 32 GB (gfx906,
PCI 1002:66a1, VBIOS 113-D1631711-100) on a Xeon E5-2696 v4 (Broadwell,
no AVX-512), in a PCIe 3.0 x16 slot, 128 GB RAM, Ubuntu, kernel
7.0.0-34-generic, ROCm 7.14 in a container.
2026-10-10 10:42:55 +02:00
phoenixclyde dc0ee3acf7 prefill: the gfx906 build reached the CUDA branch of arm_cpu_share
Prefill::arm_cpu_share() asks "is this AMD?" with
#if !defined(STRATA_USE_HIP), but the gfx906 compat build (PR #638)
compiles as HIP *without* STRATA_USE_HIP -- see src/core/expert_cache.cpp
:5-6, which guards the same question with
#if defined(STRATA_USE_HIP) || defined(STRATA_HIP_GFX906).

On gfx906 the CUDA branch is therefore live: the CPU share is armed by
default for chunks below 1,024 tokens and prints its banner, against this
file's own note at :161 ("the layer split, batch slots and HIP stay off").
Seven other sites already carry the || defined(STRATA_HIP_GFX906) form.

Measured on one MI50 32 GB (gfx906), CLI generate, a 4-token prompt, one
binary with one set of weights and one config, only the environment
differing, power saving off:

  default  experts streamed 309 (295 by DMA, host 174.0 ms), resident 607,
           on the CPU 197 (share 0.90)          prefill 928.3 ms
  =0       experts streamed 506 (492 by DMA, host 177.3 ms), resident 606,
           on the CPU 0 (share 0.00)            prefill 929.5 ms

Output token ids are identical (271 248068 198 760) and the 0.13%
difference is inside this machine's run-to-run spread (1.2-1.7%), so on
this card the share engages and buys nothing.  #1595 and #1729 report the
default costing on CUDA; this is the mechanism that turns it on for AMD.

CUDA and SYCL do not define STRATA_HIP_GFX906, so the added check is a
no-op there and nothing on those backends changes.

Hardware the numbers come from: Xeon E5-2696 v4 (Broadwell, 22C/44T, no
AVX-512), one Radeon Instinct MI50 32 GB (gfx906, PCI 1002:66a1, VBIOS
113-D1631711-100, 34,342,961,152 B of VRAM), PCIe 3.0 x16 effective (the
root port runs 8 GT/s x16 and the bridge above the card is downgraded;
the card's own hop negotiates 16 GT/s, about 15.75 GB/s end to end),
128 GB RAM, Ubuntu, kernel 7.0.0-34-generic, ROCm 7.14 in a container,
built with -DSTRATA_HIP_GFX906=ON and -DCMAKE_HIP_ARCHITECTURES=gfx906.
2026-10-10 10:42:55 +02:00
phoenixclyde 6a7c6f8275 web: the unlabelled sparklines were scaled by their own noise
spark(id, values, max) sets the axis top to max(|max|, the series' own
maximum, 1e-9).  gpu_util, temperature, cpu, vram and power pass an
explicit maximum, but the pcie, tok/s, prefill and disk cards do not, so
those axes zoom onto whatever the series happens to hold.

An idle amdgpu pcie counter reads 0.01-0.02 MB/s (50 packets x 256 B per
second), and that becomes a full-amplitude square wave in the Monitor
while the tile's own text says 0.0 MB/s.  Measured in the live DOM: the
pcie line's y values are only {4.0, 30.0}, i.e. the baseline and the top.

A floor of 1 only ever lowers the axis - the series maximum still wins -
so a real burst keeps its scale, and the cards that pass a maximum are
untouched.
2026-10-10 10:42:55 +02:00
chaog992 031fddc62c tools tests: test_amd_is_not_asked mocks the Linux HIP engine build too (it wrote sys.prefix/.strata-rocm.json)
IQ4XS.test_amd_is_not_asked mocks get_prebuilt_hip, the engine setup.main takes on Windows. On Linux setup.main
compiles the HIP engine instead (build_engine_hip), which the test did not mock. Without a usable system ROCm 7, as
on the tested machine, rocm_root then fetched AMD's ROCm index
(https://rocm.nightlies.amd.com/v2/gfx120X-all/rocm/), wrote Path(sys.prefix)/.strata-rocm.json and tried to run
<python dir>/rocm-sdk. Where sys.prefix is not writable (the system Python) that is a PermissionError. Where it is
(a venv), the test passed while setup.main stopped at the ROCm step with exit code 1, and a stamp with the same
version and index can make a later setup in that venv skip the ROCm install.

The test now mocks build_engine_hip the same way, so on Linux setup.main runs to the end (exit code 0) with no
network request, no subprocess and no write outside the test's temp folder. setup.py is unchanged; the Windows path
never reaches build_engine_hip, so it runs as before.

Tests: python3 -m unittest tools.test_setup_unsloth (WSL2): with the system Python 3.12.3 FAILED (errors=1) before,
38 tests OK after; in venvs on 3.10.22, 3.12.3, 3.13.7 and 3.14.8 OK before and after, and no stamp after. With the
system Python the other 27 tools/test_setup_*.py files give the same results as before.
2026-10-10 10:42:55 +02:00
Hugua700 db5b1f19d6 setup: find_vcvars() without CUDA asks vswhere for no version range (#881)
The `[16.0,18.0)` range is CUDA 13's requirement, but it was applied to every caller: with the
only Visual Studio installed being 2026 (version 18), vswhere finds nothing and find_vcvars()
returns None.  That makes the CPU image encoder unbuildable on such a machine - build_vision_cpu()
cannot run - so --vision cpu stays unreachable there even with the gate in hip_vision() open.

#985 relaxed the range for CUDA 13.3+ by adding `cuda_v`, which is right for CUDA callers.  A
caller with no CUDA now passes nothing and gets no range, which is what
tools/hip/build_windows.bat already does when it calls vswhere directly.

- `cuda_v is None` -> no -version argument at all.  CUDA callers are unchanged: cuda_v=(13, 0)
  still refuses VS 18, cuda_v=(13, 3) still takes it.
- STRATA_VCVARS=<vcvars64.bat> picks an install by hand, for a vswhere that cannot be used.

Verified on the RX 6800M / gfx1031 machine of #881 (Windows 11, Chinese locale, the only C++ tools
are Visual Studio 2026 / 18.4.11605):

  find_vcvars()               -> <VS 18>\VC\Auxiliary\Build\vcvars64.bat
  find_vcvars(cuda_v=(13, 0)) -> None
  find_vcvars(cuda_v=(13, 3)) -> <VS 18>\VC\Auxiliary\Build\vcvars64.bat
2026-10-10 10:42:55 +02:00
leadgtr7 7650292cfd sycl mtp: slot drafters copy the shared draft head's type, as src/core/mtp.cpp does since #1110
With a draft-vocab head (rt/draft_vocab.bin) the slot drafters of --batch-mtp share the main drafter's head but kept
dhead_type_ at -1, so the first draft_first() of a batch admission threw 'mtp: unsupported native MMVQ GGML type'
and the engine exited (#1121). The CUDA source got this line in #1110; the SYCL port did not.
2026-10-10 10:42:55 +02:00
David d73108c7e8 build: link CUDA MMQ error reporting to the driver API 2026-10-10 10:42:55 +02:00
GRAB Racem 253e09f18d serve: the reasoning loop check also sees a loop written as one long word (#1753)
A periodic loop emitted as digits (a 24k-digit string cycling an 11-digit
pattern) is a single word to the word count and never one token twice
(the tokenizer has no multi-digit tokens), so both #606 and #728 passed
it and the run burned the whole reasoning budget.  The reasoning's last
2,048 chars are now checked for a period of at most 64 chars, and a
periodic tail reads as full coverage.  The opt-in path only: the default
path and the word count are unchanged.
2026-10-10 10:42:44 +02:00
leadgtr7 b9cba34bc2 serve: effort_position "end" reads the SYCL engine binary, not its launcher script, for --tail-role-token
With the Intel port the configured exe is sycl/serve/strata-sycl.sh, a shell script that runs the repo's
STRATA_SYCL_BIN (default build-sycl-aot/strata) in a container. effort_end_args looked for --tail-role-token in the
script, never found it, and fell back to the top-of-prompt effort with "needs engine 0.1.39 or newer", even on a
0.1.41 engine that has the option. engine_binary() resolves the launcher to the binary it runs; any other exe is read
as before. tools/test_effort_end_args.py covers both paths.
2026-10-10 10:42:44 +02:00
nekomario28 72843d4ca9 sycl: use the source version for setup model checks
The SYCL project declaration uses a CMake variable, not a literal
version. Reuse setup's parser so the stub no longer reports version 0.
2026-10-10 10:42:44 +02:00
Hulk 153416719d fix(setup): discard damaged llama.cpp source archives 2026-10-10 10:42:44 +02:00
Niko1221 cdbf458684 Version 0.1.42: project version and the setup minimum engine (0.1.41.1 was not released; its fixes are in 0.1.42) 2026-10-10 01:23:45 +02:00
Niko1221 d5f319bffc Merge f-142-cuda-p1r into f-142 (inert lookahead stats, cache-policy replay tool, docs) 2026-10-10 01:23:07 +02:00
Niko1221 136d33cdc3 Merge f-142-tailskip into f-142 (STRATA_ROUTE_TAIL_SKIP=7 default on CUDA serve) 2026-10-10 01:23:07 +02:00
Niko1221 7bbfab810b docs/DETAILS: the tail skip default (KL, task check, speeds, opt-out); start-up line wording 2026-10-10 01:22:27 +02:00
Niko1221 51cc86291b docs: STRATA_LOOKAHEAD_STATS and tools/replay_cache_policy.py 2026-10-09 23:10:26 +02:00
Niko1221 b4e3407bd2 STRATA_ROUTE_TAIL_SKIP=7 on by default on CUDA serve (=0 off): single GPU and layer splits; not with every expert in VRAM, --batch slots, a peer or helper GPU, HIP; one start-up line names the opt-out 2026-10-09 22:52:22 +02:00
Niko1221 223655ac91 replay_cache_policy: policy options on the command line 2026-10-09 21:09:03 +02:00
Niko1221 9462c58287 tools/replay_cache_policy.py: replay a --dump-routing trace through the adaptive expert tier's policy offline 2026-10-09 21:08:49 +02:00
Niko1221 b02a03d97c Look-ahead stats: vote-ranked top-M coverage of the misses and of the PCIe share 2026-10-09 20:47:12 +02:00
Niko1221 1d735110ae Router look-ahead stats: report the predictor thread's time per window 2026-10-09 20:47:12 +02:00
Niko1221 e247c61127 Router look-ahead: STRATA_LOOKAHEAD_STATS scores each layer's prediction against the real routing (measurement only) 2026-10-09 20:47:12 +02:00
Niko1221 8b0c241ca0 sycl: build q2_avx1.cpp and iq_avx1.cpp (the AVX1 expert rows native_expert.cpp calls; the SYCL strata target failed to link without them) 2026-10-09 19:09:54 +02:00
Niko1221 3ff62ec140 docs: batch-MTP on a layer split with all experts in VRAM lost 21.9% (2x R9700); it pays only when the experts do not fit 2026-10-09 18:58:08 +02:00
Niko1221 7a4bc813ba conversation_transfer_test: wrap the calls of the non-blocking copy helper 2026-10-09 18:57:40 +02:00
Niko1221 a6346cdcea conversation snapshot/checkpoint copies: private non-blocking stream instead of the legacy stream (HIP capture conflict) 2026-10-09 18:57:40 +02:00
Niko1221 bb930bc8be Merge f-142b-amd into f-142 2026-10-09 18:57:36 +02:00
Niko1221 5a90e36cb3 Merge f-142b-intel into f-142 2026-10-09 18:57:36 +02:00
Niko1221 54d7f7d7c5 Merge f-142b-cuda into f-142 2026-10-09 18:57:36 +02:00
Niko1221 6e1085c1de Merge f-142b-serve into f-142 2026-10-09 18:57:35 +02:00
Niko1221 ca2730b14b docs: PF_FUSED KL re-check on IQ3_S / IQ3_XXS / Q2_0 (stays opt-in) 2026-10-09 18:11:08 +02:00
Niko1221 0b3139ec18 gfx906 build: refuse a wave32 card at start with a message (#1728); docs: community-tested, opt-in note 2026-10-09 18:11:01 +02:00
Niko1221 e33edd05bf bench #1726: keep the reporter's background-workload list complete (a desktop chat app was open) 2026-10-09 16:51:42 +02:00
Andrey Bash 354f982118 gfx906: add opt-in reduce12 QSA score reduction
Follow query-swizzle #1661 with the existing reduce-scatter pairing for
12 heads. Keep default paths and softmax/value/merge unchanged. Public
component parity and timing passed; model evidence uses frozen stack.
2026-10-09 16:48:03 +02:00
Andrey Bash 254f5aeca9 docs(gfx906): record 200K and API qualification for the combined candidate
Record archived-control output parity, verified candidate process, tools, CPU vision and cache restoration. No fresh 200K speed comparison or production promotion is claimed.
2026-10-09 16:48:03 +02:00
Andrey Bash 94bf9e5870 perf(gfx906): add guarded opt-in query LDS swizzle for batched QSA
Preserve FP operation order and invert a bijective shared-query layout at its vector loads. Limit the path to the current gfx906 wave64 device, INT8 pools and warp scoring. Default remains off.

Add deterministic GPU parity/timing and host-guard probes, plus scoped component and model receipts. The clean upstream build passed both-card byte checks; model gains are recorded separately on the frozen experimental stack.
2026-10-09 16:48:03 +02:00
Niko1221andmaxious 3540500232 sycl: the PCIe probe's pinned source (PR #1669) only on a layer split; a single card keeps the pageable source
Pinned memory reads the link 1.7x faster (A750 10.3 -> 17.2 GB/s, B70 13.6 -> 25.6), which moves the derived default pcie_frac
(A750 0.28 -> 0.47) and cost 10% of decode there (median 12.07 -> 10.67 tok/s, 4 of 5 interleaved pairs slower). The B70 is unchanged
(0.37 -> 0.55, 31.4 vs 31.4). The fault the PR fixes is a layer split's.

Co-authored-by: maxious <[email protected]>
2026-10-09 16:45:52 +02:00
Niko1221 a084fd2d72 serve: opt-in prevent_sleep keeps Windows awake while a request is running or queued (#1727) 2026-10-09 16:45:37 +02:00
Niko1221 392841e586 bench: COMMUNITY.md rows and the docs list entry for #1722 #1725 #1726; AI-tool mentions removed from the #1722 and #1726 READMEs 2026-10-09 16:44:46 +02:00
mgeldi 3971f78c98 bench: RTX 5090 Swift IQ3_XXS 1M tuning and native NVMe restore (PR #1722) 2026-10-09 16:44:46 +02:00
libratechw c82d2c16f6 bench: community report, RTX 3060 12 GB, Flash-Next Q2_0/IQ2_XS/IQ3_XXS/IQ3_S, engine 0.1.41 (PR #1725) 2026-10-09 16:44:46 +02:00
Enky 31946c78b9 bench: community report, RTX 5080 16 GB + Xeon w7-2475X, UD-IQ4_XS at a 786K context (YaRN x3), engine 0.1.41 (PR #1726) 2026-10-09 16:44:46 +02:00
Katsumi Takeuchi 9310d3bdf8 sycl serve: _pump routes --batch slot lines (BT/BDONE) and ends on a fatal line, as serve/server.py
server_intel.py replaces StrataEngine._pump to keep LP lines beside their token, but its copy predates --batch: the
slots' BT / BDONE lines went to the main queue, so with "parallel" every request stopped after one token. It also
missed the fatal-line handling and last_err. Both are now as in serve/server.py's _pump.
2026-10-09 16:44:40 +02:00
Katsumi Takeuchi 10be33ff9a sycl: with STRATA_VERIFY_NO_HOST and a mirror, adaptive swaps are off (they left experts in neither)
The host mirror holds the experts missing from VRAM at the start. An adaptive swap moves an expert that was resident
then out of VRAM, so it is in neither; the device plan then waits for the host to serve it, and with
STRATA_VERIFY_NO_HOST (the xe default in strata-sycl.sh) nobody does. Seen after a --batch window, whose host path
counts the misses the tier swaps on: the next single request on one card gave token 0 ("!!!!") at 0.8-1.3 tok/s.
The start already refuses experts in neither under NO_HOST; this keeps that true after the start.
2026-10-09 16:44:39 +02:00
Eli Elgaev e910e41034 dequant: coalesced Q8_0 path (four threads per block, 16-byte stores)
The prompt path dequantizes each dense Q8_0 projection (GDN qkv/gate/out,
QSA q/k/v/out, shared expert, hc) into the GEMM scratch before cuBLAS.
dequant_kernel runs one thread per 32-value block, so a warp's 2-byte
stores land 64 bytes apart: about 70 GB/s of writes on a V100.

dequant_q8_0_kernel gives each block to four threads, 8 values each,
written with 16-byte stores: a warp reads 8 consecutive blocks and writes
512 consecutive bytes. The arithmetic per value is unchanged (d * q), so
the output is the same bits. It is used for Q8_0 when the output rows are
contiguous (ld == cols) and the pointers are aligned (blocks to 2 bytes,
output to 16); everything else takes dequant_kernel as before.

V100-SXM2-16GB, every 2-D Q8_0 tensor of Qwen3.8-Flash-Next UD-Q4_K_XL
(498 tensors), each call timed on its own:

  attn_qkv 10240 x 2560, FP16   0.740 -> 0.106 ms   (70 -> 495 GB/s written)
  attn_q   12288 x 2560, FP16   0.887 -> 0.126 ms
  ssm_out   2560 x 6144, FP16   0.451 -> 0.065 ms
  all 498 tensors, FP16        138.5 -> 20.4 ms
  all 498 tensors, BF16        140.8 -> 20.5 ms
  all 498 tensors, FP32        190.9 -> 35.4 ms

An FNV-1a hash over all outputs is equal before and after for FP32, BF16
and FP16. Whole prompts on 4x V100 (where this was first used), 4,096-token
chunks: 18K 4,537 -> 4,708, 65K 6,540 -> 6,766, 123K 6,783 -> 6,985 tok/s.
2026-10-09 16:43:30 +02:00
bossman aab7e30d01 cpu: iq_avx1 - IQ4_XS gate/up stays on ggml's generic dot (the 128-bit grid kernel measures 0.7-1.0x against its auto-vectorized generic on an E5-2470 v2; the kernel stays built and parity-tested, only the dispatch declines it) 2026-10-09 16:29:59 +02:00
bossman dd7b0b5f07 cpu: AVX1 128-bit expert kernels (iq_avx1) - the i-quant rows on CPUs with no AVX2
The whole i-quant family (IQ3_XXS, IQ3_S, IQ2_S, IQ4_XS gate/up, IQ4_NL down) and the
fork's Q2_0 down have x86 dots only under __AVX2__ in ggml-cpu; on an AVX1 floor
(STRATA_ISA_FLOOR=avx, issues #595/#623/#1375: Xeon E5 v1-v2, and AVX-512-only packs)
the CPU-computed experts fall to the scalar _generic dots.  iq_avx1.cpp is the older
CPUs' copy of iq_avx2.cpp in the kq_avx1.cpp pattern: a separate TU compiled
-msse4.2 -mavx -mno-avx2 -mno-fma -mno-f16c, gated at runtime behind cpu_avx1_ok() and
only when AVX2 is absent, so no mainstream path changes.  128-bit integer lanes
(pshufb/sign/maddubs are SSSE3), 256-bit float accumulation, software half-to-float,
no gathers (the Fmt32 non-gather path), one IEEE mul per value like the AVX-2 copy.
Plus q8k_quant_avx1, the byte-identical Q8_K activation quantizer in SSE4.2/AVX1.

iq_avx1_parity: all six formats, nt=1..8, bit-exact across widths (#152) and within
1e-5 aggregate of ggml's generic dots; q8k_quant_parity now checks the AVX1 copy too.
gu rows take the AVX-2 mt_min policy (STRATA_IQ_MT_MIN), down rows always; kill switch
STRATA_NO_IQ128=1.  E5-2470 v2 (Ivy Bridge): 1.2-3.1x per expert at nt>=2, 2.0x on the
Q8_K quantizer; serve prefill +36-41%, warm decode 22-25 -> 46 tok/s.

Known pre-existing gap found while testing: pool_tasks_test SIGILLs on non-AVX-512
machines even on a clean tree (it predates the ISA floor and is not gated).
2026-10-09 16:29:59 +02:00
bossman 7b14a443ee cpu: q2_avx1 - the Q2_0 rows and the Q8_1 quantizer for CPUs with AVX but no AVX2 (fixes #1699)
q2_rows_any and act_quant_any had two rungs, AVX-512 and AVX2, and the else branch
checked nothing: an AVX-only CPU was handed AVX2 code and died with SIGILL (found on
an E5-2470 v2; pool_tasks_test and q2_bitplane_parity crash on a clean tree, and any
future caller inherits the trap).  The third rung: q2_avx1.cpp in the iq_avx1.cpp
containment pattern (-msse4.2 -mavx -mno-avx2 -mno-fma -mno-f16c, a #error guard,
called only behind cpu_avx1_ok() && !cpu_avx2_ok()).  128-bit integer lanes (maddubs,
madd), 256-bit float accumulation, software fp16 scales; the legacy kernel keeps the
AVX-2 kernel's lane pairing and reduction tree (its FMA becomes a mul+add, last bits),
and the bit-plane kernel and its image builder port as-is (pure SSE).  The AVX-2 and
AVX-512 paths are untouched.  q2_bitplane_parity now runs on the AVX-2 pair where the
CPU has AVX2 and on the AVX1 pair below it, and skips cleanly without AVX.

E5-2470 v2: pool_tasks_test 168 bitwise comparisons pass, q2_bitplane_parity PASS -
both SIGILL'd before.
2026-10-09 16:29:59 +02:00
Niko1221 72a67c8e90 sycl: the grouped XMX kernels (sub-group 8, joint_matrix 8x8x16) are Arc A-series only: an AOT build for Xe2 / Xe-HPC leaves them out (STRATA_NO_DG2_XMX), STRATA_PF_XMX is ignored with a message on Battlemage, and xmx_group_parity skips there
PR #1711 as posted made the Arc Pro B70 AOT build (bmg-g31) fail to link: the backend compiler rejects the sub-group 8 kernel.
2026-10-09 16:01:57 +02:00
Niko1221 115e8ce334 Windows HIP: the auto cache's 2560 MiB reserve floor also covers 16 GB cards (they report 15.9 GiB) (#1709; Windows HIP test pending) 2026-10-09 15:50:58 +02:00
W1nge 268647d2ec prefill: take expert blobs from a helper's VRAM cache when it holds them
A helper GPU's expert cache already holds a copy of the blobs the prompt path
streams, and the helpers compute nothing while a prompt is read on a serial
request path. Add RemotePrefillSource, an ExpertSource that wraps the file
source and asks the helpers first: a blob a helper holds is copied
cache-to-host on that helper's own stream, misses are merged into one call to
the fallback's existing batched reads, and every other decision is forwarded
unchanged.

Opt-in through STRATA_PREFILL_HELPER_SOURCE; it applies only where the serve
path is serial (one GPU, no layer split, no batch slots) and the prompt path
streams every expert the main card does not hold. Same-day A/B on an
11-request fixture: 91.5 s -> 65.7 s of prefill (-28.3%) and 92.1 GB (64.8%)
fewer bytes read from the files.
2026-10-09 15:49:23 +02:00
Niko1221 acee40a597 serve: a sampling field sent as a numeric string is read and used, not a 400; only values that cannot be read as the right type are refused (#1663) 2026-10-09 15:43:57 +02:00
Niko1221 40bdd86d6d serve: content parts the server does not read (tool_reference, document, search_result, server tool blocks, file, audio) are left out as before, with a debug line, not a 400 (#1664)
Real clients send them next to text the model can answer from.
2026-10-09 15:37:32 +02:00
Niko1221 a6ae247561 serve: a reasoning_effort key in the run config sets the thinking level of requests that name none; --reasoning-effort in args is lifted into it with a note (#1641) 2026-10-09 15:36:37 +02:00
Niko1221 ccd0062fa8 bench: drop the tool-attribution line from the 2x RTX 3080 report (the numbers are the submitter's) 2026-10-09 15:36:37 +02:00
Niko1221 5267eb6565 docs: COMMUNITY_BENCHMARKS.md lists the three new reports once, after the existing ones 2026-10-09 15:36:36 +02:00
Niko1221 4ed0ed68d7 bench: COMMUNITY.md rows for the community folders ported from #1638 #1647 #1655 #1681 #1692 #1695 #1716 #1717 #1657 #1640; the 6900 XT 0.1.40.1 row now has the numbers #1657 added 2026-10-09 15:36:36 +02:00
zihaomu 22b3b80970 bench: single R9700 performance report (PR #1638) 2026-10-09 15:36:36 +02:00
noon-at-cgn 9a06a54aa8 bench: community report, 2x RTX 3080 20 GB, UD-Q4_K_XL, 2 x 262K lanes in a 512K pool (PR #1640) 2026-10-09 15:36:36 +02:00
Mitsuasa513 3c2eb23a00 bench: community report, a Tesla V100 (sm_70) confirms the opt-in Volta decode kernels (PR #1647) 2026-10-09 15:36:36 +02:00
Cass 2d49e6178d bench: community report, 4x RX 7900 XT (gfx1100), Flash-Next IQ3_S at 262K, five opt-ins off vs on, engine 0.1.41 (PR #1655) 2026-10-09 15:36:36 +02:00
Vop3p fed30695a1 bench: community report, 2x RX 6900 XT (gfx1030) + Ryzen 5 5600X, IQ3_S at 131K on 0.1.41 (PR #1657) 2026-10-09 15:36:36 +02:00
KarlGrier c8250a09b4 bench: community report, RTX 5080 16 GB + RTX 4060 Ti 8 GB (helper cache), IQ3_S at 4K / 32K / 128K on 0.1.41 (PR #1681) 2026-10-09 15:36:36 +02:00
David 5d7b7b6719 bench: existing RoPE vs YaRN at 512K and 1M, with setup guide (PR #1692) 2026-10-09 15:36:36 +02:00
Your Name a43b94c4c4 bench: P4 prefill 78.8 -> 114.8 tok/s with existing auto sizing; expert reuse roofline (PR #1695) 2026-10-09 15:36:36 +02:00
nsbradley88 257b6c2b4c bench: community report, Strix Halo laptop (Radeon 8060S, 70 W), v0.1.41, UD-IQ4_XS (PR #1716) 2026-10-09 15:36:36 +02:00
libratechw 1fe02e5de2 bench: community report, Radeon 8060S (gfx1151) on a Ryzen AI Max+ 395, UD-IQ4_XS, engine 0.1.41 (PR #1717) 2026-10-09 15:36:36 +02:00
Moge800 e651c0cd16 docs: improve Japanese README wording and readability 2026-10-09 15:36:36 +02:00
nekomario28 cc75882480 docs: STRATA_PREFILL_CPU_SHARE=auto is not repeatable run to run (#1684)
The share is picked from wall-clock timings, so the same short-prompt
request can split the experts differently each run and the answer moves
in its last bits - a resumed conversation's re-read need not match its
first (31 of 1,536 teacher-forced argmax tokens differed in the
reporter's test).  A fixed share repeats; 0 gives a run without it
byte-for-byte.
2026-10-09 15:36:36 +02:00
Niko1221 92507aac38 setup: on a unified-memory APU write the expert cache as a number that leaves the OS its RAM, not auto (#1715) 2026-10-09 15:24:07 +02:00
LeGeRyChEeSe 66aaed2119 tools/agent_turns_bench.py: decode A/B on coding-agent turns over this repository, per-turn ratios
Five simulated agent sessions over the repository's own files (system prompt, four tools, real files as tool
results, at most 2000 lines per read): 23 turns from ~2.5K to ~100K tokens, consecutive turns sharing their prefix.
Arms as ab_engine.py takes them; rounds mirror the arm order on fresh servers; prints the median of the per-turn
decode ratios to the first arm and the turns it won. Nothing personal and nothing downloaded.
2026-10-09 15:23:32 +02:00
LeGeRyChEeSe 65268d9961 STRATA_ROUTE_TAIL_SKIP=R: missed experts routed only at rank >= R are neither copied nor computed
A missed expert that every token of the verify window routes at rank R or lower in weight contributes zero
(no renormalisation): less PCIe traffic and CPU expert work. Counted in the request log. Opt-in, changes
the output.
2026-10-09 15:23:32 +02:00
LeGeRyChEeSe 2de9ae061f STRATA_ROUTE_PRIOR=lambda: cache-conditioned routing in verify windows (opt-in, changes the output)
The top-10 is picked again with a logit bonus lambda for the experts resident in VRAM; the weights stay the
router's probabilities renormalised over the 10 picked. After 'Mixture of Cache-Conditional Experts'
(arXiv 2412.00099), without training. Complements STRATA_ROUTE_RESIDENT (a tail swap within a margin):
every rank can move, by an amount the logits decide.
2026-10-09 15:23:32 +02:00
sergiywith f04ab568fe Resident RAM mode on Windows: the GPU cache's fill and the RAM copy read experts.bin unbuffered, the view closed meanwhile
With --resident-experts (setup's low-RAM mode when the experts the GPU does not hold fit the RAM) the start copies
the profile fill of the GPU cache and the RAM copy (pin_cache_complement) through the mapped view of experts.bin.
Every page it touches sits in the process working set beside the page-locked copy until #467's trim after the fill
and the VirtualUnlock after the copy: on a PC short of RAM that pages other memory out, and the copies run at the
page-fault rate (~0.85 GB/s here).

FileExpertSource::begin_startup_unbuffered closes the view, the section and the buffered handle, opens the
unbuffered handles (open_direct) and reads the first blob both ways; if the open fails or the two differ, the view is
mapped again and the start copies through it as before.  The fill (prefetch_pairs) and the copy (pin_cache_complement)
then take their existing unbuffered paths (read_direct: requests merged up to 32 MiB).  end_startup_unbuffered maps
the view again (map_view, the size checked) and closes the unbuffered handles and the stage buffers, so the file tier
is chosen as before.  Same bytes in the same places.

Windows and experts.bin only: the call is under _WIN32, and the GGUF in place, an unbuffered file tier and runs
without a RAM copy are unchanged.  STRATA_STARTUP_UNBUFFERED=0 gives the mapped copy back; STRATA_VERIFY_COMPLEMENT=1
prints checksums of the filled cache slots and of the RAM copy.  pin_cache_complement no longer VirtualUnlocks the
range of a closed view.

IQ2_XS (33 GiB experts.bin), RTX 5090, Samsung 9100 PRO, --resident-experts --expert-cache auto (~15,200 slots, a
16.4 GiB RAM copy), interleaved against fb58e0db:
- 21 GiB available (a RAM ballast; #467's 32 GB PC had 20.7 GiB), 6 pairs: ready 42.5 -> 9.6 s (chat), 40.7 -> 10.2 s
  (32K); the fill 842 -> 6,914 MB/s, the copy 13.7 -> 3.3 s; lowest available 0.25 -> 4.5 GiB; pages written out
  ~60,000 -> 579-3,340; 3 of 12 base starts held available RAM under 1 GiB for 2 s, none of 12 here.
- 128 GB: peak working set 32.6 -> 18.3 GiB; a restart with the file cached 13.6 -> 8.3 s; the 32K prompt and decode
  unchanged.
- Same bits: p2k 200 tokens and first-token logits, 32K 64 / 256 tokens, as fb58e0db; STRATA_VERIFY_COMPLEMENT=1
  checksums identical to STRATA_STARTUP_UNBUFFERED=0 (12,595 and 15,438 slots).
2026-10-09 15:23:14 +02:00
Niko1221andRealScaniX 1a62f6df6e serve: a tool set that comes back in another order keeps the order it was first seen in, so the prompt prefix and its cache survive (#1624)
Based on RealScaniX's sorted-by-name change; the first request of a set stays byte for byte as the client sent it.

Co-authored-by: RealScaniX <[email protected]>
2026-10-09 15:23:02 +02:00
W1nge 29d93c6407 test: keep vision device isolation independent of local GPU order 2026-10-09 15:22:14 +02:00
chaog992 efaa87aba2 tools tests: align test_setup_risk's WSL mock with install() (the KV-streaming rule test failed inside WSL)
RulesAgreeWithTheWrittenConfig.test_the_streaming_rule_is_the_flag_setup_writes runs setup.main() through
test_setup_golden's install(), which mocks is_wsl() to False, and then asked setup.kv_streaming_wanted() outside
that mock. Inside WSL the real is_wsl() is True, so the rule answered "no streaming" while the config setup wrote
for the mocked PC has --kv-resident, and the test failed at its first 131072 case (Q2_0, int8, auto). With is_wsl()
the same on both sides, setup and the rule agree in all 54 cases, inside WSL and out.

The test method now runs under the same is_wsl mock as install(), the test-only fix the unsloth tests got for #974
(d1ec32b). setup.py is unchanged; where is_wsl() is already False (Windows, native Linux) the test runs as before.

Tests: python3 tools/test_setup_risk.py (WSL2, Python 3.12.3: was FAILED (failures=1), now 40 tests OK); the other
27 tools/test_setup_*.py files give the same results as before.
2026-10-09 15:22:14 +02:00
hash13 bafc8dd556 monitor: show prefill rate in recent requests 2026-10-09 15:22:13 +02:00
zeevo 95452866b2 add a prometheus metrics endpoint 2026-10-09 15:22:01 +02:00
midagedev 1f6958c13b serve: a sampling field of the wrong type is a 400 that names it
A request with "temperature": "0.7", "top_k": 40.0 or "seed": true was
answered 200 and sampled greedy: the value replaced the configured
default in run()'s merge, then sampling_keys dropped it as not a number.
check_request_sampling refuses it by name before anything is sent, as
POST /settings and the run config already do. Only types are checked;
ranges, null fields and in-type values behave as before.
2026-10-09 15:21:45 +02:00
Niko1221 fbc60ee157 logit_bias: leave the contributor's raw receipts out of the tree, point the doc at the reproduction script 2026-10-09 15:21:38 +02:00
midagedev 5804094b2f serve: a content part the server does not read is a 400 on chat and messages, as on responses
A chat "file" or "input_audio" part, and a messages "document" block
(also inside a tool_result), were dropped without a word: the request got
a 200 and the model answered about a file it never saw. _check_parts
refuses them while the request is converted, naming the type and where
it sits, with /v1/responses' wording. Text, images, thinking, tool_use
and tool_result read as before; redacted_thinking is still dropped; a
chat "refusal" part reads as its text, as on /v1/responses.
2026-10-09 15:21:32 +02:00
David 18e7487e52 docs: demonstrate Hello token bias with measured before and after requests 2026-10-09 15:21:30 +02:00
David dcc6f756cc fix(api): apply request logit biases in CUDA and HIP sampling 2026-10-09 15:21:29 +02:00
Niko1221 52f548f45a server: death_note scans the last 256 KB for the watchdog line, so a long verify trace cannot hide it (#1705) 2026-10-09 15:20:31 +02:00
CC-David-CC c81f99849e fix: defer pipelined one-token commits to the host verdict 2026-10-09 15:20:28 +02:00
ruibeikaa dc9b219944 prefill: let native-layout packs take STRATA_MMQ_RESIDENT_SORT_NE
The row-count sort of a fully resident layer's experts skipped native-layout packs, though slot/src/off are
built the same way for both layouts. 4x GP100 (sm_60): 1.2K prompt 7.27 -> 5.32 s, 35K 63.3 -> 54.1 s, the
same bits. The switch stays opt-in.
2026-10-09 15:20:05 +02:00
sergiywith d0336684e5 kv-grow: kvg_trim uploads the residency table with res_put, which waits for the copy
kvg_trim refilled the slots the K/V gave back and then uploaded host_res with a plain cudaMemcpy from pageable memory.
That returns once the table is staged, before its DMA lands, and nothing after it waited for stream 0: the next
request's session_zero syncs main_stream only, and a prompt part of --short-read tokens or fewer goes straight to
ver.run, whose graph reads hits_.d_res on the verifier's cudaStreamNonBlocking stream. res_put (#550) is the upload
every other residency change uses: the same copy, then cudaStreamSynchronize(nullptr).

kvg_ensure's upload needs nothing: qsa_kv_elastic_grow ends in cudaDeviceSynchronize before anything reads the table.
Only --kv-grow (opt-in) runs kvg_trim, so the default path is unchanged.
2026-10-09 15:20:05 +02:00
nekomario28 14b9dd2e52 serve: keep-alive while a tool call's held-back argument is generated (#1666)
With stream_tools an array or object argument is held until it is whole,
so a large one (a measured 66 KB edit_file call) left the stream silent
for minutes while tokens were generated - a client's idle timeout then
ended a healthy request (Forge's is 120 s).

The token loop now tracks when anything last went out and sends the
engine's own keep-alive ping after KEEPALIVE_S (10 s) of silence - the
same ping the quiet-engine heartbeat sends, so clients already ignore it.
The reporter's reproduction measured a 27.8 s longest gap on a 7 KB
array (minutes on the 66 KB one); with this change it is 10.0 s and the
call arrives intact.

Change and live measurements from the issue; adds the ping-cadence unit
test (a patched clock over a held-back <tool_call>).
2026-10-09 15:19:48 +02:00
Niko1221andjkuepker 3a55dfd253 HIP: STRATA_HEAD_MIX_MULTI and STRATA_ONE_TOKEN_COMMIT as opt-ins (=1), default stays off (#1479)
Patch from the issue by jkuepker.

Co-authored-by: jkuepker <[email protected]>
2026-10-09 15:19:04 +02:00
Niko1221 00b14af827 sycl: #1710 stays opt-in: STRATA_SYCL_A770_FAST=1 gates the fast router, the paired payload stores and the paired mapped read; plain payload stores (i915, stepped window) are never paired
The default router, publish and copy_or_zero kernels are the f-142 ones. The probes set the variable (and
STRATA_DOORBELL_CHECK=1 for publish_res, which copies x only on a miss otherwise).
2026-10-09 15:18:03 +02:00
duncanmcqueen 174c74c07f sycl: add experimental A770 router and mapped-transfer tuning
Port the fork's fast router with a bracketed reciprocal and serial fallback. Add bitwise comparisons against the existing router, including ties, NaN tails and the forced serial path. On the A770's 256-expert/top-10 shape, measured host submission plus execution fell from 395 to 119 us per call.

Pair activation payload stores and mapped result reads into 64-bit uncached transfers, preserving the checksum and unaligned/odd-tail fallback. Register router and payload probes with CTest. The two probes, elementwise_parity and router_top10_parity passed on the A770.

Add the fork's opt-in host-clock stage profiler for further investigation. Profiling remains experimental: enabled runs encountered a payload-arrival timeout, and review flagged concurrent atomic host writes versus non-atomic device clock reads as a synchronization concern. The profiler is not needed for normal inference.

With CPU-computed misses, cache borrowing and an 8.98 GiB VRAM cache, unprofiled Coder IQ1_M runs generated 128 tokens at 11.9-13.5 tok/s and returned final arithmetic answer 41. These runs retained 128K context capacity but used short prompts; the 20 tok/s goal has not been achieved.
2026-10-09 15:17:50 +02:00
midagedev e85567b617 generate, mtp: near the context's end the spec window and its drafts stay inside it
The admission checks prompt + max_new <= max_context, but the spec loop
opened every window at S_mtp rows, so a request that fits ended with
"ran out of context" (rc 2) once p + S passed the end, and printed
nothing. The window now takes only the rows the context still holds.
MtpDrafter::draft wrote draft j at cell p + a + j for every step, past
max_cells near the end, where the page table has no entry; it now drafts
only the cells that exist. Far from the end nothing changes.
2026-10-09 15:17:33 +02:00
saikiran-rs 730700126b verify: initialize remote expert helper state in both batch paths 2026-10-09 15:17:33 +02:00
nekomario28 092fc3c5c6 setup: pass CC/CXX/CUDAHOSTCXX to the CUDA build's cmake (#1645)
CUDA 12 refuses a GCC newer than 14, and the docs already tell the user
to run CXX=g++-14 ./setup.sh - but the environment only reaches CMake on
a build folder's FIRST configure, so on a retry (or the encoder's own
folder) the flag never landed: the reporter's run died at the last step.

host_compiler_defs() maps them onto -DCMAKE_C_COMPILER /
-DCMAKE_CXX_COMPILER / -DCMAKE_CUDA_HOST_COMPILER for the engine and the
CUDA image encoder; CUDAHOSTCXX defaults to CXX, as CMake's does.  A
C++-only build (the CPU encoder) is not given the CUDA one.
2026-10-09 15:17:13 +02:00
noon-at-cgn 2eafe7f90d docs: the adaptive tier in batch windows - measured on 2x RTX 3080 20 GB, tier on, blocking, frozen and off 2026-10-09 15:16:22 +02:00
Andy Slater c2d008bc80 sycl: stepped verify window - CPU experts served between graph segments, cached host buffers and plain payload stores
STRATA_VERIFY_STEPPED=1 (Arc A-series): the doorbell window is captured as graph segments ending at each layer's
publish; the host serves the CPU experts between segments, pipelined with the plan copy + VRAM groups on a helper
thread pinned to the host thread's SMT sibling. No kernel then reads a host store while it runs and no host reads a
kernel store while it runs, so the window's mapped buffers are ordinary host USM and the publishes use plain stores
(STRATA_VERIFY_HOST_UNCACHED=1 keeps both uncached). A770, Coder IQ1_M, 8K prompt: GPU wait per window ~70 -> ~43 ms,
decode 22.8-25.0 -> 31.8-32.5 tok/s; 32K 27.2 tok/s.
2026-10-09 15:16:22 +02:00
noon-at-cgn b5d520313a batch-groups: --adapt-async 1 also keeps the 0.1.41 auto default at one group (said once)
The pipelined slot groups never run the adaptive tier, so --adapt-async 1 with --batch on a layer split and no
--batch-groups resolves auto to one group, like --batch-mtp (batch_groups.hpp, batch_groups_test); --batch-groups G
or auto given on the command line is honoured and turns --adapt-async off through the existing gate.
2026-10-09 15:16:22 +02:00
noon-at-cgn 495dc43690 batch: tick the asynchronous adaptive tier (--adapt-async 1) between batch windows; --adapt-min-gain
batch_step advances the async round one step per window (never between a prompt's chunks), commands drain the round in
flight before they run, a round with nothing to swap ends at its first step, and --adapt-min-gain sets the bar a swap
must clear (default 1.5, as before) for both tiers. --adapt-async is no longer refused with --batch slots (still with
--batch-groups).
2026-10-09 15:16:22 +02:00
noon-at-cgn e989c67194 batch: the adaptive VRAM expert tier keeps adapting in batch windows; per-stage timing in the strata batch line
batch_step now runs adapt() every --adapt-every windows on a thread beside the commit and the drafts, lands the swaps
(apply_pending) before the window ends, and pauses while a prompt holds a loan of cache slots. The timing line adds the
adaptive tier's rounds/swaps and wait, each stage's GPU-reach wait and pool time, and the VRAM/PCIe/CPU entry split.
2026-10-09 15:16:22 +02:00
noon-at-cgn de71d7bef4 docs: batch MTP on a layer split - measured on 2x RTX 3080 20 GB (0.1.41 build and the earlier 0.1.40.3 runs), with what was not measured 2026-10-09 15:16:16 +02:00
noon-at-cgn f90bec6f44 batch-mtp: not with helper expert caches on a layer split (not run); docs say how it meets --batch-groups
The 0.1.41 default --batch-groups auto, the pipelined path's lack of slot drafts and the helper-cache refusal are
written down in docs/BATCHING.md (the limits list and the --batch-mtp paragraph).
2026-10-09 15:16:16 +02:00
noon-at-cgn cb6c549c8a batch-groups: the 0.1.41 auto default gives way to --batch-mtp (one group, said once)
--batch-groups auto became the default for a layer split with --batch >= 2, but the --batch-mtp gate ran before
the groups were resolved: it saw 1 group, kept batch-MTP on, and the groups were then raised to 2 - pipelined groups
do not run the slots' MTP drafts (#1413). The resolution is now a pure helper (include/strata/program/batch_groups.hpp,
CPU test src/program/batch_groups_test.cpp): with --batch-mtp and no --batch-groups the default is one group and
the log says why; --batch-groups N (N > 1) or --batch-groups auto given on the command line pipelines the groups and
turns --batch-mtp off, as before.
2026-10-09 15:16:16 +02:00
noon-at-cgn 70fc0ba46d batch-mtp: log the accepted proposals in the batch summary 2026-10-09 15:16:16 +02:00
noon-at-cgn 48f2c47d54 batch-mtp: work with a layer split (slot drafters on the last stage's GPU, grouped slot rows through every stage)
- generate.cpp: --batch-mtp accepted with a layer split on separate GPUs (refused with --batch-groups); slot drafters
  load from the last stage's slot sessions under the last stage's device guard and bind to its weights/head; their
  residual buffers, row copy (async on the drafter's stream) and draft-KV copies run on the drafter's device; every
  stage verifier gets the 8-row window size and the batch graph limit; slot-drafter logits are not charged to CUDA0.
- verify.cpp: stage_batch's layout rules move to include/strata/core/batch_rows.hpp (CPU test batch_rows_test); grouped
  rows are allowed on a stage that hands rows on, the hand-off bound is kVerifyMaxT rows.
- docs/BATCHING.md, --batch-mtp help text.
2026-10-09 15:16:16 +02:00
sergiywith c396dc9c39 setup: the page-file warning advises a fixed size, and says why when Windows is set to grow the file
Step 1 warns below 4 GB of page file (issue #60) and advised "Set it to System managed". In #60 that did not help:
"System managed" and 4096-32768 MB still ran out of commit, and a fixed 64 GB worked - a page file Windows grows on
demand may not grow in time while WDDM charges the card's memory to the commit.

- The advice is a fixed size: Custom size, the same initial and maximum size, 65536 MB (PAGE_FILE_FIXED_MB).
- page_file_setting() reads the Virtual memory setting (PagingFiles; "?:" is the drive in ExistingPageFiles). When a
  file is system-managed or its initial size is below its maximum, the warning says that it grows on demand and that
  a growing page file may not keep up (#60).
- page_file_gb() still counts only what is there for sure: the commit limit minus RAM, as before, or a file's
  configured initial size when that is larger (within its drive's free space). Never the maximum or what a
  system-managed file may grow to. The 4 GB threshold is unchanged.
- docs/AI_SETUP.md and docs/DETAILS.md give the same advice.
- tools/test_setup_page_file.py: the parsing, the reading, the advice, and step 1 under --check on the golden
  harness's 32 GB PC (registry, drives and GlobalMemoryStatusEx mocked; runs on Linux too).
2026-10-09 15:16:03 +02:00
SuperLee 52c743d550 mcp: let the swift family take IQ3_S too; docs
The MCP server's FALLBACK_MODELS carries its own copy of the model
table, so #1688's setup.py fix leaves the MCP install path rejecting
--family swift --model IQ3_S. Drop the families gate there as well,
flip the validation test, and update the three doc lines that say
IQ3_S is original-model-only.

Measured on an RTX 3090 pair (engine 0.1.40.3, Swift IQ3_S, 200K
context, MTP spec on): 91-137 tok/s decode, expert cache hit rate
96-99%, KV streaming 98.6-99.6% VRAM hits.
2026-10-09 15:15:53 +02:00
nekomario28 614053a4d7 setup: let the swift family take IQ3_S (#1651)
UkisAI published a Swift 1.5 IQ3_S tier on 2026-10-08; the
"Swift 1.5 has no IQ3_S" restriction is stale.  Both shards match the
family's file pattern exactly (names/sizes/SHA-256 verified against the
Hub listing), the shard count and the shared BF16 mmproj are unchanged,
so it takes the same path the other Swift tiers already use.
2026-10-09 15:15:53 +02:00
Andy Slater 0e583707a5 sycl: grouped XMX passes the decoder thread that writes [o, o+8) for the run-of-8 IQ decoders
The IQ2/IQ3/IQ4_NL/IQ1_M decoders now write values [8 tid, 8 tid + 8) of a superblock (coalesced runs, as Q2_0
already did); iq_xmx_grouped still passed the IQ4_XS-style (sub-run, 32-block) index for them, so the B tile held
the wrong values (xmx_group_parity relerr 7-17 on types 18/20/21/22).  All six types back to ~1e-6.
2026-10-09 15:15:25 +02:00
Andy Slater ce60cc373f sycl/A770 prefill: grouped XMX expert GEMM (opt-in STRATA_PF_XMX=1/2)
- xmx_moe: joint_matrix FP16 GEMM over a group of up to 16 experts per launch (grf 256).
  Standalone vs oneMKL on uneven routing: down 1.75-6x, gate/up 0.8-3x (oneMKL kept above
  STRATA_PF_XMX_GU_MAX=250 tokens/expert).
- STRATA_PF_XMX=2: iq_xmx_grouped decodes IQ3_XXS/IQ3_S/IQ2_S/IQ4_XS gate/up and IQ4_NL/Q2_0
  down rows in-kernel (no FP16 staging); xmx_group_parity: rel err < 2e-6 vs dequant+oneMKL.
- End to end (8K/32K prompt, A770): +2-5% only; default off.
2026-10-09 15:15:25 +02:00
W1nge 538ef3aa5b prefill gemm: run a failing FP16 shape on a fixed cuBLAS algorithm; the default one fails once a second device's context exists (issue #1650) 2026-10-09 15:15:06 +02:00
Cass 553b0b788c pipeline-windows: overlap three or more layer-split stages; HIP drafter scratch race
--pipeline-windows 2 now runs on any layer split with two or more stages. Every
stage but the last is a front stage: the guessed window follows the verified
one through them one card behind (an odd verifier and an odd hand-off on every
boundary, a pair of GDN snapshots on every front stage, commits and the undo
per stage), and only the last stage waits for the verdict. The pipelined prompt
read (--pipeline-windows 1) and --adapt-async stay two-stage.

HIP: the draft layer's per-row gr_read used the last stage's session scratch
(ss.block.gr), which that stage's verifier also uses in lm_head_mix. A chain
launched at a verdict and the next window's last stage can run on the card at
once after a guess that held, and the window's head then read a clobbered mix.
The draft layer now carves its own GrWorkspace.

Measured on 4x RX 7900 XT (Flash-Next GSQ-RCO IQ3_S, split 12,24,36, every
expert in VRAM, greedy, reasoning off): decode 59.4 -> 67.6 tok/s at 4K and
51.8 -> 65.6 at 32K. Greedy text identical to the serial loop on four prompts
with STRATA_IQ_MT_MIN=1 --pcie-frac 0 --adapt-every 0, also with every guess
forced wrong. Debug knob STRATA_PIPELINE_SPEC_DEPTH. Two-stage path not re-run
on two cards.
2026-10-09 15:14:37 +02:00
Alex Sadleir 7e940789e6 Clean up comments in generate.cpp
Removed comments regarding memory allocation issues and synchronization behavior.
2026-10-09 15:14:19 +02:00
maxious 4340476438 sycl: pin the PCIe probe's source buffer
probe_pcie_h2d_gbps: a pageable 256 MiB source made the copy engine fault
inside the driver's bounce path on a layer split (755x bcs VM_NOT_FOUND at
the fixed VA 0xffffc001ffa10000, L4 = the region never mapped for that VM,
intermittent on fresh boots, wedging the whole load in both the 0.1.40
port and 0.1.41). Pin the source (sycl::malloc_host) and free it with the
matching sycl::free. Validated: two consecutive loads reached the window
without the wedge.

All the split-fault instrumentation (guard watch, stage stamps and the
STRATA_STAGE_WAIT serial gate, terminate dumps, coredump grabbers) lives
on scratch/split-fault-instruments.
2026-10-09 15:14:19 +02:00
Katsumi Takeuchi c98902a657 sycl: MtpDrafter's queue defaults to nullptr, so a run without --mtp no longer crashes at exit
dpct turned `cudaStream_t cs_ = nullptr` into `&dpct::get_in_order_queue()`: a drafter that never loads keeps it, and
its destructor destroys the device's default queue (exit 139, sometimes UR_RESULT_ERROR_UNINITIALIZED, after the output).
Arc Pro B70 at fb58e0d without --mtp: 3 of 3 runs crashed before, 0 of 3 after; output unchanged with and without --mtp.
2026-10-09 15:14:19 +02:00
Niko1221 04dbe56145 CPU prefill share: skip the default when host memory is short, and never fail a chat on its buffers
0.1.41 report (RTX 5090, tight RAM): chats failed and the engine restarted until STRATA_PREFILL_CPU_SHARE=0.
- share_rules: default_share_memory_verdict. The default is not armed when the RAM this process can get
  (cgroup-aware) is below the share's buffers + 3 GiB, when the Windows commit left is below buffers + 4 GiB,
  or when the experts it would read are file pages the free RAM cannot keep. One log line says why and that
  STRATA_PREFILL_CPU_SHARE=auto turns it on anyway. An explicit setting is kept and only warns.
- prefill: the share's two pinned buffers are taken once at their largest (an expert takes at most MAXT
  tokens' rows); a failed allocation ends the share for the run instead of failing the request.
- unit test, docs/DETAILS.md.
2026-10-09 15:10:32 +02:00
Niko1221 65d97b0ab6 memory_guard_test: build on Windows (process.h getpid instead of unistd.h) 2026-10-09 15:00:53 +02:00
Niko1221 abbd0ea9aa sycl prefill: bytes_needed* take the src flag like the shared header (merge of the multi-GPU stream) 2026-10-09 15:00:53 +02:00
Niko1221 76790f277a test_slots: compare the relative save dir through realpath (8.3 temp dir names on Windows) 2026-10-09 14:47:49 +02:00
Niko1221 3d247dd49c Cap a thinking budget at max_tokens less an answer reserve, so the reply is not cut off while still thinking (#984) 2026-10-09 14:47:32 +02:00
Niko1221 bce19db445 Progress line shows the thinking budget, not max_tokens, while one applies (#984) 2026-10-09 14:47:31 +02:00
Niko1221 5a6ef1d768 Merge branch 'f-142-amd' into f-142
# Conflicts:
#	serve/server.py
2026-10-09 14:47:31 +02:00
Niko1221 3fb23b146a Merge branch 'f-142-intel' into f-142 2026-10-09 14:47:08 +02:00
Niko1221 43b105a753 Merge branch 'f-142-mgpu' into f-142
# Conflicts:
#	src/program/generate.cpp
2026-10-09 14:47:08 +02:00
Niko1221 e85f56209c Merge branch 'f-142-engine2' into f-142
# Conflicts:
#	src/core/conversation_memory.cpp
2026-10-09 14:46:54 +02:00
Niko1221 760007e311 Merge branch 'f-142-carry' into f-142
# Conflicts:
#	src/program/generate.cpp
2026-10-09 14:46:36 +02:00
Niko1221 f5aa356f14 Merge branch 'f-142-small' into f-142 2026-10-09 14:46:18 +02:00
Niko1221 2dc6eeae42 Merge branch 'f-1411' into f-142 2026-10-09 14:46:18 +02:00
Niko1221 b70bc033b5 sycl: plain payload stores on the i915 driver (the uncached hint is for xe); STRATA_DOORBELL_PLAIN overrides 2026-10-09 14:08:15 +02:00
Niko1221 f583670cab sycl: Alchemist own-buffers prompt path only on cards with >= 12 GiB; the payload copy stays conditional unless STRATA_DOORBELL_CHECK=1 (A750 A/B, PR #1602) 2026-10-09 13:09:03 +02:00
Niko1221 68d66b117f memory_guard_test: a .. component in the cgroup path falls back without a cap 2026-10-09 12:38:17 +02:00
Niko1221 b0531c8dad memory_guard_test: memory.current "max" without a cap falls back too 2026-10-09 12:37:43 +02:00
Niko1221 c9680eeff6 memory guard: unusable cgroup files without --memory-limit-mib fall back to MemAvailable with one warning; with the flag the sample stays unknown 2026-10-09 12:37:05 +02:00
Niko1221 399a75c8ad tools: batch_groups_cache_test.py reads the engine log only 2026-10-09 11:51:43 +02:00
Niko1221 da6f23f8e9 ple: print the direct-read probe's rate when it is fine 2026-10-09 11:21:33 +02:00
Niko1221 3157c78935 sycl: the doorbell payload checksum wait is opt-in (STRATA_DOORBELL_CHECK=1): on an Arc A750 it cost 17% of decode (#1606, PR #1602) 2026-10-09 11:15:36 +02:00
Niko1221 5025d63a22 sycl: STRATA_SYCL_JIT_STATELESS option for the JIT build's 64-bit-address flag (default ON, as in #1602) 2026-10-09 11:15:35 +02:00
Niko1221 7609cc18a1 arch_defaults gfx12: drop STRATA_HCD_EXACT (inert without the gfx1151 tuning table, and noisy); five prompt switches, measured +0.8/+1.6/+1.8% at 4K/16K/32K on an R9700 (IQ3_S) 2026-10-09 10:51:41 +02:00
Niko1221 31ffa153c6 tools: session_pressure_test.py and batch_groups_cache_test.py 2026-10-09 10:30:16 +02:00
Niko1221 22a893d0c6 Version 0.1.41.1: project version and the setup minimum engine 2026-10-09 10:24:42 +02:00
Niko1221 8bf203f56b STRATA_ROUTE_RESIDENT: per-GPU counters owned by the verifier (#1578)
The counter block was one process-wide cudaMalloc on whichever device was current at the first Verifier::init, unreachable from
the other stage's kernels on a layer split without P2P. Each verifier now takes its own device's block in init (outside graph
capture), keeps the pointer, and a launch never picks by the host thread's device; a failed allocation runs without counters.
The exit report prints one block per GPU when there are several (single GPU: the same lines as before). The reporter's rare
fault with --pipeline-windows 2 after this fix is not reproduced here (needs two GPUs).

STRATA_ROUTE_RESIDENT_MTP=1 (experimental, opt-in, changes the output): the MTP draft's router gets the same residency bias,
with the last layer's residency row as its table (single GPU).
2026-10-09 10:11:10 +02:00
Niko1221 56354964d8 Stage buffers: fence the lent-slot refill copies too (#1237)
The refills of the slots a prompt borrowed (and the VRAM-resize refill) queue hundreds of cudaMemcpyAsync out of blob_stable()
pointers, which for a GGUF source are the recycled stage buffers. With STRATA_STAGE_PIN=1 the 257th assembly overwrote a
buffer whose copy had not run: IQ3_XXS lowmem (budget 12) decoded token id 0 after a 4096-token prompt. Each queued copy now
records the fence of its buffer. exact_pp 4096 and 20000 tokens with STRATA_STAGE_PIN=1: EXACT.
2026-10-09 10:11:10 +02:00
Niko1221 0650f31670 tools: cancel_park_test.py, a cancelled request must not lose the conversation it restored (#1620) 2026-10-09 10:01:59 +02:00
Niko1221 caba015bff serve: a parallel request is never neither running nor queued (#1603)
A busy slot flag with no owner kept every later request waiting for a slot nothing
would free, and a request waiting for the fifo (a load, an image encode, /v1/vram)
was counted nowhere. Now: slots have owners (claims / BDONE waits) and an ownerless
busy flag is freed after 3 s; the request waiting for a slot is counted
(admitting) and, past the stall time, named in the log; a parallel request on a
loaded engine no longer takes the fifo, and one that waits for it stays queued.
Tests with the fake engine.
2026-10-09 03:31:47 +02:00
Niko1221 5728828a84 Windows HIP: a bare --prefill auto stops at 6144; commit-limit check at serve start (#1630, #1607)
On an RX 7900 XTX the first long prompt at 8192-token chunks charged ~70 GB of
commit and the engine died (0xC0000409); 7500 and below were flat. A bare auto
now stops at 6144 on Windows HIP builds. A chunk the user names stays, with a
warning from 8192 up.

Windows: the serve start prints the commit left and warns when the planned host
allocations plus a 4 GiB floor do not fit; the conversation cache's admission
reads min(available RAM, available commit).
2026-10-09 03:31:47 +02:00
Niko1221 eaaa237e85 CPU prefill share: not armed by default when the cache holds every expert (#1595)
With every expert resident nothing streams, so the share has nothing to take and
only moves the staged-chunk limit (+4.8% at 1,000 tokens on a V100). An explicit
STRATA_PREFILL_CPU_SHARE is honoured as before; the default is unchanged where
experts are not all resident.
2026-10-09 03:31:47 +02:00
Niko1221 1c6c82ef4f layer split auto: avoid a one- or two-layer stage (#1616)
A stage with one layer holds a handful of expert slots (a 12 GB 3060: 512), and the
prompt chunk of every card in the pipeline is clamped to what that cache can lend.
When the pick has such a stage, search again among placements that give every
stage at least three layers; keep the new pick and print both and why. With none
startable the first pick stays. Identical cards pick balanced stages and never
reach the second search.
2026-10-09 03:31:47 +02:00
Niko1221 316097d2d9 sycl: recognise a Battlemage / Alchemist card by the PCI id in a name like 'Intel(R) Graphics [0xe223]' 2026-10-09 03:31:16 +02:00
Niko1221 793ca909c1 iq_multi_parity: an independent mmvq reference for the grouped expert kernels (the old-vs-new comparison was the same kernels by default) 2026-10-09 03:31:16 +02:00
noon-at-cgn 1aa4198804 docs: replace the evidence placeholder in the memory guard section with the measured result 2026-10-09 03:28:31 +02:00
noon-at-cgn daa765524d memory guard: parking and session save/restore read the RAM the engine really has (--memory-limit-mib)
available_host_bytes() = min(MemAvailable, the room under every cgroup limit above the engine, the operator's cap).
Parking and session save/restore read MemAvailable alone: in a container whose real limit sits on a parent cgroup it
cannot see, MemAvailable is stale (about 25 GiB shown when about 1 GiB was left) and the engine was OOM-killed.

conversation_available_memory() is replaced by available_host_bytes() / sample_host_memory() (the SYCL port's
generate.cpp too). The cgroup term covers memory.max and memory.high (v2) and memory.limit_in_bytes (v1) of the
process's group and every visible ancestor, with clean inactive file cache credited; an unparsable cgroup file makes the
sample unknown (fail closed). --memory-limit-mib / STRATA_MEMORY_LIMIT_MIB names the total RAM the container or cgroup
may use. Refusals say how much was available and from which source; one start-up line reports the limit.
tests/core/memory_guard_test.cpp runs on fixture directories (CPU only). Docs: DETAILS.md.
2026-10-09 03:28:31 +02:00
tuandat3019 bbb74cfca4 serve: wait for measured RAM before retrying reclaimed SAVE 2026-10-09 03:27:49 +02:00
tuandat3019 73ed39ae8e serve: optionally reclaim host caches before SAVE admission 2026-10-09 03:27:49 +02:00
Niko1221 8824e217c4 serve: a request cancelled between two prompt windows keeps the conversation it restored as the live one (#1620) 2026-10-09 03:27:49 +02:00
Niko1221 152292b633 tests: the AVX2 warning test of the docker entrypoint 2026-10-09 03:21:46 +02:00
Hulk 71eaee17b9 spec: re-probe stale full chains when shorter chains pay
A profitable shorter chained lookup returned before the stale full-size check, so the full chain could stay excluded for good
although the policy re-probes stale sizes in the other selection paths (#1252). Check a confident stale full size before
returning the profitable shorter chain. The 300-round policy, the EMA, the header and plain lookup are unchanged; a regression
test covers the bounded re-probe spacing and the MTP confidence gate.
2026-10-09 03:18:07 +02:00
sergiywith a26fcbcdba prefill: bytes_needed sizes the MoE buffers for the layout init picks (with or without an expert source)
0.1.39 took this PR's first part with #583 (the xn / grs count, STRATA_RING_BYTES). What is left: bytes_needed
counted the fused layout's smaller MoE buffers for every caller (fused_layout(T, true)), while init takes them only
when it gets an ExpertSource (fused_layout(T, m.src != nullptr)). A prompt path without one was lent fewer bytes than
its unfused buffers take. bytes_needed, bytes_needed_no_ring and (0.1.40) bytes_needed_owned - STRATA_OWNED_PRICE=exact's
sizing - now take the same src the callers pass to init.
2026-10-09 03:17:42 +02:00
Niko1221 8bb250d846 Layer split auto: warn on a near tie between two different splits on a mixed pair (#1428)
When the best split and a different one (boundaries at least 3 layers away, startable) are within 4% predicted decode window
(STRATA_SPLIT_NEAR_TIE_PCT, 0 = off) and the cards differ in speed or link, print the other K and --layer-split K. The model
prices decode only, not the prompt. Print only: the choice is unchanged.
2026-10-09 03:17:32 +02:00
Niko1221 cec7e2b1f6 --batch on a layer split: a BYIELD is taken even when no slot is decoding
A prompt read on a layer split with no slot decoding was one run, so the BYIELD of the batch_interleave_test step 4 was
dropped at the end of the read; the long prompt kept its slot and the next BGEN failed with 'ERR BGEN: no such free slot'
(pre-existing since 0.1.40.4). That read now goes in pieces of STRATA_BATCH_YIELD_CHUNKS chunks (default 8, 0 = one run), the
same chunks a single run reads, with the yield check between pieces. batch_read_piece holds the arithmetic; batch_read_test covers it.
2026-10-09 03:17:31 +02:00
Niko1221 9f5504d8a1 tests: test_ready_timeout is skipped on Windows, where its .cmd wrapper keeps the pipe open after the kill 2026-10-09 03:17:23 +02:00
Niko1221 f71c5307c6 Stage buffers: fence a pinned buffer's queued copy before the buffer is recycled (#1237)
A page-locked stage buffer is the source of the cache fill's cudaMemcpyAsync, which the DMA reads after the call has
returned; a pageable one is staged by the driver before it returns. The pool recycles a buffer after kStageAge layers or 256
assemblies, so a 4096-token prompt read (up to ~500 distinct experts per layer) could overwrite a buffer whose copy had not run
yet: the corrupted IQ3_S decode of #1237. note_async_read(blob, stream) records an event per pinned buffer after each queued
copy (hit-path fill, peer fill, remote-expert swap); claim_stage waits for it before it hands the buffer out again.
Pageable buffers and every other source are unchanged. STRATA_STAGE_PIN stays opt-in.
2026-10-09 03:17:17 +02:00
Niko1221 8d276c1997 Data-parallel replicas (opt-in): N engines on their own card groups behind one server, requests spread least-loaded and a conversation routed back to its replica; replicas in the config, --replicas in the server and setup 2026-10-09 03:15:34 +02:00
Niko1221 438019e36a hip: gfx1152 (Radeon 860M / 840M) in the unvalidated architecture list, like gfx1150 (#1625) 2026-10-09 03:15:13 +02:00
Niko1221 df29e3b2e6 prefill: guard std::atomic::wait/notify_all with __cpp_lib_atomic_wait so a GCC 10 build spins as before #1057 (#1488) 2026-10-09 03:15:13 +02:00
Niko1221 0ff45cffb2 hip: the Q2_0 signed-zero fix also on gfx1200/gfx1201 (ROCm 7.10), where hip_q2_zero failed (#1474) 2026-10-09 03:15:13 +02:00
Niko1221 3d6793a408 tests: expert_cache_segmented_test skips on a HIP build, where --vram-elastic is CUDA-only (#1475) 2026-10-09 03:15:12 +02:00
Niko1221 9d64b93a0f docker: warn, not stop, on a host CPU without AVX2 (#1584) 2026-10-09 03:15:12 +02:00
Niko1221 224bb6f899 setup: an engine update compiles for the cards the installed models use, not the biggest one (#1485) 2026-10-09 03:15:12 +02:00
Niko1221 598df11158 setup: --gpus when starting an installed model on a PC with AMD cards and no NVIDIA ones names the AMD cards (#1594) 2026-10-09 03:15:12 +02:00
BlueKingMuch 0d613031a9 serve: a </think> the reasoning quotes stays reasoning instead of ending it (#537)
The reasoning ended at the first </think> in the output, also when the model only
quotes the tag while it thinks (`</think>`, "</think>\n\n2" from a test). The rest
of the thinking then came back as the answer, and a tool call quoted in it could
be run.

A </think> right after a quote or backtick and followed by anything but a line
break is now reasoning text; the end of the thinking is followed by a line break
and ends it as before. Until the character after such a tag arrives, the tag is
held like a partial one, so streamed and whole outputs agree. If the turn ends
right after it, it was the end of the thinking: a forced tool call still opens
there, and the #1053 retry is not triggered, as before.
2026-10-09 03:15:12 +02:00
William Charbonneau Perrier 624070c5df Conversation cache: --conversation-cache-min-tokens parks only conversations of at least N tokens
Parking happens on a switch and every parked conversation costs one of
--conversation-cache-slots whatever its length.  A client that interleaves
short side requests (an agent's tool-result follow-ups, often under 1k tokens)
with one long conversation therefore spends the cache on the side requests and,
once it is full, evicts the long one oldest-first - the reuse it wanted.

Measured on an RTX 4070 SUPER / 32 GB / Q2_0, --conversation-cache-slots 2,
one 19,198-token conversation interleaved with three ~700-token requests, then
the same conversation again:

  min 0      skip nothing, side requests park, L is evicted
             re-send L: 0 reused + 19198 read in 41,059 ms   (wall 41.4 s)
  min 12288  skip parking (703 tokens, below the 12288 minimum) x3
             re-send L: 19193 reused + 5 read in 135 ms      (wall 0.3 s, 304x)

The default 0 keeps the previous behaviour, so nothing changes without the flag:
wants() answers for the length alone and the gate stays enabled() && wants(),
exactly as the existing byte-budget and RAM-floor skips are.  The check runs
before make_room(), so a skipped park neither evicts nor stores anything, and it
returns the same non-error path those skips take.

Tests: conversation_cache_test covers the inclusive boundary, 0, a negative that
never reaches the CLI, and that a disabled cache still answers for length; both
faults were reintroduced by hand to confirm the checks fail on them.
2026-10-09 03:15:12 +02:00
Niko1221 875d6a922a serve: a request waiting for the single turn sends keep-alives, and gives its place up when cancelled (#1619) 2026-10-09 03:15:12 +02:00
Niko1221 30017336ed ple: when the default --ple-io direct measures slow at start (cold random-row probe), use mmap; an explicit direct only warns (#1425 #1549 #1629) 2026-10-09 03:14:18 +02:00
Niko1221 6c9bbc8147 sycl: spin bound for a B-series card found from the device (Windows/OpenCL has no sysfs driver): 20,000 reads, 2,000,000 when drafting without --mtp (#1473 #1397) 2026-10-09 03:14:18 +02:00
Alex Gorevski 7d771bb028 Group a launcher's allowed CPUs by physical core and split them between independent instances (serve/cpu_affinity.py, from #1436) 2026-10-09 03:10:24 +02:00
jkuepker a257e0198e docs: gfx12 (R9700) native WMMA prompt experts: switch, measurements; gfx12 in the comments
AMD_HIP.md, RDNA4 speed switches: STRATA_PF_FUSED=1 now runs the native
IQ packs' prompt experts on the int8-WMMA kernels on gfx1200 / gfx1201.
Says that it is opt-in and why (it rounds differently, so it is not an
arch default), that STRATA_PF_FUSED_NATIVE=0 turns the native path off,
the start-up line it prints, the R9700 measurements (prompt time, kernel
vs MMQ, first-token KL against the FP16 path, retrieval checks), and
what was not run (gfx1200, Windows, K-quant packs).

DETAILS.md: the STRATA_PF_FUSED=1 sentence says where the native kernels
run on HIP.  CMakeLists.txt and gfx_arch.hpp: the comments that said the
native prompt-expert kernels are gfx11 only now include gfx12 (comments
only, no build change).
2026-10-09 03:06:58 +02:00
jkuepker 7969a6ec93 HIP gfx12: native int8-WMMA prompt experts on gfx1200/gfx1201 (opt-in STRATA_PF_FUSED=1, #1277)
The native IQ / K-quant prompt-expert kernels (moe_fused_iq.cu,
native_w11_kernel) were gfx11-only; on RDNA4 a pack with
STRATA_PF_FUSED=1 silently kept the MMQ path.  They now compile and run
on gfx1200 / gfx1201.  Still opt-in: STRATA_PF_FUSED=1 stays off by
default (it changes rounding, so it is not an arch_defaults switch), and
STRATA_PF_FUSED_NATIVE=0 still turns the native kernels off.

Compile gating
- STRATA_NAT_W12 for __gfx1200__ / __gfx1201__, STRATA_NAT_WMMA for
  either family.  The kernel calls the _gfx12 spelling of the iu8 WMMA
  builtin (the un-suffixed one does not compile there).  Any other
  target keeps the trapping body.

Kernel (same arithmetic, other lane layout)
- gfx12 WMMA fragments are 8 int8 per lane and the two lane halves hold
  different k (8*hi .. 8*hi+7), not a replica as on gfx11: B and A are
  loaded as uint2 at the matching offsets, and every 16-value k step is
  still one WMMA, so the per-16 scales of the IQ2_S / IQ2_XS paths are
  untouched.
- A lane's 8 accumulator rows are consecutive (8*hi + i), so a feature's
  gate and up are in the same lane: SwiGLU needs no __shfl_xor, and the
  down tile is stored as two float4 per lane.  NW_DROW(i, hi) gives the
  row of element i for the per-row scales.

Host side
- gfx_arch_is_gfx12_wmma(): exact gfx1200 / gfx1201 match, never a
  prefix, like the gfx11 helper.
- dev_info() takes the gfx11 list or gfx12, and also checks that the
  kernel the build loaded is the real one: the trap stub (a
  gfx12-generic or SPIR-V build on a listed card) declares no LDS and
  still passes hipFuncGetAttributes and the occupancy query.
- native_supported() reads the STRATA_PF_FUSED=1 opt-in itself:
  fused::available() / requested() belong to the Q2_0 pack's
  expert_w11_kernel, which is not ported and stays false on gfx12.  Off
  gfx12 the result is what it was.  prefill.cpp fused_ring() follows:
  a native pack asks native_supported() alone (it implied enabled()
  before), so on gfx12 it gets the fused buffer layout and ring.
- Grid factor per format: gfx12 holds 3 blocks per WGP for IQ2_S / IQ2_XS
  gate/up (LDS and VGPRs), 4 for the rest.  The occupancy API cannot say
  (it models 64 KB of LDS per multiprocessor), so it is a small table;
  STRATA_PF_OCC_W=N overrides it.  gfx11 keeps 4.
- experts_native() refuses a dm that is not 16-byte aligned on gfx12
  (the float4 stores); the prompt path's buffers always are.
- moe_fused.cu compares the gfx11 list with the helper instead of the
  "gfx11" prefix (no change on any listed part).

Tests (prefill_fused_iq_test)
- Runs on gfx12 (the skip gate also asks native_supported()), covers the
  other four pairs of the Coder IQ1_M pack, S20_E sets the expert count,
  and the real-shape check of fused against MMQ is now rel RMS <= 1e-3
  and worst row <= 2e-2 instead of rel RMS <= 0.05.

Measured on a Radeon AI PRO R9700 (gfx1201), Coder IQ1_M pack:
- parity: all 10 IQ pairs plus Q4_K / Q5_K pass, rel RMS <= 9.1e-5.
- kernels vs MMQ, 48 layers: 1.48x / 1.20x / 1.14x at 4K / 16K / 32K.
- prompt speed in the engine (server-reported, alternating starts per
  arm): 4K +12% to +19% depending on the start, 16K +8.3%, 32K +8.5%.
- first-token KL against the FP16 prompt path is 1.10x MMQ's (48 prompts
  of 4K), top-1 agreement 48/48; 4/4 long-context retrieval checks.
Not run on gfx11 or CUDA hardware here.
2026-10-09 03:06:58 +02:00
noon-at-cgn 425fc48a82 docs: replace the evidence placeholder in the split MTP-batch paragraph with the measured result 2026-10-09 03:06:46 +02:00
CYoung83 5fde38470a gguf_reader: open a tensor-less shard that ends before its aligned data start
unsloth/Qwen3.8-Flash-Next-GGUF UD-Q5_K_XL shard 1 holds metadata only and
is 10,946,618 bytes; its 32-byte-aligned data start is 10,946,624, so the
engine refused it with 'GGUF: data section starts past EOF'. llama.cpp's
gguf-py reads the same file (0 tensors, data_offset 10,946,624).

A file with tensors is still refused. A file without them gets data_start
clamped to the file size, so file_size() - data_start() stays 0 in the
payload checks downstream.
2026-10-09 03:06:46 +02:00
Niko1221 dd106e7de5 arch_defaults: gfx1200/gfx1201 get the six exact prompt switches of the gfx1151 table (decode half stays off); STRATA_GFX12_DEFAULTS=0 opts out (#1478) 2026-10-09 03:06:46 +02:00
noon-at-cgn a22a0e8c46 split: batched draft-layer prompt K/V on the last stage (STRATA_SPLIT_MTP_BATCH=1, opt-in)
On one GPU the prompt path computes the draft layer's K/V for each chunk in batches (Prefill::draft_kv, E-9). A layer
split always ran the drafter's own per-group pass instead, because the drafter sits on the last stage. With
STRATA_SPLIT_MTP_BATCH=1 the last stage's prompt path runs E-9 there. Default off: the default path is unchanged.

The drafter's K/V come from Q8_1 x Q8_0 MMQ instead of its own mmvq, so its drafts can differ; the target's tokens are
decided by the verify window.

Prefill::draft_kv says why it declined (optional out-parameter, also in the SYCL port's prefill.cpp), and the split
logs once what the variable did. Which path runs is a pure header (include/strata/program/draft_kv_plan.hpp) with a
CPU test. tools/split_mtp_batch_parity.py compares the greedy texts of the two arms on a live server. Docs: MULTI_GPU.md.
2026-10-09 03:06:46 +02:00
CYoung83 de21371b8a gguf_reader_test: a tensor-less file that ends before its aligned data start
Two synthetic cases: no tensors and the file ends at its header (should
open; data_start() clamped to the file size), and one tensor with the same
truncation (still refused). The first fails on 0.1.40.3: unsloth's
UD-Q5_K_XL shard 1 is that case.
2026-10-09 03:06:46 +02:00
noon-at-cgn 12019dc4a3 docs: replace the evidence placeholder in the thread-placement section with the measured result 2026-10-09 03:06:46 +02:00
noon-at-cgn 91f1e85dd1 --aux-cpus: put the engine's threads that are neither pool workers nor the host on spare CPUs (Linux, off by default)
Header-only placement (include/strata/platform/aux_cpus.hpp): the spare CPU set (the host core's SMT siblings, then
CPUs of physical cores with no worker; never a CPU that shares a core with a worker), the list parser, threads the
engine creates pin themselves, threads somebody else created (the CUDA driver's) are moved by a sweep over
/proc/self/task after the pool exists and at every request; pool workers and the host thread are never moved.
--aux-cpus off|auto|LIST / STRATA_AUX_CPUS; per request strata_tune aux_cpus=0|1 (serve/server.py).
Default off: the hooks do nothing. tests/platform/aux_cpus_test covers the derivation and live threads (CPU only).
2026-10-09 03:06:46 +02:00
noon-at-cgn f47349002c docs: replace the evidence placeholder in the lend-sizing paragraph with the measured result 2026-10-09 03:06:43 +02:00
noon-at-cgn d3fb3e6f75 split: decide STRATA_SPLIT_RING before sizing the lend regions; log borrowed vs RAM-kept slots
Builds on #1190. The layer split sized its lend regions (the RAM copies of the experts the prompt path borrows) with
Prefill::bytes_needed before the split's ring override was applied (Prefill::set_ring_override says it must be set
before any bytes_needed), so with STRATA_SPLIT_RING=384 the prompt path borrowed 5396 slots and the copy kept 4662.
The override is now decided first (decide_split_ring); include/strata/program/split_ring.hpp holds the pure helpers
(split_ring_slots, split_lend_slots - the scan's own tail-slot arithmetic, now one function for the region and the
borrow), src/program/split_ring_test.cpp covers them on the CPU. A start line reports borrowed vs kept slots and warns
when kept < borrowed. Docs: MULTI_GPU.md.
2026-10-09 03:06:43 +02:00
Evan 51d55a9ab4 generate: size every part's lend region for a layer split by the serve scan's own arithmetic (auto ceiling or the explicit chunk, the 128-slot floor, the auto percentage cap), #340's own-buffer rule deciding which parts lend nothing - and hand them to the resident copy before it pins 2026-10-09 03:06:39 +02:00
Evan aea9dc3ed8 resident split: pin_cache_complement keeps each stage's lend region in the RAM copy too (stage_lend_regions, append_stage_lend_regions) - the experts the prompt path borrows under a split streamed from the mapped files through page faults (3.7 GiB read for one 73K prompt, 1.5 s of a 3.9 s chunk on host staging, 2026-10-06 rig); one budget caps complement and regions, experts past it keep the file fallback 2026-10-09 03:06:39 +02:00
Vadim Shevchenko 3afbe5f7c2 setup: UD-Q4_K_XL / UD-IQ4_XS keep their RAM budget on a layer split (#642)
Engine 0.1.40 (fe9c10ca, #848) keeps the resident RAM copy on a layer split -
every stage's cache left out of it, the rest ranked by the whole expert
profile - and --resident-budget-gib runs through the same code
(pin_cache_complement), so the old engine refusal of the budget on a split is
gone. setup.py's split_mmap() got the matching exemption (43c8cfa9), but the
budget's three paths still assumed the old engine (#498):

- split_budget(): a start of a budget config on several GPUs deleted
  --resident-budget-gib (and asked for ~135 GB of RAM below that);
- offer_together(): a budget config was offered both cards only with ~135 GB
  of RAM, and "on both it has no budget";
- unsloth_together() / q4_split at install: --gpus, or "2" when asked, dropped
  the budget, and an explicit --resident-budget-gib kept one GPU.

From RESIDENT_SPLIT_ENGINE the budget is kept in all three, at any RAM. The
recommendations stay as they were (the golden configs are unchanged): one GPU
by default at install, "n" by default when a start offers both cards, --yes
alone one GPU; the split is taken only when asked for.

Tests: the #498 tests run with MIN_ENGINE pinned before RESIDENT_SPLIT_ENGINE
(as test_setup_risk's StartOnSeveralGpus does for #642), three new ones run the
kept budget; tools/test_setup_*.py on Linux: 421 tests, the 2 failures of
test_setup_engine_hash are the same without this change (no ready-made 0.1.41
engine to download in the test box).

Measured with engine 0.1.41 on an RTX 3090 (x16) + RTX 3080 Ti (x4), 91 GB of
RAM, UD-Q4_K_XL, budget 64 GiB: the auto split gives the x4 card layers 0-1
(678 slots) and the 3090 layers 2-47 (4,182 slots); 57.6 GiB of experts in RAM,
no file-tier reads per request; decode 45.0 / 45.1 tok/s (two runs) against
38.4 on the 3090 alone, a 20K prompt in ~18 s on both. Answers checked by hand.
2026-10-09 03:06:38 +02:00
Destrayon 1decc04efe setup: skip a cmake/ninja on PATH that does not run, for the one next to this Python
A pip launcher whose Python was removed stays on PATH ('ninja --version'
fails) and stopped every build there, though setup had just installed a
working copy in .venv.
2026-10-09 03:06:34 +02:00
BonifacioCalindoro 573642fc4c Fix PowerShell 5.1 command shown by UPDATE.bat
UPDATE.bat currently prints a command using `&&` for users to copy and
run manually. `&&` is not supported by Windows PowerShell 5.1, which
is still included with Windows 11.

Print the two Git commands separately instead.
2026-10-09 03:06:33 +02:00
nekomario28 9aba7baa28 sycl: keep the null-stream sentinel a sentinel in Verifier (#1555)
dpct migrated cudaStream_t ext_stream_ = nullptr to a pointer to the
in-order queue, evaluated at construction on whichever device is current.
Two places then misread it: init() compared ext_stream_ against the
*init-time* device's in-order queue, so a stage built on one device and
init'ed on another adopted the constructing device's queue
(cross-context submit failures, UR_RESULT_ERROR_DEVICE_LOST); teardown's
'cs_ != ext_stream_' ownership test drifted the same way.

Restore the CUDA contract: nullptr means 'no stream until set_stream()',
matching include/strata/core/verify.hpp:319 and src/core/verify.cpp:604.
2026-10-09 03:06:31 +02:00
duncanmcqueen 0b063ce804 sycl: fix the GEMM staging scratch lifetime and unregister freed allocations
Follow-up to the >4 GiB GEMM staging.

- gemm_stage freed its scratch as soon as a larger operand arrived, but the
  in-order queue is asynchronous: a previous staged GEMM could still read the
  buffer.  Wait for the queue before the free.
- The staging used dpct::get_in_order_queue() while the GEMM runs on the
  descriptor's queue (strata::q_of(stream_)); pass the descriptor's queue so
  the copy is ordered before the GEMM for a non-default stream, and so two
  Gemm objects on different queues do not share the thread_local scratch.
- Staging that could not allocate fell silently back to the direct (wrong)
  GEMM; it now says so.
- device_alloc_map had no unregister, so a freed cache base left a stale
  entry that device_offset_end could mis-attribute after the VA was reused.
  Add device_alloc_unregister and call it where the expert cache frees base_.
2026-10-09 03:06:31 +02:00
duncanmcqueen edff903d7a sycl: setup picks the Intel card by PCI id and configures Alchemist
setup reads the Intel cards from sysfs but to_sycl() decided the expert
mode from the driver alone, so a card's device id was thrown away and an
A-series on xe (the A770) could not be told from a Battlemage.  Carry the
PCI device id through intel_gpus() into to_sycl(), and treat every
Alchemist id (A770/A750/A580/A380/A310/A770M) as an A-series: it has no
FP64 hardware, so its config carries the driver's FP64 emulation (a kernel
on the sampled path declares double and is refused without it).

On i915 (the A750) the experts still load into a CPU-computed RAM arena
(--mmap-experts + STRATA_VERIFY_NO_HOST=0) - a single pinned host
allocation past ~3 GiB fails on that die.  On xe (the A770) the A-series
now takes the ordinary streamed path: the mirror is built in chunks (so
the >3 GiB limit is not hit) and the build passes
-ze-opt-greater-than-4GB-buffer-required, so the prompt path is correct
past the cache's 4 GiB.  tools/test_setup_sycl.py passes.
2026-10-09 03:06:31 +02:00
duncanmcqueen defadd6e0e sycl: build Alchemist kernels for allocations past 4 GiB
On an Arc A770 every prompt answered "!!!!" (token 0). The experts in the
last 0.46 GiB of the 4.46 GiB VRAM expert cache dequantized to NaN in the
prompt path. The slots' bytes were right (a memcpy read back equal to the
GGUF, and a fresh copy dequantized clean); only kernels reading them in place
got wrong bytes. In a JIT build IGC addresses kernel pointer arguments as a
32-bit offset from the allocation's base.

The JIT (non-AOT) build now passes -ze-opt-greater-than-4GB-buffer-required
to the spir64 backend, in the same argument as the divide/sqrt option (CMake
drops a repeated -Xsycl-target-backend=spir64). Measured on the A770, Coder
IQ1_M, --stream-experts: prompts of 73-542 tokens have 0 non-finite values
under STRATA_DBG_NAN=1, and the 542-token prompt answers right. The AOT build
is unchanged. Speed cost not measured.
2026-10-09 03:06:31 +02:00
duncanmcqueen 56efb70939 sycl: stage oneMKL GEMM operands that lie more than 4 GiB into their allocation
oneMKL's GEMM reads the last, partial tile of an operand from the wrong
place when that operand lies more than 4 GiB into its device allocation
(measured on an Arc A770, oneMKL 2026.1: a BF16 B past 4 GiB gave its last
21 columns wrong, none below 4 GiB). The prompt path's GEMMs meet this:
their buffers come from the top of the multi-GiB expert cache / prefill
region. This is the leading explanation for the A770's garbage on prompts
longer than a few dozen tokens - it repros with every expert
device-resident, so it is not the CPU handshake.

Register every big device allocation (malloc_device_guarded, and the
segmented expert cache's reserved VA) in a small registry, add
device_offset_end(ptr, bytes), and in Gemm::bf16 / Gemm::f16 copy either
operand into a scratch buffer of the queue's own (a fresh allocation, so
at offset 0) when its end lies past 4 GiB, then run the GEMM on the staged
operand. STRATA_SYCL_GEMM_NO_STAGE=1 is the negative control.

Not validated end to end: the run that showed the garbage was an
all-resident mirror, which no longer fits in this machine's RAM, and the
runnable --mmap-experts config crashes on this A770 with or without the
staging (exit 139, UR_RESULT_ERROR_DEVICE_LOST - verified a separate,
pre-existing fault with the negative control), so the staging has not yet
been shown to change an answer.
2026-10-09 03:06:31 +02:00
duncanmcqueen 18ee03e55c sycl: add the doorbell payload checksum
The decode handshake publishes a payload (the layer's activations, the
routed expert ids and weights) and then rings a sequence number the host
polls. The fences alone do not order the ring after the payload stores:
measured on an Arc A770, 14 of 2,832 rings in one run became visible
before the last activation store, so the CPU expert pool sometimes
computed on a partly stale payload. This is the fix the port's own
comments call for (elementwise.hpp: "the host can observe the ring and
then read a payload that has not landed").

The publish kernels now write, next to the sequence number, the ring it
belongs to and a checksum of the payload (the wrapping sum of every 32-bit
word mixed with its position, murmur3's finalizer - a plain sum missed a
reordering, [3,7] -> [7,3], of the expert ids) in a per-ring slot modulo 4,
and the payload is stored with the uncached write hint (a plain store to
host USM reaches the host only when the kernel ends on xe). The host waits
for the whole payload (doorbell_wait_payload) before the pool reads it, in
session_loop and both verify-window loops; the checksum is accepted in
either publish shape (with or without the routing weights), so a variant's
choice need not be plumbed to the reader. The sequence word grows from 1 to
9 mapped words (layer.cpp, verify.cpp) and the ring slot is reset with it.

Note this is a robustness fix, not the cure for the A770's garbage on
longer prompts: that one repros with every expert device-resident (no CPU
handshake) and points at oneMKL's >4 GiB operand bug, which needs the BLAS
staging the CUDA/HIP shims have.
2026-10-09 03:06:31 +02:00
duncanmcqueen 0d9d8541f4 sycl: keep Alchemist's prompt path on its own buffers
At a long context this port borrows expert-cache slots for the prompt
path. Those buffers are carved from the top of the ~10 GiB expert-cache
allocation, and oneMKL's BF16 GEMM reads a wrong tile when an operand lies
more than 4 GiB into its allocation (docs/INTEL.md). The borrowed operands
hit that, so from the first full prefill chunk the router read garbage.

Measured on an Arc A770: a 236-token prompt failed with "prefill: routed
id out of range", and a 452-token one segfaulted the engine (exit 139),
on both the mirrored and mmap configurations. With the prompt path on its
own buffers (its own allocation, reserved exactly by owned_prefill_mib)
and an automatic chunk capped at 2048 tokens, 452 / 1,748 / 3,476-token
prefill chunks ran to completion with no prefill failure (prompt reads
22 (cold) / 676 / 835 tok/s). Whether the *answers* of that configuration
are correct is a separate, still-open issue: the base port has no doorbell
payload checksum and its longer-prompt path garbles on some inputs, with
or without borrowing.

Scoped to Alchemist (PCI device ids, the same table setup_intel.py uses):
Battlemage's long prompts were measured correct with borrowing, so it
keeps the old default. --prefill-borrow still borrows, for testing a
future BLAS staging fix (the fix that lets borrowing return, as the
CUDA/HIP shims already stage).
2026-10-09 03:06:31 +02:00
duncanmcqueen e02bd9955e sycl: chunk the expert mirror and raise memlock in the wrapper
Two Alchemist (Arc A-series) fixes for the streamed-expert mirror:

- gguf_expert_source: a single sycl::malloc_host past ~3 GiB fails on DG2,
  so the mirror (thousands of experts, tens of GiB) could not be built on
  an A770 at all. It is now allocated in <=2 GiB chunks; the per-(layer,
  expert) address, the layer's first blob and close() use the chunk
  pointers. A cleanup guard frees every chunk on an allocation failure,
  a failed read, or any exception; the read phase is unchanged.
- strata-sycl.sh: docker's default memlock is 8 MiB, far below what pinning
  the mirror or the resident arena needs; pass --ulimit memlock=-1.

Measured on an Arc A770: the Coder IQ1_M (8,474 experts missing VRAM,
16.15 GiB) mirrors and the engine starts; before, it refused with
"8474 experts are neither in VRAM nor mirrored".
2026-10-09 03:06:31 +02:00
duncanmcqueen 9c790e3dde sycl: publish the doorbell with the uncached write hint
The decode handshake publishes a ring sequence the host polls while the
window graph still runs. On an Arc (xe) a system-scope atomic store, even
with a release fence, is served from the GPU's cache and reaches the host
only when the kernel ends, so a host polling mid-window never sees the
ring. That is why the verify window had to wait for the whole graph
(STRATA_VERIFY_NO_HOST) and every expert had to be device-resident or in
the pinned mirror.

STRATA_DOORBELL_ATOMIC_LOAD already fixes the load side the same way; this
does the store side with the Intel uncached L1+L3 write hint. The release
fence stays (ordering only: a later payload store follows the ring).

Measured with sycl/probe/doorbell_store.cpp on an Arc A770: the host sees
the store at 230 ms with the kernel running to 2030 ms; with the old atomic
store it sees it at 2038 ms, exactly when the kernel ends (exit code 3).
-DSTRATA_DOORBELL_ATOMIC_STORE restores the atomic store.

This is a prerequisite for running with the CPU expert pool on xe (the
per-layer handshake); it does not by itself fix longer-prompt prefill.
2026-10-09 03:06:31 +02:00
Evan 7a7c568b9a resident split: a test that the RAM copy keeps each stage's lend region (red: FileExpertSource has no stage_lend_regions, and no append_stage_lend_regions) 2026-10-09 03:06:15 +02:00
Chen Cheng 8e9d3d0a19 serve: the hardware sampler keeps running where psutil finds no disk counters
psutil.disk_io_counters() returns None when it finds no disk (Modal's gVisor
has no /proc/diskstats; psutil's docs say Windows can need `diskperf -y`).
_disk() then raised AttributeError at the first sample and the sampler thread
ended: /metrics' hardware stayed empty, the Monitor tab had nothing to show,
and the frozen-engine check's gpu_busy() read a GPU load of 0. It now returns
no disk reading and the other readings go on.
2026-10-09 03:04:22 +02:00
InB4DevOps 2a91a14414 serve: compact streamed line fragments into blocks 2026-10-09 03:04:22 +02:00
InB4DevOps ac1df2c416 tokenizer: initialize caches once per instance 2026-10-09 03:04:21 +02:00
InB4DevOps b2b94e3f6f tokenizer: sweep protected spans in match order 2026-10-09 03:04:21 +02:00
Hulk 8c6b1cc5b5 Limit body framing fix to incomplete chunked trailers 2026-10-09 03:04:21 +02:00
Hulk d467b3d2bb Reject incomplete HTTP request body framing 2026-10-09 03:04:21 +02:00
Hulk 2bf1d07c05 fix(serve): make context overflow errors recognizable 2026-10-09 03:04:21 +02:00
nekomario28 32a2439de0 serve: answer the request that hit the READY timeout (#1527)
The READY-read timeout from #1317 kills the engine, but the request that
was loading it ended with no response: the RuntimeError from __init__
matched none of the request path's handlers, and on a lazy first start
restart()'s finally crashed on slot_cv (created only after READY), so a
misleading 400 masked the failure instead.

- raise EngineDied (still RuntimeError) for the start failure, so the
  request ends 503 'did not report READY ...; the next request restarts
  it' on the load, restart and /v1/load paths alike
- skip the slot_cv wakeup when a lazy first start never created it

serve/test_ready_timeout.py covers both paths with silent fake engines:
the lazy load (unfixed: 400) and a restart that never reports READY
(unfixed: connection closed with no answer).
2026-10-09 03:04:14 +02:00
nekomario28 efe3bfcf87 serve: keep the given card order when the reorder starves the last stage (#1576)
#1352 orders an auto layer split faster-card-last because the last stage
runs the head, the draft layer and the verify. On a VRAM-asymmetric pair
that rule can put the heaviest stage on the card least able to hold it:
the reporter's 12 GB 4070 Ti ended up with 31 layers + head + verify and
a 3 GiB expert cache (decode ~58 -> ~44 tok/s, the 'card this small'
warning on).

Skip the reorder when it would put the last stage on a card with less
total VRAM than the one the config placed there (NVML / amdgpu sysfs;
unreadable cards keep the reorder).  scores/vrams stay injectable for
tests.
2026-10-09 03:03:14 +02:00
Niko1221 fb58e0dbc8 Stage buffers: pinning is opt-in (STRATA_STAGE_PIN=1); on by default it corrupted IQ3_S decode after a long prompt in the release gate 2026-10-08 12:25:39 +02:00
Niko1221 982b8cf515 stage pin: no cudaErrorNotSupported (it is not in the HIP compat layer); same behaviour 2026-10-08 10:17:21 +02:00
Niko1221 69342e8e6c Merge main (v0.1.40.4) into f-141
# Conflicts:
#	CMakeLists.txt
#	setup.py
2026-10-08 10:07:12 +02:00
Niko1221 459bf63537 tests: drop the duplicate test_pipeline_windows_wins left by the merge; the hotfix-tag test no longer assumes a four-number MIN_ENGINE 2026-10-08 10:03:44 +02:00
Niko1221 0a45d54b2f Version 0.1.41: MIN_ENGINE 0.1.41 2026-10-08 09:59:53 +02:00
Niko1221 9b5e9ab13a 0.1.41 merge cleanups: bench key placeholder, B65 README author text, calibration time text, fixups.py regenerates the run-time spin bound 2026-10-08 09:58:59 +02:00
Niko1221 e5c5739ae0 Merge f-141-research: STRATA_DBG_FEAT and ROUTE_OVERRIDE debug hooks (env-gated) 2026-10-08 09:57:02 +02:00
Niko1221 ed13c9a10f Merge f-141-routeres: route-resident experts (opt-in, experimental) 2026-10-08 09:56:56 +02:00
Niko1221 b0bc4a70f4 Merge f-141-conc: --batch-groups auto for layer-split batches, #857 fix, listen backlog
# Conflicts:
#	src/program/generate.cpp
2026-10-08 09:56:56 +02:00
Niko1221 4aca10c5a0 Merge f-141-mgpu: no-P2P path (opt-in), #794, #793, peer knobs 2026-10-08 09:56:29 +02:00
Niko1221 a00d889383 Merge f-141-amdintel: AMD and Intel work, watchdog file-tier tolerance, ports
# Conflicts:
#	bench/results/COMMUNITY.md
2026-10-08 09:56:29 +02:00
Niko1221 ef80713032 Merge f-141-fixb: Python, docs and bench fixes and ports
# Conflicts:
#	setup.py
2026-10-08 09:56:13 +02:00
Niko1221 e50c5946c7 Merge f-141-fixa: Pascal decode regression (#1469), clamps, SYCL multi-card and spin bound 2026-10-08 09:55:58 +02:00
Niko1221 c27998f5ac Merge f-141-cuda: foresight, GDN chunked, per-arch tables, stage pin, CUDA bug fixes
# Conflicts:
#	docs/DETAILS.md
2026-10-08 09:55:41 +02:00
Niko1221 073e2fe28d Merge f-141-quick: CPU prefill share, file tier batched reads, tokenizer cache, server hardening, small fixes 2026-10-08 09:55:28 +02:00
Niko1221 5df35dcb47 Experimental STRATA_ROUTE_RESIDENT: residency-biased routing in verify windows (opt-in, changes the output) 2026-10-08 07:54:09 +02:00
Niko1221 c34dd57143 serve: STRATA_HTTP_BACKLOG, test with 60 simultaneous connections 2026-10-08 07:32:43 +02:00
Niko1221 6674a0065f Version 0.1.40.4: MIN_ENGINE 0.1.40.4; Pascal sm_61 decode back to 0.1.40 speed (#1469) 2026-10-08 07:23:15 +02:00
Niko1221 ca001df571 batch: a layer split with --batch pipelines one group per stage by default (--batch-groups 1 opts out); server listen backlog 256 (30-40 simultaneous clients got connection resets) 2026-10-08 07:10:59 +02:00
Niko1221 55dee8ed84 prompt buffers (#1454): keep counting the bytes bo shares with emb unless STRATA_EMB_REUSE_ACCOUNT=1
The VRAM reuse stays, but bytes_needed_impl counts bo again by default, so the auto chunk and the borrowed cache slots
are those of 0.1.40.3 and the prompt residuals stay byte-identical (on the RTX 3060 the planner otherwise picked a
6656-token chunk instead of 6400). STRATA_EMB_REUSE_ACCOUNT=1 lets the planner use the saved bytes (+3.9% prompt there);
documented in docs/DETAILS.md.
2026-10-08 04:44:49 +02:00
Niko1221 41a5a5488d SYCL expert cache (#1440): a one-GPU run keeps the free-memory figure it had
The previous change skipped the DRM-fdinfo subtraction whenever the driver reported usage. On a single Arc Pro B70 that
made the expert cache 12168 slots instead of 11338 (23.09 vs 21.52 GiB), so a one-GPU run was not unchanged. The
subtraction is now skipped only when the process sees more than one GPU (where it double-counts the other card) and the
driver does not report the whole card as free.
2026-10-08 03:49:18 +02:00
Niko1221 a79e7225af SYCL doorbell spin bound chosen per device at run time (#1397)
The bound of the device's spins on host flags was a build constant: 20,000 reads for a bmg AOT build, 2,000,000
for anything else. A B70 built without AOT (SPIR-V, JIT: the Windows/OpenCL build, or a Linux build with no AOT
target) got the A750-sized bound. The issue reports a 37% drop in decode between 0.1.40.2 and 0.1.40.3. On a Linux B70 JIT build the two bounds decode
alike (--spec 2, 3 pairs: 19.4 vs 19.5 tok/s), so this makes the bound independent of the AOT flag but is not shown to be that drop's cause.

spin_max(queue) decides per device: 20,000 under the xe driver (the B-series on Linux), 2,000,000 elsewhere (the i915
A-series needs it for the host's per-layer expert work; a Windows/OpenCL B-series build measured faster with the long bound).
STRATA_SPIN_MAX=<reads> overrides it at run time; -DSTRATA_SYCL_SPIN_MAX=<reads> still fixes one bound for the
whole build. The three waiting kernels take the bound as an argument.
2026-10-08 03:40:09 +02:00
Niko1221andcrobe201 a85347811a SYCL layer split (#1440): the host mirror keeps to the first stage's layers; the fdinfo free-memory fallback only where the driver reports the whole card free
- generate.cpp: with a layer split the host mirror walked the later stages' layers too (#1054, fixed in #1111 but
  lost in the re-migration): 20 GiB of pinned RAM for experts the other card holds, or one 39 GiB pinned allocation
  that fails. Both loops stop at the first stage's last layer.
- expert_cache.cpp: device_free_bytes() subtracted the DRM bytes of every card of the process from the card it asked
  about, so on a layer split the second card was charged the first one's cache and its expert cache would not open.
  The fallback is for drivers that report the whole card as free (Arc A750/i915); it now runs only then.

Reported by crobe201 with both diffs; MaineTim confirmed the second on 2x B60.

Co-authored-by: crobe201 <[email protected]>
2026-10-08 03:07:50 +02:00
W1nge ccd657a296 cuda: retain HC norm products in existing scratch 2026-10-08 03:07:20 +02:00
W1nge 199fd1d5d8 prefill: reuse embedding storage for half outputs 2026-10-08 03:07:20 +02:00
Niko1221andMatthew Hines e372d4ffd9 q8_1 emit sites (#1448): clamp the scale and quantize against it, as the other producers do
The three sites fixed by the previous commit stored a clamped (d, sum) but still
quantized against the unclamped d, so an overflowing block stored int8 values
that no longer matched its stored scale. They now use q8_1_finite(d) and
q8_1_quant like native_quantize_q8_1_kernel, the swiglu quantizer and the CPU
pool. A block whose amax is finite and below 8.3M is stored bit for bit as
before.

Co-authored-by: Matthew Hines <[email protected]>
2026-10-08 03:02:42 +02:00
Matthew Hines 7a0e30235c #606: clamp the three unclamped q8_1 scale/sum emit sites via q8_1_ds
Three kernels still emit the (d, sum) pair of a q8_1 activation block
with a raw make_half2(d, sum), bypassing the finite helper the rest of
the tree uses (q8_1_finite.hpp, from 19de7f4 / f6601bf):

  fused_gr.cu   gr_q8_tail                 - the fused gate-reduce tail (QFUSE)
  verify_kernels.cu  gdn_q8_1_store        - the batched GDN verify store (QFUSE)
  iq_kernels.cu  s26_swiglu_q8_1_kernel   - the S26 swiglu path

A block whose amax overflows (or that reads a non-finite input) emits
d = inf/NaN; fp16 makes the pair NaN (or the pathological (0,0)). Nothing
downstream tests for it: a NaN scale poisons the fp16 dot of the quantized
activation and every sum downstream of it, and a fully non-finite logit
row ends at the sampler's all-NaN fallback - vocabulary id 0, the '!'
face of the degeneration reported in #606. Same defect class already
fixed at the other emit sites; these three were missed.

q8_1_ds() clamps both halves into the fp16-finite range (finite d,
non-negative finite sum - the sum is a mean of |x|), producing for the
GPU paths the same bits the CPU pool and the clamped GPU sites produce
for the same block. A clamp at the producer, content-blind: nothing
downstream of it is model-text-aware.

Criteria and the wider carried-state RCA: docs/Q8_1_FINITE_DS.md and
docs/DEGENERATION_RCA.md.
2026-10-08 03:02:28 +02:00
Niko1221 fbb362425c Pascal decode: keep __restrict__ and drop the PDL prefetch below sm_70 (#1469, #1470)
2e4ddf6 (the #904 port) removed __restrict__ from the activation pointers of
native_quantize_q8_1_kernel, bf16_f32_mmvf_multi_kernel and
native_mmvq_multi_kernel on every CUDA architecture, and turned on the weight
prefetch before pdl_wait(). Both only matter for PDL (sm_90+). On sm_61 the
missing restrict loses the read-only data path (LDG.CI) and decode halves
(36.4 -> 17.7 tok/s on a Tesla P40, 32-36 -> 17.7-18.6 on a GTX 1080 + 1070).

STRATA_PDL_RESTRICT is __restrict__ and kPdlPrefetch false again for
__CUDA_ARCH__ < 700 in the device pass. sm_70 and newer compile to the same
SASS as before (cuobjdump -sass of native_mmvq.cu and native_bf16.cu at sm_70,
86, 90 and 120 is identical); the host pass is unchanged.

Reported and bisected by paulhothersall and lineape.
2026-10-08 02:58:45 +02:00
Niko1221 c788e2aa72 Pascal decode: keep __restrict__ and drop the PDL prefetch below sm_70 (#1469, #1470)
2e4ddf6 (the #904 port) removed __restrict__ from the activation pointers of
native_quantize_q8_1_kernel, bf16_f32_mmvf_multi_kernel and
native_mmvq_multi_kernel on every CUDA architecture, and turned on the weight
prefetch before pdl_wait(). Both only matter for PDL (sm_90+). On sm_61 the
missing restrict loses the read-only data path (LDG.CI) and decode halves
(36.4 -> 17.7 tok/s on a Tesla P40, 32-36 -> 17.7-18.6 on a GTX 1080 + 1070).

STRATA_PDL_RESTRICT is __restrict__ and kPdlPrefetch false again for
__CUDA_ARCH__ < 700 in the device pass. sm_70 and newer compile to the same
SASS as before (cuobjdump -sass of native_mmvq.cu and native_bf16.cu at sm_70,
86, 90 and 120 is identical); the host pass is unchanged.

Reported and bisected by paulhothersall and lineape.
2026-10-08 02:58:44 +02:00
Niko1221 6c51a295b4 research: opt-in prefill hooks STRATA_DBG_FEAT (routing + hidden-state dump) and STRATA_DBG_ROUTE_OVERRIDE (not default-path) 2026-10-08 02:50:46 +02:00
Niko1221 9386089d80 docs: Lock pages in memory takes effect at the next logon (#1412) 2026-10-08 02:42:07 +02:00
paulhothersall 53c9baff07 docs: a short Contributing section in AGENTS.md and a test-requests list 2026-10-08 02:37:03 +02:00
Niko1221 5343e878a1 mtp: drop the duplicate dhead_type_ line from #1329 (f-141 already copies it); the batch-mtp test recipe stays 2026-10-08 02:36:53 +02:00
Ethan d1e93c02bf sweep: the round line prints that round's draw, not the median through it
Found while running it against a live engine: the line said 0.35 as 48.7 on round 3 while the three draws were 57.0, 48.7 and 35.5, because it printed the running median. Reading a sweep's drift is the whole point of the line, so it has to be the draw. The medians and the ranges are still in the line below it.
2026-10-08 02:36:32 +02:00
Ethan d1990da1a6 calibrate: the sweep visits every value in every round instead of once
Phase 1 measures each PCIe share once and takes the maximum, with the three prompts of that rate all at the one
setting.  Neighbours cannot be ordered that way: on this box 0.75 through 0.95 come out at 72.7, 69.0, 71.0 and
71.9, which is 0.2% apart, while the same settings moved 8-28% between engine starts.  A value that was worth a
real gain can lose its single sample and never be measured again, because phase 1's winner is the share phase 2
then sweeps spec_min_p at, and the confirm step only puts that one resulting pair against the default.

sweep() measures every value SWEEP_ROUNDS times inside the one engine start, one round visiting all of them and
every other round in reverse order, and the winners come from the medians.  Drift inside a sweep then lands on
every value the same number of times, and one bad sample cannot decide.  The same function runs phase 2, and the
confirm step uses the same round count instead of its own.

Cost: the sweep multiplies by SWEEP_ROUNDS, 3 by default.  A rate is three prompts of 128 tokens, so on this box
(six values, 95-100 tok/s) phase 1 goes from about 40 s to about 120 s, on a run that already loads the model more
than once.  A machine whose two leaders stay inside each other's noise can raise the constant.

Not in this change: phase 4 needs a restart per worker count and stays sequential, so a count is still taken from
one measurement each.

Tests: python -m unittest tools.test_calibrate (the stand-in engine is a pure function of the tune keys, so the
medians and the winners are the same as before); tools.test_setup_config and tools.test_setup_golden
2026-10-08 02:36:32 +02:00
Ethan fe0df23120 calibrate: the PCIe share sweep reaches 1.0, where no expert goes to the CPU pool
PCIE_FRACS stops at 0.75, so setup --calibrate never measures the value that takes the CPU pool out of the decode
window completely. 1.0 is a separate operating point rather than 0.75 carried further: the share is applied as
m = (nmiss * pcie_num) >> 8, so at any value below 1.0 at least one miss still goes to the CPU pool, and only at
1.0 does every non-resident expert come over PCIe. A CPU expert sits on the layer's critical path while the GPU
waits for it, and on a slow CPU that is the whole difference.

On a Core Ultra 9 290HX Plus with an RTX 5090 Laptop on PCIe 5.0 x16, 0.55 gives 51.1 tok/s against 71.3 at 1.0,
with cpu_exp falling from 1.06 to 0.00 per layer-window and the window time falling by the 11.3 ms of PCIe waiting
plus the CPU's expert time. A sequential sweep puts 1.0 at 78.8 against 72.7 for 0.75 and 51.5 for 0.35.

0.9 is added beside 1.0 so a machine where 1.0 is too much gets an interior point. pick() keeps the default unless a
candidate beats it by MIN_GAIN, so this cannot make a calibration worse. The cost is one rate per value, about 8 s
here.

Tests: python -m unittest tools.test_calibrate; tools.test_setup_config and tools.test_setup_golden
2026-10-08 02:36:23 +02:00
zetlaw e0f6da57f5 Fix shared MTP draft head type for batch slots 2026-10-08 02:36:23 +02:00
Alekperov Fuad 0d994dfb9c setup: link the engine against the toolkit whose nvcc builds it (CUDAToolkit_ROOT)
With a second toolkit beside the distribution's one, CMake compiled with the chosen nvcc but took cudart and cuBLAS from /usr (Ubuntu 24.04, STRATA_NVCC=/opt/cuda-13.0/bin/nvcc: the engine needed libcudart.so.12 while built with the 13.0 headers). Pass CUDAToolkit_ROOT for the engine and the CUDA image encoder; the distribution's own /usr/bin/nvcc keeps CMake's search as before.
2026-10-08 02:36:22 +02:00
MIDHAT NASHAR bee3b1947c test: make checkpoint byte fixtures valid under MSVC 2026-10-08 02:35:37 +02:00
phoenixclyde a732a19b6f hip/gfx906: the tree does not build (q6_k MMQ instance and cudaEventBlockingSync)
Two gfx906 build failures on v0.1.40.2. Both are one-line gates that know
about the wave32 HIP backend but not about STRATA_HIP_GFX906 -- the same
class as #808 and #1083. Neither is fixed on main (e8ca9afd03).

1) src/prefill/prefill.cpp:399 builds an event with
   cudaEventDisableTiming | cudaEventBlockingSync, but the gfx906 compat
   header include/strata/platform/hip_compat/strata_hip.h maps only
   cudaEventDisableTiming and cudaEventDefault. The wave32 header
   include/strata/hip_compat/cuda_runtime.h:39 has the mapping.
   -> add #define cudaEventBlockingSync hipEventBlockingSync there.

   A diff of the whole set of cuda* defines between the two headers lists
   one other name the gfx906 header lacks, cudaDevAttrIntegrated, and that
   one is already gated upstream (#if defined(__linux__) &&
   !defined(STRATA_HIP_GFX906) in src/core/expert_cache.cpp:31), so it is
   deliberately not touched here.

2) Link error: undefined symbol
   void mul_mat_q_case<(ggml_type)14>(ggml_backend_cuda_context&, mmq_args const&, ihipStream_t*)
   (14 = GGML_TYPE_Q6_K).
   src/prefill/moe_mmq.cu:219 calls it under
   #if defined(__HIPCC__) || defined(STRATA_Q6K_EXPERTS), so any HIP
   compilation references it; mmq.cuh declares the case as an extern
   template, so the definition must come from
   template-instances/mmq-instance-q6_k.cu. The CUDA branch of
   CMakeLists.txt adds that file only under STRATA_Q6K_EXPERTS.
   STRATA_HIP_GFX906=ON forces STRATA_ENABLE_CUDA=ON, so gfx906 takes the
   CUDA branch while its .cu files are compiled as HIP.
   -> widen the gate: STRATA_Q6K_EXPERTS OR STRATA_HIP_GFX906.
   This adds one MMQ instance to the static library. It does not enable
   STRATA_Q6K_EXPERTS (the grouped Q6_K expert kernels that allocate extra
   VRAM), and STRATA_HIP_GFX906 is only ever ON in the gfx906 build, so
   CUDA and wave32 builds are untouched.

Verified on a single Instinct MI50 32 GB (gfx906, wave64), ROCm/HIP 7.14,
-DSTRATA_HIP_GFX906=ON -DCMAKE_HIP_ARCHITECTURES=gfx906
-DSTRATA_MMQ_KQUANTS=ON:

- the engine builds and serves UD-IQ4_XS (Qwen3.8-Flash-Next) end to end:
  /v1/models, chat completions, tool calls, MTP drafts accepted;
- nm -C shows mul_mat_q_case<(ggml_type)14> in the binary (13 instances:
  7 8 12 13 14 16 17 18 20 21 22 23 42);
- change 2 is load-bearing: removing mmq-instance-q6_k.cu.o from
  libstrata_mmq.a and relinking with the same link command line fails with
  exactly the undefined symbol above (the archive holds one U reference
  and that one W definition), while the untouched archive relinks cleanly.
2026-10-08 02:35:37 +02:00
Niko1221 9bf626ff67 bench: COMMUNITY.md rows for the ten community folders ported from #1429 #1432 #1433 #1443 #1452 #1459 #1460 #1462 #1467 2026-10-08 02:34:58 +02:00
Dmitry f4e2c89c55 bench: community report - RTX 4090, Flash-Next IQ3_XXS at 204800, engine 0.1.40.3
Two variants of the same sweep (code-explanation prompts and Russian prose) on
one PC and one server config, engine 0.1.40.3 built from source for sm_89.
Each folder carries the 0.1.40.2 run, the 0.1.40 baseline and an
identical-binary control run so the noise band is in the report.

Result: no measurable change from 0.1.40.2 on this machine.
2026-10-08 02:34:06 +02:00
wolffahrer baa55adc4c bench: RTX 5090 Laptop GPU follow-up for 0.1.40.2 and 0.1.40.3
Same laptop, model files, configuration and scripts as the 0.1.40.1
entry (2026-10-06-community-rtx-5090-laptop, #1263); only the Strata
version changed. Raw data for 0.1.40.3, medians for 0.1.40.2 in the
README. Decode and prompt throughput unchanged within run-to-run
variation, all needles found, coding check 10/10 on 0.1.40.3. A 0/10
coding check on 0.1.40.2 is traced to a greedy reasoning loop that
occurs in 0.1.40.1 as well (re-runs documented). Results only, no
engine changes.
2026-10-08 02:34:05 +02:00
xWinIcex 654aa2edbe Add community benchmark: RTX 5080 in Docker Desktop on Windows (WSL2)
Coder IQ1_M, one RTX 5080 16 GB, the repository's own Dockerfile and entrypoint
on a Windows host (Docker Desktop 4.84.0 / WSL2), two context limits on the same
loaded model: 131,072 and 65,536.

- Speed: three runs at each of 4,096 / 8,192 / 16,384 / 32,768 / 65,536 /
  130,560 prompt tokens (128K limit) and at 4,096 / 8,192 / 16,384 / 32,768 /
  65,024 (64K limit), greedy, reasoning off, 256-token output cap, zero reused
  prompt tokens in all 33 runs.
- Correctness: tools/needle_bench.py 9 of 9 (32k/64k/128k) at the 128K limit and
  6 of 6 at the 64K limit, plus a vision check with the encoder on the CPU.
- Provenance: all three model files SHA-256-verified against the repository's
  published LFS oids at the pinned revision; MTP manifest and inventory included.
- Environment: Docker Desktop's default WSL2 memory cap makes setup.py read ~31 GB
  of a 64 GB host and silently select the low-RAM mode; the report documents the
  .wslconfig change used instead. Also documents the 8-token headroom a request
  needs (prompt + max_tokens + 8 <= max_context) and the engine's own VRAM warning
  at a 128K context on a 16 GB card.

Results-only; no engine changes.
2026-10-08 02:34:04 +02:00
saikiran-rs 244f113c96 bench: mixed AMD helper layout at1M, long-output and quality results 2026-10-08 02:34:02 +02:00
aleesposito85 69391129fe bench: community report, Tesla V100 32 GB + P100 16 GB expert-helper (mixed Volta/Pascal), Flash-Next IQ3_XXS (#1079) 2026-10-08 02:34:01 +02:00
rvannos 47e9552be7 Add benchmark results for 2x RTX 4000 configuration
Added detailed benchmark results for dual NVIDIA Quadro RTX 4000 setup, including performance metrics, hardware specifications, and observations.
2026-10-08 02:34:01 +02:00
Sophie Andreikina df53228251 community bench: RTX 5090 + RTX PRO 4000 Blackwell, UD-Q4_K_XL helper cache, engine 0.1.40.3 2026-10-08 02:33:57 +02:00
timnevits 6fff95b8a6 bench: Arc Pro B65 Gen4 results on v0.1.40.2 2026-10-08 02:33:56 +02:00
shrisha e905bf9946 bench: 2x TITAN RTX (sm_75) at Strata v0.1.40.3, IQ3_S, 262K
Nine files following docs/COMMUNITY_BENCHMARKS.md. Engine v0.1.40.3
(tag d5ea713) built from source for sm_75, IQ3_S at the model native
262,144-token window, with --vision and --reasoning-budget-tokens 12000.

Three runs each at 4,096 / 32,768 / 128,000 prompt tokens: decode
70.9 / 77.3 / 71.1 tok/s, prompt 868.9 / 1548.9 / 1626.9 tok/s, medians
in the index and full ranges in the report. needle_bench 6/6 at 32k and
128k. Records that NVLink is present but unused by the engine, so a bridge
on consumer boards is not worth buying.
2026-10-08 02:33:55 +02:00
Alex GorevskiandCopilot 86b283e1eb docs: record RTX 8000 existing runtime switch measurements
Document opt-in architecture limits, prompt/decode trade-offs, numeric caveats and negative tuning results without changing kernels or defaults.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: a7c2bcbb-4264-4eb2-879d-c71a90285b0b
2026-10-08 02:33:19 +02:00
Lech Najdek f4ce5ac07f OLDER_GPUS.md: fill the P100 row with the 2x P100 IQ3_S measurements (#1157) 2026-10-08 02:33:16 +02:00
Niko1221 37338cadf1 setup: warn on a non-Default GPU compute mode; vision encoder start error names the busy/unavailable cause (#1445) 2026-10-08 02:31:56 +02:00
Niko1221andadambenhassen 54d26589ca setup: --remote-expert-opt only beside a helper cache; docs: --pipeline-windows and --adapt-async combine (#1447)
The flag acts only on --expert-cache-device1..3; a plain layer split got a no-op flag that the docs listed as a reason
for the pipeline and the async tier to switch off. Built on f-141-quick's #1352 pipeline-windows rule (same hunk).
MULTI_GPU.md: the two combine (only STRATA_PIPELINE_ADAPT_ASYNC=0 turns async off); adds the 2x3090 table.

Co-authored-by: adambenhassen <[email protected]>
2026-10-08 02:28:41 +02:00
Ron Forrester dd667967b9 serve: a tool call written <function= NAME> is the call NAME
The function name was everything between `<function=` and `>`, untrimmed. A model that writes a space after the `=`
(seen from Qwen3.8-Flash-Next IQ2_XS in Zed's agent panel) got:

- inside the thinking: the name " capture" matched no declared tool, so the #804 rescue kept the call as reasoning
  text; the reply ended with finish_reason "stop" and no tool_calls, and the agent stopped mid-task;
- after the thinking: a call named " capture", which the client does not know, its arguments typed without the
  tool's schema.

The name is now stripped where the call is parsed (parse_tool_call), where the #804 rescue checks it against the
declared tools, and where a finished call looks up its schema (_finish_call). A name with surrounding whitespace is
never meaningful, so no text becomes a call that was not one: the #804/#1058 rules still decide whether a call
inside the thinking is an act.

serve/test_tool_name_space.py: both places, streamed and not, fed in 3-character pieces; fails 4/4 before, passes
after. test_reasoning_rescue, test_reasoning_tools and test_frontend unchanged and passing.
2026-10-08 02:25:52 +02:00
Niko1221 c85b7c87b9 sm_70 (PR 1401): the Volta table, expert mode 8 and the fast norm/up are opt-in with STRATA_SM70_TABLE=1 until confirmed on a V100; default sm_70 behaviour as in 0.1.40.3 2026-10-08 01:24:59 +02:00
Niko1221 162ce64eff prefill: the CPU share is on by default on CUDA, one GPU, no batch slots, chunks below 1024 tokens (STRATA_PREFILL_CPU_SHARE=0 turns it off); start line; docs 2026-10-08 01:06:02 +02:00
Niko1221 eb98c622e4 docs: measured numbers for STRATA_STAGE_PIN, STRATA_ADAPT_LAG, STRATA_GDN_CHUNKED 2026-10-08 01:04:50 +02:00
Niko1221 fc8444367d batch pipeline: a pipelined slot is a cache again, so a request left alone goes back to the solo path (#857) 2026-10-08 00:27:40 +02:00
Niko1221 5aee72b107 serve: the frozen-engine check does not end an engine whose GPU is busy (telemetry reading) 2026-10-08 00:27:08 +02:00
Niko1221 d2e3f1c9dd Stage buffers: pinned by default while the host has 3 GiB of RAM to spare (STRATA_STAGE_PIN=0 off, =1 always); docs: the 0.1.41 opt-in switches; DETAILS: several API keys, the frozen-engine check 2026-10-08 00:01:29 +02:00
Niko1221 421e163a30 serve: STRATA_DUMP_FIRST_LOGITS also writes each request's first-window logits (<path>.<n>), to compare prompt paths by KL 2026-10-07 22:55:42 +02:00
Niko1221 21d4165b16 tests: --batch-groups auto follows the engine's INFO batch_groups 2026-10-07 22:25:11 +02:00
Niko1221 886bc47742 docs: --batch-groups auto in BATCHING.md 2026-10-07 22:23:40 +02:00
Niko1221 aa19d0d395 docs: stager profile for a warm GGUF read in place (#1353) 2026-10-07 21:37:19 +02:00
Niko1221 b2c5b22131 docs/INTEL.md: the A750 sampled path needs no FP64 emulation (measured) 2026-10-07 21:24:46 +02:00
Niko1221 ff3bda728d fix: newline in the batch-groups auto log line 2026-10-07 21:21:44 +02:00
Niko1221 67fba9c1fc serve: card order on AMD/Linux too (KFD topology: SIMDs x clock) 2026-10-07 21:19:17 +02:00
Niko1221 786b3ca016 serve: card order on AMD/Linux too (KFD topology: SIMDs x clock) 2026-10-07 21:19:07 +02:00
adonizm 0b030b96fc prefill: batched unbuffered expert reads in the stager; close the experts.bin view once reads are unbuffered
NTFS serves unbuffered reads of a mapped file one at a time (WD SN580: 2 MiB reads at QD32
2.35 GB/s mapped vs 3.57 GB/s unmapped). With the file tier unbuffered (--resident-budget-gib
or STRATA_UNBUFFERED_LOAD=1) the mapped view is only a fallback after startup, so it is closed
(STRATA_KEEP_MAPPING=1 keeps it). The prompt path's stager claims runs of consecutive jobs and
reads them with one read_direct batch (STRATA_STAGER_BATCH, default 8 for file-backed blobs).

RTX 5070 Ti, 32 GB RAM, IQ3_XXS, 47.6K prompt, --resident-budget-gib 18 --prefill auto:32768:
1260 -> 1370 tok/s (batch 8) -> 1669 tok/s (view closed).

Ported from #833 onto the rewritten main. Two fixes on the way: unmapped_ is declared on every
platform (mapped_blob reads it outside the Windows block, so Linux did not compile), and
copy_blobs asks resident_blob, which follows the exchange rotation (STRATA_EXCHANGE_ROTATE).
2026-10-07 21:18:06 +02:00
Niko1221 d9621d1e23 serve: a request with no free slot gives the control lines back while it waits (long, long, short under parallel: 2 deadlocked: the slot a yielded read held needed the lines); regression test 2026-10-07 21:15:34 +02:00
Niko1221 039f9e9018 serve: malformed-shape bodies get a 400 with the traceback logged, a body-size cap (256 MiB, STRATA_MAX_BODY_MIB) and a bad Content-Length are answered before the read; engine warns on --batch-groups with --batch-mtp (#1413) 2026-10-07 21:08:03 +02:00
Niko1221 1bf6344cb3 Concurrency stage 1: --batch-groups auto (one pipeline group per GPU stage), INFO batch_groups; tools/serve_load.py load generator 2026-10-07 21:07:59 +02:00
Niko1221 f346dc6b0c tokenizer: remember the ids of repeated pre-tokenizer pieces (a resent 89K-token agent prompt encodes in 0.07 s instead of 0.32 s; same ids) 2026-10-07 21:05:33 +02:00
Niko1221 dfa26c19c9 tests: qsa_select_bench accuracy floor 1e-5 of the score scale (the fast scorer sits at 1e-6 against FP64; warp kernel 7e-8) 2026-10-07 21:02:45 +02:00
Niko1221 bad9a8f22e serve tests: a stopped server stops its hardware sampler (21 samplers leaked after test_responses skewed test_parallel's timing) 2026-10-07 21:00:39 +02:00
Niko1221 533667d06d fix the escape in the watchdog IO note (#1407) 2026-10-07 20:58:08 +02:00
Niko1221 37fa9f7817 sycl: the sampler's launchers no longer require FP64 (dpct left aspect::fp64 checks; samp_acc_t is float by default), so a sampled token no longer needs Intel's FP64 emulation on an A750 2026-10-07 20:56:08 +02:00
Niko1221 125f6ff05f native_mmvq, mmvq_il_parity, generate: fix newline escapes in messages 2026-10-07 20:50:30 +02:00
Niko1221 b7dd070d96 strata --version; build warnings: weights.cpp index scanf formats, qsa_layer unused variables and one-line statements, ignored cudaFreeHost results 2026-10-07 20:47:36 +02:00
Niko1221 ef362cc4ab One prefetch design (#1348): the router look-ahead feeds the Foresight swap space (ForesightSwap::predict, STRATA_FS_AHEAD), slot state under a mutex 2026-10-07 20:47:21 +02:00
Niko1221 b9aa4558a4 setup: checksum verified line, .previous keeps its BUILD.json; update scripts drop the backup branch when the move fails 2026-10-07 20:46:11 +02:00
Niko1221 d7f039992a sycl: expert dequant stores coalesced (lane tid writes the tid-th 16-byte run), bench dequant_bench; IQ3_S 200 -> 423 GB/s of writes on the B70, output bit-identical 2026-10-07 20:43:21 +02:00
NoxestNoz b04507c12f bench/results: V100 decode kernels (GPU work per verify window 22.8 -> 21.2 ms); docs/NVIDIA_V100.md: the decode side; tools/ab_engine.py 2026-10-07 20:42:39 +02:00
NoxestNoz aaa323fec5 fused_gr: Volta runs gfx906's latency-hidden norm/up (STRATA_GR_FAST), which now also writes the QFUSE q8_1 tail
The fast kernels build for CUDA; sm_70 defaults to them, other CUDA cards keep
the plain ones.  gr_up_fast_kernel ends with gr_q8_tail like gr_up_multi_kernel,
so STRATA_QFUSE gets its q8_1 image from it too.  In the engine (staged read)
only the up projection changes: hc-read0's up 1.08 -> 0.96 ms per window on a
V100-SXM2.  fused_gr_bench bitwise, gr_multi_parity OK.
2026-10-07 20:42:39 +02:00
NoxestNoz e509ba5e3d native_mmvq: the interleaved 2-4 column kernel on Volta, with a rows table read off a V100
Read off mmvq_il_parity --bench on a V100-SXM2-32GB with the table's own rule
(every measured shape of a class at least 3% faster, else native_mmvq).  At 3
columns: the Q5_K head 1094 -> 821 us, Q6_K 12288 rows 68.9 -> 47.9 us, Q4_K
10240 rows 43.9 -> 32.5 us.  Bitwise native_mmvq's output (mmvq_il_parity OK).
2026-10-07 20:42:39 +02:00
NoxestNoz f6330507db iq_kernels: Volta runs the routed experts in gfx906's expert mode 8 (bitwise the CUDA layout)
The AMD grouped-expert layouts (STRATA_EXP_MODE) now build for CUDA too.  sm_70
defaults to mode 8 - the codebook grid and the group's q8_1 activations in
shared memory, SwiGLU + q8_1 fused into the gate/up epilogue - every other CUDA
card keeps mode 0.  On CUDA, IQ2_XXS gate/up joins the shared-memory kernels,
a format without one falls back to the CUDA layout (not R2), and a strided call
(the verify window's PCIe call) keeps the CUDA layout.

V100-SXM2, setup's IQ2_XS, a verify window's VRAM call (28 groups): IQ2_S
gate/up layers 180 -> 150 us at 3 tokens, IQ2_XXS 170 -> 134 us; bitwise mode
0's output on all 48 layers (native_expert_bench), native_grouped_parity 0
failures in modes 0/2/7/8.
2026-10-07 20:41:10 +02:00
Niko1221 51d1ef4b8a Stage buffers: STRATA_STAGE_PIN switch (default decided by the A/B) 2026-10-07 20:40:45 +02:00
Zhong Uncle 475912c62a perf(expert): the stage buffers the cache fill copies from, pinned with cudaHostAlloc
A staged blob (the GGUF role slices assembled into one buffer) is handed to
ExpertCache::fill_slot's cudaMemcpyAsync as-is.  From a pageable buffer the
driver stages every copy through an internal bounce buffer - an extra copy at
a fraction of the transfer rate, on the calling thread - so the admit path,
the profile prefill and the lent-slot refills all pay it per expert.  Allocate
the pool's buffers with cudaHostAlloc instead, falling back to pageable for a
buffer when the driver refuses (pinned memory is scarce on the small-RAM
machines this path serves).
2026-10-07 20:40:45 +02:00
Niko1221 311c983964 serve: a config with a vision section but no --vision in its args starts the engine with images (#1322) 2026-10-07 20:39:21 +02:00
architectds ed6a02cdb3 docs: the CPU share up to 3,072-token chunks, from mapped experts, on a split - and what it measured
DETAILS.md's STRATA_PREFILL_CPU_SHARE section: chunks below 3,072 tokens are staged when it is on
(STRATA_PREFILL_CPU_SHARE_MAX=1024 keeps the old limit), the experts the CPU may take (the arena, the
page-locked copy, the mapped experts.bin's pages), the layer split, and a table measured on an RTX 5070 Ti
alone and an RTX 3060 + RTX 5070 Ti split (PCIe 3.0 x8, DDR4-3200, --mmap-experts): 512-token prompts
-49% / -41%, 3,000-token -22% / -24%; a recorded 100K-token coding conversation on the two cards 137.4 ->
130.6 s.
2026-10-07 20:39:12 +02:00
architectds 92bce20e4b prefill: the CPU share up to 3,072-token chunks, from mapped experts too, and on a layer split's stages
STRATA_PREFILL_CPU_SHARE handed the CPU pool its share of a chunk below 1,024 tokens only, only experts in
page-locked RAM, and only on one GPU.  With it set:

- the chunks staged after their routing (the only ones the share applies to) go up to
  STRATA_PREFILL_CPU_SHARE_MAX tokens (default 3072, at least 1024) instead of 1024; Prefill::arm_cpu_share
  sets the limit before any loan is sized, and init sets it too;
- an expert whose blob the CPU reads from RAM without assembling it (the arena's unpinned part, the mapped
  experts.bin's pages) can go to the CPU, not only a page-locked one; the blob pointers are taken when the
  experts are chosen, and one without a stable blob stays on the GPU;
- on a layer split every stage has the pool (serve, no --batch), and one stage at a time takes it for a
  chunk while the other reads its chunk on the GPU alone.

Unset, nothing changes.
2026-10-07 20:39:00 +02:00
sergiywith f2a3b946e9 prefill: STRATA_PREFILL_CPU_SHARE=auto times layers with and without the share and shares only while that is faster (#1282)
auto balanced the CPU's time per expert against the GPU's per streamed expert, both measured with the share on, so
nothing compared a layer with and without it.  On jase100k's RX 7900 GRE + Ryzen 7 5700X3D the host's time per
streamed expert rose 107 -> 200 us with auto (0.47): the streaming side saved nothing (0.53 x 1.87 = 0.99) and the
CPU's work came on top, -4.5%.  Now every eligible layer is timed with CUDA events from before its routing sync to its
combine, per non-resident expert; adjacent layers alternate no share / the measured share until three ratios are in,
then auto shares while the median of the last five is below 1, with one layer in 29 in the other arm.  Layers with a
one-time cost and the first layer are left out.  STRATA_DBG_CPU_GATE=1 prints each measurement.

RTX 5090 + 9950X3D, IQ2_XS, 250 / 600 / 900 tokens against off: DMA streaming -9.8/-9.9/-10.4% -> -8.8/-9.1/-10.5%;
an emulated host-copied stream -18.0/-16.7/-14.4% -> -16.1/-15.2/-13.0%; one core + one worker (where the old auto
lost up to +11.5%) +1.4..+2.4% -> +2.0..+2.3%, the cost of the first measurements.  Unset: byte-identical.
2026-10-07 20:38:56 +02:00
Niko1221 804dc7c326 setup: no --remote-expert-opt beside --pipeline-windows (the first turns the second off); docs: card order (#1352) 2026-10-07 20:38:27 +02:00
Niko1221 ae083aebbf calibrate: a later engine start that fails drops that candidate and keeps the earlier measurements (#1337) 2026-10-07 20:37:56 +02:00
Niko1221 be8db787c2 serve: card order for an auto layer split, faster card last (SMs x clock), gpu_order as_given opts out (#1352) 2026-10-07 20:37:35 +02:00
Niko1221 f60a12d407 conversation_cache_test: fix check call 2026-10-07 20:37:20 +02:00
Niko1221 f2e07afbf1 serve: the #29 watchdog waits (up to 10x, STRATA_WATCHDOG_IO_S) while the OS is still reading the file tier (#1407) 2026-10-07 20:37:15 +02:00
Niko1221 9b4336a225 conversation_cache_test: evict_oldest with pinned entries 2026-10-07 20:37:15 +02:00
alanthinker 940011dad9 serve: a full conversation cache evicts to pass the physical-RAM gate instead of dropping the snapshot
The parking path has two gates in series: make_room() balances the cache's own budget, then conversation_memory_admit() checks the host's free RAM against the snapshot plus --conversation-cache-min-free-mib. The second was final: a failure dropped the snapshot even though the cache held conversations whose RAM it could have, and the next request read its whole prompt again (270k tokens, 96 s, measured by the reporter in #1347).

Evict the least recently active parked conversations until the gate admits the snapshot, bounded by the slot count; entries that hold a pinned shared prefix are not evicted. The failure line says how many were evicted and how many remain.

Reported with the fix by alanthinker in #1347.
2026-10-07 20:37:05 +02:00
Niko1221 d7932123e1 serve: #1317 part 2 quick watchdog for a frozen engine (90 s silent, no CPU or disk work; STRATA_ENGINE_STALL_S); #1344 several API keys (comma list or config list) 2026-10-07 20:36:40 +02:00
Niko1221 f23913a0ac tests: expert_multi_test skip message had a literal newline in its string 2026-10-07 20:33:49 +02:00
Niko1221 f780961c44 fix the escape in the lend-coverage warning (#1389) 2026-10-07 20:32:56 +02:00
Go Kudo 61a76de08d hip: hipBLASLt tuning table for gfx1150 (Radeon 890M, ROCm 7.14.1 / hipBLASLt 100401)
The 16 dense GEMM geometries a Qwen3.8-Flash-Next prompt logs on a Radeon 890M (gfx1150, Strix Point) at
T = 64 .. 16384, 144 rows, calibrated with tools/hip/tune_hipblaslt.cpp against AMD's TheRock ROCm 7.14.1 for
gfx1150 (hipBLASLt 1.4.1 cd957402). Two passes back to back; each row keeps the solution with the lowest median of
all six repetitions (a single repetition stalled now and then, and the tool's mean-of-three picks differed in 26 rows
between the passes).

Every row beats plain hipBLAS, 1.55-22.6x per GEMM. End to end on GSQ-RCO IQ3_XXS, three interleaved rounds: prompts
read 1.5-1.7x faster (3.6K tokens 124 -> 216 tok/s, 7K 130 -> 226 tok/s), decode unchanged; hip_prefill_hipblaslt_gemm
passes with it and a server run logged no fallbacks. Setup does not install for gfx1150, so the table is set by hand
(STRATA_HIPBLASLT_TUNING). Method and data: bench/results/2026-10-07-community-gfx1150.
2026-10-07 20:32:04 +02:00
Chen Cheng a0181802b0 generate: say that CUDA_LAUNCH_BLOCKING=1 hangs the verify windows (#1341 #964)
A verify window's graph waits on the GPU for flags the host raises once
cudaGraphLaunch has returned. A blocking launch returns only when the
graph has finished, so with CUDA_LAUNCH_BLOCKING=1 the first window after
a prompt never ends and the watchdog stops the engine, whatever the prompt
length, GPU count or --pipeline-windows. The engine now says so at start,
and the watchdog's line names it. Nothing changes without the variable.
2026-10-07 20:31:54 +02:00
Niko1221 2d6c6ae5ce MMVQ_IL rows table per architecture: kIlArch hook, STRATA_MMVQ_IL_ROWS override, mmvq_il_parity --emit-table, the sm_89 method 2026-10-07 20:31:41 +02:00
Niko1221 83a7dbc0f1 prefill: the stager takes the RAM profile when the GGUF experts' pages are in the page cache (#1353) 2026-10-07 20:29:45 +02:00
Niko1221andblange48 2c7c02ab72 serve: with pipelined groups, a request fills a hole below a busy slot of its group first (#793)
A pipelined group's window spans its slots up to its last busy one, and the engine keeps no pipelined slot as a
conversation cache, so the least-recently-used pick left leading holes ([_ B]) that cost a whole row each.  pick_slot
now fills such a hole first, then follows the spreading order.  The engine reports INFO batch_groups so the server
knows the groups really run pipelined.  serve/test_slot_alloc.py covers the allocator.

Co-authored-by: blange48 <[email protected]>
2026-10-07 20:29:09 +02:00
shy 0391e478ce split: the auto placement sees each stage's PCIe link
The miss cost (190 ms per unit of routed mass, fitted on x16 links) is one
constant for every stage, so --layer-split auto cannot see an asymmetric
pair: on 2x 4090 D with CUDA1 on a physical x1 link (1.6 GB/s measured) it
gives that card 26 layers (K=22) and reads a 29K prompt at ~350 tok/s.

Probe each stage's link inside the search (probe_pcie_h2d_gbps; the
stage-setup probes are skipped when --pcie-frac is given, and the search
needs the reading either way) and scale that stage's miss term by
clamp(20 GB/s / bw, 1, 64).  20 GB/s is pcie_frac_for_gbps' x16 reference;
the lower clamp keeps a symmetric machine on the exact formula the sweep's
K was fitted with (every scale 1.0 -> the objective is unchanged).

With it, auto picks the fit-in-cache boundary on that machine (K=38: the
narrow card's cache holds all of its layers' pairs, zero streaming there)
and reads the same prompt at ~1410 tok/s, ~4x.

The 190 ms mixes the decode CPU-pool cost with the prompt link cost;
scaling the whole per-stage term is the first-order fix (splitting the
constant would need a fresh fit on both a narrow and a wide link).
2026-10-07 20:28:02 +02:00
Niko1221 85a108a751 gdn_rec_parity: same STRATA_GDN_CHUNKED=2 SM-count bypass as the engine copy 2026-10-07 20:27:55 +02:00
Andy 0c7eb37d1b Foresight swap: slots copied in but never used again become reusable once landed (they leaked as 'loading') 2026-10-07 20:26:47 +02:00
Andy 44f650f972 Foresight swap: STRATA_FS_VERIFY test mode (read every landed slot back, compare with its source) 2026-10-07 20:26:47 +02:00
Andy 14f9e4c50b Foresight swap space: per-layer VRAM slots refilled on their own stream with recently missed experts, served as GPU groups at the pool's existing per-layer decision (STRATA_FS_SLOTS, off by default) 2026-10-07 20:26:47 +02:00
Niko1221 b92a44bfc7 peer: STRATA_PEER_NO_P2P=1 test knob (pretend the pair has no P2P) 2026-10-07 20:25:14 +02:00
Niko1221 f0880b3251 STRATA_GDN_CHUNKED=2: skip the 128-SM test so the chunked recurrence can be measured on smaller cards 2026-10-07 20:25:11 +02:00
Jim Routh 8d6e8ca3fc docs: STRATA_PREFILL_STREAM_MIN=128 in the gfx1151 fast configuration
STRATA_PF_FUSED=1 reaches only chunks of stream_all_min() tokens or more (1,024, measured on a discrete card); a
smaller read runs its experts through MMQ. On Strix Halo every expert is resident and MMQ is ~3x slower on a
~1,000-token read, the size of an agent's turn: 441 / 855-token reads 3.1 / 4.0 s -> 0.9 / 1.2 s, a 12-turn agent
replay 3.4 -> 2.1 s per turn (C2T8, --no-prefill-borrow). It changes bits (the fused path's output, which reads of
1,024+ tokens already get), so section 5: teacher-forced KL against today's path 0.016 on reads under 1,024, the same
as fused vs MMQ on longer reads (0.019); reads of 1,024+ unchanged (KL 0).
2026-10-07 20:25:01 +02:00
alexdns1 94f9d169af hip: gfx1151 hipBLASLt tuning table (hipBLASLt 1.4.0)
Setup installs the ROCm 7.14.0a20260608 wheels for gfx1151 since 0.1.40.2 (#1267); their hipBLASLt is 1.4.0
(100400), which neither gfx1151 table matched, so setup's engine read the prompt through plain hipBLAS.
tune_hipblaslt --workspace-mib 32 on the 100401 table's 90 shapes, on a Ryzen AI Max+ 395 (Radeon 8060S, gfx1151,
128 GB, Fedora 44, kernel 7.2.8) with those wheels: 3.1-14.1x faster than plain hipBLAS per shape (median 6.2x).
hip_prefill_hipblaslt_gemm and hip_prefill_hcd_exact_parity pass with it (fallbacks=0); HIP ctest 61 of 62
(ple_parity needs the Q2_0 fixture). setup picks it up with no code change.
2026-10-07 20:25:01 +02:00
Niko1221 14d9d2e1a5 Warn when the prompt path's lend coverage is short; document STRATA_RESIDENT_HEADROOM_GIB and the long-prompt method (#1389)
Below N of N lendable slots the uncovered experts are read from the pack during a long prompt (reporter: 18% uncovered = -26% prompt speed on gfx1100). A WARNING line now says so, and docs/AMD_HIP.md documents the headroom knob and the levers from the report. No behaviour change otherwise.
2026-10-07 20:25:01 +02:00
sergiywith 0f78032fdc DeltaNet recurrence: STRATA_GDN_CHUNKED=1 - the prompt's recurrence in 32-token chunks (opt-in, 1.8x faster, other bits)
The gated delta rule in the WY form flash-linear-attention uses, all FP32: gdn_chunk_prep_kernel forms k.k / q.k and
T = (I + A)^-1 for every chunk of a 2048-token super-block at once; gdn_chunk_scan_kernel (128 blocks, a key head x
16 value columns each) walks the chunks with its state slice in registers.  Every exponent <= 0 (the gates are
negative).  The scratch (25.6 MB) is allocated once per device: freed and reallocated per call it cost more host time
than the kernels saved on short prompts.  sm_80+, >= 128 SMs, enough shared memory, chunks of >= 128 tokens; else
gdn_rec_kh_kernel.  Unset: unchanged.

gdn_rec_parity: against gdn_rec_kh_kernel <= 1.13e-6 (T = 1..4099, 8192, 32768, a non-zero starting state); against
an FP64 recurrence <= 1.05e-6 (the sequential kernel <= 3.2e-7) - FP32 rounding; 199 restarts of 7 shapes bit for
bit (NaN-filled scratch between).  RTX 5090: 1.81x the recurrence at 2K-32K; IQ2_XS: the recurrence phase of a 32K
prompt 520-532 -> 306-334 ms (-210 ms, -3.7%), 8K -2.1..-3.6%; first-token KL 0.0016-0.0152, 1.15-1.83x a harmless
control arm (STRATA_PROMPT_ATTN_V1), the same top token.
2026-10-07 20:24:14 +02:00
Thig 59ea77e5b3 prefill: the no-P2P hand-off copies on a stream of their own
The MoE input and the routing weights go to host memory on a stream beside the primary's compute stream, so its
own experts start at once instead of behind the copies over its link (mixed_h and w are rewritten only after this
layer's combine, which waits for the peer).  STRATA_PF_PEER_XSTREAM=0: the compute stream.  With
STRATA_PF_PEER_RING=160, prompt tok/s: 100K 1,854 -> 1,918; 8K and 3.4K unchanged.
2026-10-07 20:24:05 +02:00
ThigandEddoursul da0024fed1 prefill: the peer's per-token sums in one launch per group instead of one per expert
Each group's tokens and their row lists (in the experts' order) are built on the host from the rows' pairs and
uploaded once per layer; peer_gather_add is eddoursul/Strata's moe_gather_add with the weight looked up through
the row's pair, so the sums are bitwise those of the per-expert launches.  STRATA_PF_PEER_GATHER=0: per expert.
Prompt tok/s against the per-expert launches: 3.4K 825 -> 832, 8K 1,468 -> 1,500, 100K 1,821 -> 1,858.

Co-Authored-By: Eddoursul <[email protected]>
2026-10-07 20:24:05 +02:00
ThigandEddoursul 550159fc59 prefill: the no-P2P sums mode's two transfers in FP16
The MoE input goes out as the FFN input's existing FP16 image (mixed_h) and the per-token sums come back as
FP16, saturated at +-65504 (eddoursul/Strata f8de703's FP16 input, 2acfac4's sums_to_f16): half the bytes over
the primary's link both ways.  STRATA_PF_PEER_F16=0: FP32.

Peer stream 0.6, prompt tok/s: 3.4K 805 -> 831, 8K 1,284 -> 1,495, 100K 1,578 -> 1,855 (the share off: 1,712).
The 8K and 3.4K answers stay byte-identical to the share off.

Co-Authored-By: Eddoursul <[email protected]>
2026-10-07 20:24:05 +02:00
ThigandEddoursul f4bfedc5ce prefill: the no-P2P peer share sends each token's weighted sum back instead of its rows
The design is eddoursul/Strata's second-GPU prompt path (f8de703, 2acfac4, e8ade66): the second GPU adds its
experts' rows into each token's weighted sum and the first GPU's combine adds that to its own experts' rows.
Over the host route the peer scatter-adds its rows into per-token sums (peer_scatter_add, after the fork's
moe_scatter_add: one launch per expert in a fixed order, so the sums are repeatable) and sends T x N floats
back instead of rows_peer x N (up to K = 10 times more); moe_combine_peer adds them to the primary's rows.  The
2a row buffers are not allocated (3.4 GB of pinned RAM at 32K-token chunks).  STRATA_PF_PEER_SUMS=0: rows.

Same setup, peer stream 0.6, prompt tok/s against the share off: 3.4K 332 -> 805, 8K 762 -> 1,284, 100K
1,712 -> 1,578.  The 8K and 3.4K answers are byte-identical to the share off (greedy, 256 / 3,221 tokens).

Co-Authored-By: Eddoursul <[email protected]>
2026-10-07 20:24:05 +02:00
Thig 81af1da0b1 prefill: --peer-device's prompt share without P2P, through the layer split helper's mapped host route
set_peer refused a peer that cannot access the primary directly, so on GPU pairs without P2P (most GeForce
pairs) the prompt path stayed on the primary GPU.  Without P2P the share now takes the route the layer split's
helper already uses: the activations and the result rows through mapped pinned host buffers read by copy
kernels, with the compact group buffers (the layer-sized path copies over P2P only).  STRATA_PF_PEER_HOST=0
restores the refusal.

RTX 3090 (PCIe 4.0 x4) + RTX 3080 Ti (x16), no P2P, UD-Q4_K_XL, --prefill auto:32768, peer stream 0.35 / 1.0,
prompt tok/s against the share off: 3.4K 332 -> 492 / 551, 8K 763 -> 907 / 776, 100K 1,713 -> 1,072 / 734.
The peer's rows coming back over the primary's x4 link cost more than they save on long prompts; the next
commit sends sums instead.
2026-10-07 20:24:05 +02:00
Niko1221 344ea1cd4b tools: check_isa_guard flags std:: instantiations only inside the wide-ISA objects 2026-10-07 20:23:53 +02:00
Niko1221 f02c244628 tests: expert_multi_test skips on a CPU without the AVX-512 kernel's features 2026-10-07 20:23:53 +02:00
Niko1221 90dd021a70 tests: AVX2 dispatch never enters wider code (#795)
- tools/check_isa_guard.py disassembles the CPU kernels (objdump + nm on the Linux kernel library, capstone + the /MAP
  file of a linked binary on Windows) and fails when a function holding AVX-512 / AVX-VNNI-class instructions is
  not a kernel of a wide-ISA file: a weak or COMDAT copy (the linker may pick it for every caller), a std:: /
  template instantiation, a probe or helper that every CPU runs, or code in another object.  MSVC's own
  `__isa_available` vectorizer guard is accepted.  ctest isa_guard_check runs it; exit 77 (skip) when the tools
  are missing.
- -DSTRATA_SDE=<sde> adds sde_{hsw,adl}_* tests: the CPU parity tests on an emulated Haswell (AVX2 only) and Alder
  Lake (AVX-VNNI, no AVX-512); SDE stops at the first instruction the chip lacks.
- tools/test_check_isa_guard.py: unit tests of the policy.
2026-10-07 20:23:53 +02:00
Niko1221 680d499291 cpu: keep the AVX-512 probe and the "supported?" helpers out of the wide-ISA files (#795)
expert.cpp and iq_avx512.cpp are compiled for AVX-512, iq_avx2.cpp and kq_avx2.cpp for AVX2, and a file compiled
that way may use the wide ISA anywhere in its code.  Code that every CPU runs was still defined in them:

- cpu_features, CpuFeatures::reason, cpu_require_expert_support (the function that finds out whether the CPU has
  AVX-512) and expert_set_oracle_q8_0 / expert_oracle_q8_0_enabled (the first call of every generate, on any CPU)
  were in expert.cpp;
- iq512_supported, iq256_supported and kq256_supported (a comparison native_gu_rows asks before it has checked
  what the CPU can run) were in the kernel files; iq512_supported was called on every AVX2-only CPU for every
  gate/up call.

They now live in expert_layout.cpp and native_expert.cpp, which are built for the baseline.  No kernel changes;
the compiled code of the moved functions is identical, so default output is byte-identical.
2026-10-07 20:23:53 +02:00
Niko1221 d5ea713374 Version 0.1.40.3: MIN_ENGINE 0.1.40.3; Intel Arc setup text says supported (B70, A750) instead of 'no Intel card here' 2026-10-07 18:20:34 +02:00
Niko1221 34ff00bdd0 Merge branch 'f-hf-small' into f-1402 2026-10-07 18:16:39 +02:00
Niko1221 6d83a1ace3 Merge branch 'f-hf-intel' into f-1402 2026-10-07 18:16:39 +02:00
Niko1221 8aeefca2d8 Merge branch 'f-hf-misc' into f-1402 2026-10-07 18:16:38 +02:00
Niko1221 88d090bbef Merge branch 'f-hf-amd' into f-1402 2026-10-07 18:16:38 +02:00
Niko1221 41016e3221 Merge branch 'f-hf-nv' into f-1402 2026-10-07 18:16:38 +02:00
Niko1221 3581f853a6 sycl/tools/build.sh: executable in git (docker run ... sycl/tools/build.sh failed with Permission denied) 2026-10-07 18:16:10 +02:00
Niko1221 c5db5ea803 docs: Intel install path from zero (Docker, ./setup.sh --backend sycl), the missing How to build it section, the Arc A-series path 2026-10-07 18:15:57 +02:00
Niko1221 c454a4bddd sycl setup and serve: Intel wording in setup's output, an explicit --vram-reserve-mib is kept, the Arc A-series (i915) gets a config that starts (no --stream-experts, --ple-io ram, 300 MiB reserve, English draft vocabulary, NO_HOST off, FP64 emulation keys), strata-sycl.sh forwards STRATA_/ONEAPI_/UR_/IGC_/SYCL_/ZES_ variables, the start narrator no longer warns about a slow PC when experts are streamed 2026-10-07 18:15:57 +02:00
Niko1221 013015934a sycl engine: the device waits long enough for the host on an A-series card (first request after a start gave token 0 or crashed the CPU pool), STRATA_VERIFY_NO_HOST=0 means off, version follows the project, no CUDA-runtime warning, no FIXME placeholder in the verify lines; build.sh is executable and takes SPIN_MAX 2026-10-07 18:15:56 +02:00
Niko1221 c13ddd62d5 web: send an assistant turn with no answer text back in the history (#1392); docs: STRATA_STAGER_SLEEP numbers (#1101)
Reported-by: demetree
2026-10-07 18:05:29 +02:00
Niko1221 5600305998 docker: keep a config link that points into /data/config, add CONFIG, print the config used (#1244) 2026-10-07 18:04:02 +02:00
Niko1221 4895f63a61 #1376: the Windows auto-cache floor applies to HIP builds only; CUDA on Windows keeps its sizing (no change measured there) 2026-10-07 17:35:33 +02:00
Niko1221 d7c5dd0fc5 dashboard: say why a Windows AMD card shows no load or VRAM instead of an empty tile or 'not readable (NVML)' (#1380) 2026-10-07 17:34:36 +02:00
Niko1221 044c59fee6 Windows: an auto expert cache on a card of 16 GiB or more keeps 2560 MiB free after the cache (#1376)
RX 7900 XT 20 GB, IQ3_S, Windows HIP, auto cache, default reserve 700: 0.1.40 ended with 4,255 slots (8.12 GiB) and 2,170 MiB free, 0.1.40.2
with 5,512 slots (10.48 GiB) and 323 MiB free; WDDM paged and decode fell from 51.7 to 21-25 tok/s. With --vram-reserve-mib 2600: 4,299 slots
(8.20 GiB), 2,224 MiB free, long prompt 1,015.8 -> 1,355.9 tok/s (+33.5%), decode 24.3 -> 55.5 tok/s (+128%), back inside 0.1.40's ranges.
A reserve given on the command line is kept; a smaller card keeps its own sizing. The cache size changes no output bit.
2026-10-07 17:34:36 +02:00
Niko1221 4b3ff7a1b4 Windows HIP: setup and the package put the whole dependency closure of amdhip64_7.dll and amd_comgr.dll beside strata.exe, and the engine says why the bundled runtime did not load (#461)
amdhip64_7.dll imports rocm_kpack.dll, which imports the MSVC runtime. Only the two runtime DLLs were copied beside the exe, so a
full-path load of the bundled one failed with error 126, Windows fell back to System32's copy, and big prompts died with
hipErrorInvalidDeviceFunction (RX 7800 XT, gfx1101). The closure is now read from the PE import tables of the files in the zip
(tools/hip/dll_closure.py, no pefile), by setup.hip_runtime_beside_exe and tools/hip/package_windows.py. On the 0.1.40.2 HIP zip it is
amdhip64_7, amd_comgr, rocm_kpack, msvcp140, vcruntime140, vcruntime140_1. The engine, when the loaded runtime is not the one beside it,
also tries LoadLibraryEx on the bundled one and prints GetLastError. Diagnostic only. Not tested on a Windows AMD card.
2026-10-07 17:34:36 +02:00
Niko1221 4dc544eda7 tokenizer: _bpe reads self once instead of per symbol pair, and a threads-vs-serial encode test (#1385; the 'int has no attribute ranks' fault could not be reproduced and looks like an interpreter bug) 2026-10-07 17:21:48 +02:00
Niko1221 668072aa49 Revert "Verify window: the interleaved q8_1 projections (STRATA_MMVQ_IL) are opt-in again"
This reverts commit f7610d2c32.
2026-10-07 17:18:45 +02:00
Niko1221 ca016f230d sycl mtp: the same n_expert == 512 and k == 10 guard on the per-token native router call
Reported-by: rwkeyes (#1357)
2026-10-07 16:16:11 +02:00
Niko1221 6c0fca38ee mtp: guard the drafter's per-token native router call with n_expert == 512 and k == 10
The native router reads 512 logits a row; the window path two lines above already tested n_expert == 512 and K == 10, the
per-token fallback tested only native_router_enabled(), so a model with fewer experts read past each row. It now takes the
generic router_top10 like its siblings. native_multi_parity checks the generic router on 256-wide rows.

Reported-by: rwkeyes (#1357)
2026-10-07 16:16:11 +02:00
Niko1221 821651a44e engine: fix the newline in the kfd permission message 2026-10-07 16:15:55 +02:00
Niko1221 9993c62209 setup: arch-exact iGPU text (gfx1103 is not Strix Halo), warn on a missing render group or /dev/kfd access, engine names it instead of blaming another program; troubleshooting entry for Windows AMD driver resets 2026-10-07 16:14:03 +02:00
Niko1221 f7610d2c32 Verify window: the interleaved q8_1 projections (STRATA_MMVQ_IL) are opt-in again
0.1.40.2 turned fork F4 on by default after single-GPU measurements (+4.2% on an sm_86, +1.3 to +2.2% on an sm_120). A
two-GPU Windows layer split (RTX 4080 SUPER + RTX 5070 Ti) was reported 15% slower with it on (#1373). The outputs are
bitwise the same either way, so only speed changes. STRATA_MMVQ_IL=1 turns it on; docs/DETAILS.md says why.
2026-10-07 16:13:19 +02:00
Niko1221 e8ca9afd03 setup: engine versions compare all four numbers and MIN_ENGINE is 0.1.40.2, so an installed 0.1.40 engine is replaced by the new one 2026-10-07 13:22:57 +02:00
Niko1221 b8617c4b63 Merge branch 'f-141-pf' into f-1402 2026-10-07 12:40:07 +02:00
Niko1221 8f70a52cfa Merge branch 'f-141-intel' into f-1402 2026-10-07 12:35:33 +02:00
Niko1221 3838a40a0c sycl: the sampler's top_p/temperature tail accumulates in float (FP64 is emulated on Arc Alchemist); B70 IQ3_S findings in docs/INTEL.md
samp_acc_t is float by default (-DSTRATA_SYCL_SAMPLER_FP64=1 keeps the reference's double). On an Arc A750 one row of the
tail cost 451 us in double and 20 us in float; a sampled decode ran at 7.62 -> 8.10 tok/s (medians of 5 interleaved
pairs, runs 7.1-9.2). sampler_parity: 0 failures on the A750 (JIT build) and on the B70; both cards' 10-prompt gates pass
10 of 10, and the B70 gate's sampled cases pick the same tokens as before. Of the kernels that use double, only the
sampler runs on a native pack (the router, combine, gate and norm kernels there are float).

Probes: sycl/probe/fp64_cost.cpp (FP64 against float in the small reduction kernels), gemm_vary.cpp.
docs/INTEL.md: the aliased-pages bug, NO_HOST, the prompt-chunking numbers, draft-length sweep, FP64 and XMX findings;
docs/INTEL_ARC.md: a row for the B70/A750 runs.
2026-10-07 12:22:27 +02:00
Niko1221 f1ed65b92e docs: STRATA_PREFILL_CPU_SHARE (opt-in) with the measured prompt times and its output difference 2026-10-07 11:51:33 +02:00
Niko1221 4322e2411f prefill: STRATA_DBG_NAN does not count the CPU-share rows of GU and H (the GPU never writes them)
With STRATA_PREFILL_CPU_SHARE on, the last rows_cpu rows of GU and H hold whatever the buffer held before; the check read
them and reported them as non-finite (about 1 in 256 garbage floats), with Dm and the layer output clean.
2026-10-07 11:51:33 +02:00
Niko1221 667f2ecadf prefill: the CPU-share thread takes no ExpertSource call (blobs taken on the prefill thread, blob_stable)
blob() bumps a plain reads_ counter and, for a file source, its mapped/staged paths; calling it from the share's
thread raced with the prefill thread's own blob_stable()/pinned() calls (the streamed experts). The blobs of the CPU's
experts are now taken before the thread starts, so the thread reads only the arena bytes, the activations and the
grouping tables, none of which the prefill thread writes until the thread is joined.
2026-10-07 11:51:33 +02:00
sergiywith 978d355852 prefill: STRATA_PREFILL_CPU_SHARE=auto hands a small chunk's least-routed experts to the idle CPU pool
A chunk below stream_all_min() (1,024 tokens: an agent's tool result, a test's
output) streams every routed expert outside VRAM over PCIe while the decode CPU
pool sits idle and the arena holds the same experts in RAM.  Opt-in,
STRATA_PREFILL_CPU_SHARE=auto|x:
- the non-resident experts routed by at most MAXT of the chunk's tokens,
  fewest first, run on the pool on a thread while the GPU streams and runs the
  rest; their unweighted rows go to Dm's tail (as a peer GPU's do) and the
  combine weights them as any other row;
- auto: the share measured, each layer's CPU time per expert and the GPU's per
  streamed expert (CUDA events around its expert work, read after the next
  layer's routing sync), and the next layers hand the CPU g / (c + g), where
  both end together - a small CPU takes a small share;
- Q2_0 layers take the pool's Q2_0 kernels' activations (ActQ, as decode's),
  i-quant layers the native vec_dot type.

Serve without batch slots and generate, one GPU.  Unset (the default): no
expert moves, byte-identical to v0.1.39 (IQ2_XS 600 and 2,100 tokens, fixed
cache, logits and 32 tokens).

RTX 5090 + 9950X3D, 600-token prompts, ms (off / auto):
- UD-Q4_K_XL: 1,548 / 1,369, 1,528 / 1,296 (share 0.65; KL between 0.006);
- Q2_0: 472 / 445, 491 / 448;
- IQ2_XS: 493 / 481, 502 / 460, 527 / 529; 250 tokens 424 / 355, 402 / 382,
  379 / 367 (KL to STRATA_PREFILL_MMQ=0 0.0053 off, 0.0080 auto);
- IQ2_XS on 2 AVX2 workers (emulated): 512 / 487 (share 0.49).
2026-10-07 11:51:33 +02:00
Niko1221 9476ef1ade Merge branch 'f-1402-dec' into f-1402 2026-10-07 11:43:30 +02:00
Niko1221 960de8bb3d Engine SHA-256: warn and install when no digest is available, STRATA_SKIP_SHA256=1; docs note for STRATA_SH_STREAM (#1218 #1272) 2026-10-07 11:42:58 +02:00
Demetree Kallergis cf5fa3af91 Check the ready-made engine against GitHub's SHA-256 before installing it (#1218) 2026-10-07 11:42:03 +02:00
Niko1221 6fd12038b1 sycl: no short first prompt chunk while part of the experts is in the host mirror; setup asks a 24 GB+ card for 4096-token chunks
Every prompt chunk streams the experts the VRAM cache does not hold, over PCIe, and a 4K prompt touches nearly all 512
experts of a layer. The short first chunk (STRATA_PREFILL_FIRST, 256 tokens, there to start the GPU while PLE rows are
read) therefore cost a whole extra pass of that stream whenever the cache holds only part of the model. Its default is
now 256 only when the cache holds every expert, else 0 (an explicit STRATA_PREFILL_FIRST still decides).

Arc Pro B70 32 GB, IQ3_S (11.5k of 24.6k experts in VRAM, the rest in the pinned mirror at 13 GB/s), a 4,095-token
prompt, MTP on, interleaved runs, medians of 3: 2048-token chunks 618 -> 730 tok/s, 4096-token chunks 784 -> 980.
sycl/setup_intel.py now writes --prefill 4096 for a card with 24 GB or more (it costs ~700 expert cache slots, about 3%
of decode on the 200-token story).
2026-10-07 11:31:11 +02:00
Niko1221 c4c3221c97 Version 0.1.40.2 2026-10-07 11:29:31 +02:00
Niko1221 57747c1aa9 Merge branch 'f-1402-s' into f-1402
# Conflicts:
#	serve/server.py
2026-10-07 11:27:21 +02:00
Niko1221 2b87574b30 serve: STRATA_ENGINE_READY_S / STRATA_VISION_READY_S / STRATA_VISION_ENCODE_S override the #1317 read timeouts (0 waits for ever, as before) 2026-10-07 11:26:48 +02:00
Niko1221 78f6174474 sycl: guard big device allocations against aliased pages on the xe driver (Arc Pro B70)
An Arc Pro B70 (xe) returned the 22 GiB expert-cache arena with two of its 2 MiB pages mapped onto the same memory:
a write to the page at +2 MiB showed at +1022 MiB, so cache slots there held other experts' bytes and the prompt
came out NaN at the first layer that routed to one of them (layer 4, expert 249). Which pages alias depends on the
allocation history (the context size, the MTP window), so it comes and goes between runs.

strata::arena_alias_check tags every 64 KiB of a fresh range and reads the tags back in a second kernel;
strata::malloc_device_guarded allocates again behind a growing spacer until the range is clean (and warns if it
never is). It guards the expert cache arena and every device allocation of 32 MiB or more (session arena, weights,
MTP, verify, head, prefill). STRATA_ARENA_ALIAS_CHECK=0 skips it. Two debug aids: STRATA_VERIFY_ALL_SLOTS=1 (with
STRATA_VERIFY_FIND=1, STRATA_VERIFY_DUMP=dir) reads every filled slot back and compares it with the GGUF, and the
probes sycl/probe/arena_rw.cpp and alias_scan.cpp.
2026-10-07 10:53:07 +02:00
Niko1221 ec7f6a0275 serve: a restart clears max_context before the new engine process is visible, and close() does not join an output thread a restart never started 2026-10-07 10:35:58 +02:00
Niko1221 5e4de5c738 docs: STRATA_SPEC_GUMBEL on an RTX 3060: acceptance up, decode within noise
12 interleaved pairs at temperature 1.0 with MTP, default / coupled / coupled + Gumbel: accepted drafts 62.1 / 66.9 / 68.0%, median decode 43.4 / 43.3 / 42.8 tok/s.
2026-10-07 10:15:44 +02:00
Niko1221andbtc1000w 02be84a09c serve: time out the untimed reads of the engine's READY line and the image encoder's pipe (#1317)
A blocking read of the engine's READY line or of the vision encoder's pipe that
never returned held the request turn for good: the engine sat idle in its command
loop, so no watchdog could fire and every later request queued behind it (the lost
step class of #481). The READY read now has a 900 s timeout, the encoder's READY
and ENC reads 300 s each. A process that says nothing for that long is killed, the
read raises, and the next request starts a fresh one. Slow loads and slow encodes
stay far inside the limits.

Only the read timeouts of the reporter's patch are taken. The watchdog that ends
an engine after 90 s without output and CPU progress is left out for 0.1.41.

Tests: a silent engine, a silent encoder at READY and at ENC, and an in-time answer.

Reported-by: btc1000w
Co-authored-by: btc1000w <[email protected]>
2026-10-07 10:15:25 +02:00
Niko1221 e9e1757a7f hip: one-line start hint for STRATA_HIP_WMMA=1 on gfx12 with --kv int8
A hint only: it never enables the kernel and says the output bits change. Silent when the variable is set to anything. Follows the R9700 numbers of PR #1318 (+31% prompt at 8.8K).
2026-10-07 09:09:40 +02:00
l33tm4st3r 3512797dba docs(AMD_HIP): what a gfx1201 hipBLASLt 1.4.1 table measured on my R9700
I calibrated a table for hipBLASLt 1.4.1 with tune_hipblaslt over the 32 geometries of
gfx1201-hipblaslt-100500.txt (R9700, system ROCm 10.1.0, hipBLASLt 1.4.1): 1.15x geometric
mean per GEMM over hipBLAS, but only 1,606 -> 1,613 tok/s median prompt over three matched
trials (a 1.008x geometric mean) and decode unchanged, so within noise - which is what the
R9700 paragraph above already reported, now from a second machine. The cold first prompt of
each start is left out of those numbers.

I dropped two rows from it: bf16 N=10240 K=320 ldy=10240 at T=4096 and at T=8192. hipBLAS
won T=8192 in five separate runs (0.609-0.619 ms against 0.648-0.677 ms for the best of 16
candidates), and dropping only the T=8192 row would send that call to the T=4096 row rather
than back to hipBLAS, because closest() takes the nearest T bucket.

Two calibration traps I ran into are recorded too: the first case of a fresh tune_hipblaslt
process times its hipBLAS baseline while the clocks are still ramping, and the first fresh
prompt after a model load is not a measurement (949 vs 1,423 tok/s in the two arms there).
2026-10-07 09:07:39 +02:00
l33tm4st3r bc34b89e50 docs(AMD_HIP): STRATA_HIP_WMMA on my R9700: +27% prompt, decode unchanged
On my R9700 (engine 0.1.40, IQ3_S at 431072 ctx, --kv int8, the hipBLASLt 1.4.1 table of
the previous commit, GPU idle): fresh prompts of 4,210 and 8,830 tokens, three matched
trials each and the cold first prompt of each start left out - 1,571 -> 2,064 and
1,568 -> 2,065 tok/s at 8,830 tokens (+31.4 and +31.7%) and 1,699 -> 1,990 at 4,210
(+17.1%), medians 1,571 -> 2,064, a 1.27x geometric mean over those three - with decode
unchanged (89.8 -> 86.9 tok/s). hip_prompt_attn_wmma passes on this card.

STRATA_SELECT_WMMA=1 on top of it changed nothing at 4-9K prompts (2,026 tok/s), which is
consistent with its +1.5% being a 16K/262K-ctx number.
2026-10-07 09:07:39 +02:00
Leon Strong a45783c7f9 hip: the gfx1100 100401 table covers the small-T expert GEMM (prompt +31.9%)
The shipped gfx1100-hipblaslt-100401.txt calibrates "the 26 dense GEMM
geometries of the shipped gfx1100 table", all at T=4096/8192. A prompt on
Qwen3.8-Flash-Next reads a whole other geometry through the prompt path, and
it is issued once per expert group at whatever T the group has - so it never
matches a row:

  STRATA_HIPBLASLT_VERBOSE=1, GSQ-RCO IQ2_XS, 192000 ctx, 3151-token prompt
    hipBLASLt tuning enabled (26 rows, gfx1100, version 100401)
    642 fallbacks, all f16 N=1280 K=2560 ldy=1280, over 642 distinct
    T values from 1 to 1873

That is the small-T expert GEMM, and it is nearly every launch in the prompt.
TuningTable::closest() matches on (n, k, ldy) and takes the nearest t_bucket,
so the rows below are a spread of buckets over the small-T range rather than
one row per T; bf16 10240 2560 10240 (the hyper-connection projection, one
fallback) is covered the same way.

tools/hip/gfx1100-hipblaslt-100401.txt: 17 rows added with tune_hipblaslt
from this tree, against the engine's own 32 MiB workspace (gemm.cu:441), on
the same RX 7900 XTX and packaged ROCm 10.0.0 the existing rows were
calibrated on. No existing row is changed. 26 rows -> 43.

Same binary, same config, the table as the only difference, 3 interleaved
pairs at a 3151-token prompt:

  691.6 -> 940.7 tok/s  (+36.0%)
  712.3 -> 943.2 tok/s  (+32.4%)
  727.7 -> 928.7 tok/s  (+27.6%)
  mean 710.5 -> 937.5 tok/s (+31.9%)

Decode unchanged (58.6 -> 60.6 tok/s, inside this box's 2.3% run drift), and
STRATA_HIPBLASLT_VERBOSE=1 reports fallbacks=0 with the 43 rows, which the
26-row table does not.
2026-10-07 09:07:39 +02:00
Jim Routh 2088bd45f5 docs: coupled drafts with Gumbel-max picks (STRATA_SPEC_GUMBEL), measured on Strix Halo 2026-10-07 09:05:11 +02:00
Jim Routh 6381df34f6 sampler: Gumbel-max coupled drafts (STRATA_SPEC_GUMBEL=1, opt-in)
The pick becomes argmax_i p_i / E_i with E_i = -log(u_i) and u_i a hash of (seed, counter, token id), an exact
sample of p.  The noise a token gets does not depend on which other tokens survived the draft's or the target's
cut, so a coupled draft (STRATA_SPEC_COUPLED=1) agrees with the target on the tokens they share.  Off by default:
nothing changes without the variable.  sampler_parity: kernel vs host reference under three chains, frequencies
vs softmax, the flag is read and greedy is unchanged.
2026-10-07 09:05:11 +02:00
Niko1221 29897d780f docs: DETAILS links VRAM_ELASTIC.md from the vram_elastic paragraph 2026-10-07 04:17:28 +02:00
Niko1221 7dbd8c09fc Merge branch 'f-1402-d1' into f-1402 2026-10-07 04:15:44 +02:00
Niko1221 bd5b1a2789 tests: pdl_parity runs with STRATA_DF_PDL=1 (PDL is opt-in; without it the test's forced PDL capture reads as unsupported) 2026-10-07 04:15:44 +02:00
Niko1221 2a8cdeddda prefill: the Stager's sleeping waits (#1057) are the default on Linux only; Windows keeps the yield spin (5070 8K Q2_0 prompt -1.2% with them). STRATA_STAGER_SLEEP=1/0 forces either 2026-10-07 04:08:11 +02:00
Niko1221andFrancesco Albano 2e4ddf6e63 verify window: PDL on sm_90+, mixer graph branches, batched Q4_0 query path, PLE post-ops in one pass (all opt-in)
Ported from #904 onto the 0.1.40 verify window. Every switch is OFF by default, so
default output and the default launch sequence are unchanged:

- STRATA_DF_PDL=1: programmatic dependent launch (pdl.hpp) for the window's
  quantizations and dense projections (multi-column MMVQ, multi-row BF16 GEMV);
  sm_90+ only, CUDA 12.3+; HIP and older cards keep ordinary launches.
- STRATA_DF_PLE=1: the window's hyper-connection write and PLE post-ops in one
  pass, which also writes the commit's history snapshots (native_gr_post_multi
  with strides, native_ple_postops_batch_snap).
- STRATA_DF_QB_Q4=1: a Q4_0 KV cache takes the batched QSA query path too.
- STRATA_DF_BRANCH=1: the mixer's side branches inside the window graph.
  Left opt-in because the author saw a stall on Linux with NVML queries.

The q/gate split now uses copy_rows_strided (a kernel) instead of a
cudaMemcpy2DAsync so a PDL chain stays kernel to kernel; the bytes are the same.

Left out: STRATA_DF_KVAPP (kv_append_q4_step_multi). 0.1.40 already appends a
window's K/V cells in one launch (kv_append_q4_steps, #783 PR-d).

Merged by hand with the 0.1.40 forms of the same code: the exact-T BF16
multi-token kernel, the PAIR/TS MMVQ variants, QFUSE (q8_attn), the fused
norm+rope and the batched residual write.

Co-authored-by: Francesco Albano <[email protected]>

(PR #904)
2026-10-07 04:08:11 +02:00
Niko1221andStefanos Anastasiou d6850f31ee cpu: IQ3_S and IQ2_S AVX2 decode without per-index shift/mask/or
The high index bits of a 32-value half are spread one per byte by a table and
punpcklbw puts them next to the low index bytes, so each grid lookup is a
16-bit load plus the table load. The sign vector is one broadcast load plus one
pshufb, and the (2s+1) scale vectors are table loads. Arithmetic is untouched
(same words in the same lanes). The decode is ported onto the 0.1.40 layout of
iq_avx2.cpp: the gathered IQ3_S/IQ2_S forms (Fmt32<121>, <122>) and the VNNI
rows are left as they are.

Co-authored-by: Stefanos Anastasiou <[email protected]>

(PR #930)
2026-10-07 04:07:51 +02:00
Chromadera 818ec1c6e9 expert cache: --expert-cache-per-layer sizes each layer's slots to that layer's own blob (native packs)
Opt-in (only with --expert-cache-per-layer). The same VRAM holds more experts
(7,008 -> 9,312 slots at equal VRAM per the author). --expert-cache N stays the
budget of N largest blobs; shrink_to keeps whole per-layer lists so a blob never
lands in another layer's range. The free-VRAM reading is device_free_bytes(), as
in the shared-cache branch. The PR's png is left out.

(PR #934)
2026-10-07 04:07:51 +02:00
Unmaple 6b1e303f16 cpu: --pool-tasks N sets the row tasks per batched expert phase
Opt-in (default 0 keeps three tasks per participating thread, so default
output and timing are unchanged). Adds pool_tasks_test, which compares the
split against the default partition bitwise for 0..4096 tasks.

(PR #949)
2026-10-07 04:07:51 +02:00
uncle daddy bc089d5d97 prefill: let Stager threads sleep instead of yield-spinning
The prefill Stager waited by spinning: a yield loop on `issued`, and
cudaEventSynchronize on events created without cudaEventBlockingSync
(which spins under the default scheduler). With --mmap-experts there
are 32 stagers. On Linux they inherit the host thread's single-core
pin and starve it: on a DGX Spark (GB10) prompts were read ~10x slower
(IQ3_S 8K 91 -> 1,222 tok/s, 30K 88 -> 1,388 tok/s with this change).
Unpinned instead, the 32 spinners took every core for the whole prompt
(process CPU ~1,800%); with the waits sleeping it is ~100%.

- ring wait: std::atomic::wait, with notify_all in issued_one() and
  finish()
- dma_done events: cudaEventDisableTiming | cudaEventBlockingSync
- HIP shim: cudaEventBlockingSync -> hipEventBlockingSync (untested)

Decode speed and output tokens are unchanged.
(PR #1057)
2026-10-07 04:07:42 +02:00
Zhong Uncle 49c50f49f7 tools: bench_eviction replays --dump-routing traces against candidate eviction policies
ExpertCache::admit never evicts because eviction policy is a measured
question (R4.1's LFU-decay vs LRU sweep).  This is the harness for that
measurement: a policy's hit rate depends only on the order (layer,
expert) pairs were routed, which --dump-routing records, so replaying a
trace through simulated slot pools needs no GPU.  fill-only (the
engine's current behaviour), lru, lfu-decay and an oracle (Belady's
MIN, the upper bound); global or R4.2g per-layer scope matching
ExpertCache::layer_slot_range; any slot count; optional profile prefill;
per-window rates, so a topic shift shows up as a per-window dip.

tools/test_bench_eviction.py runs without a GPU (14 cases): each
policy's victim choice, the per-layer quota arithmetic, the windows
split, and profile prefill order.
(PR #991)
2026-10-07 04:07:42 +02:00
Niko1221 d19c07650b Merge branch 'f-1402-r' into f-1402 2026-10-07 04:05:58 +02:00
Niko1221 c954f5bd1e Merge branch 'f-1402-p' into f-1402 2026-10-07 04:05:58 +02:00
Niko1221 1be32e9c7b Merge branch 'f-1402-d2' into f-1402 2026-10-07 04:05:01 +02:00
Niko1221 881328bfb2 Merge branch 'f-1402-k' into f-1402 2026-10-07 04:02:23 +02:00
Niko1221andEddoursul ffac97f74f Verify window: 2-4 token dense projections read an interleaved copy of the q8_1 activations (fork F4)
A one-int-a-column interleaved copy of the window's q8_1 blocks lets a warp decode each weight block once and apply it
to every token with the single-column dot's own expression (the same *_q8_dot_impl calls), walking the chunks of the
exact multi-column layout and adding the four virtual warps' sums in warp order, so every output is bitwise the
multi-column kernel's. IQ4_XS, Q4_K, Q5_K and Q6_K projections (and the head) of 2-4 tokens use it; the copy is made
from xq_ by one small kernel, so the fused HC-read producer works too. STRATA_MMVQ_IL=0 keeps native_mmvq's kernels.
CUDA only (HIP keeps its kernels). mmvq_il_parity checks every case bitwise for rows 1/2/4 and the table; --bench times it.

Ported by hand from Eddoursul's fork (738e166, b67a291), with the IQ3_S and Q8_0 parts and the 5-8 column widths left out.

Co-authored-by: Eddoursul <[email protected]>
2026-10-07 04:00:38 +02:00
Niko1221 5adacbc4e1 Merge branch 'f-1402-v' into f-1402 2026-10-07 03:59:01 +02:00
Niko1221 8aa07e5ef0 docs: TROUBLESHOOTING for the update after the history rewrite (#1276), the unbuffered file tier (#1194) and the page-locked copy (#1250) 2026-10-07 03:57:39 +02:00
Niko1221 5738d38d50 resident RAM mode: give the model files' cached pages back before page-locking the expert copy, and when the free RAM still does not cover it take the pages in steps and page-lock what the RAM allows, with a warning, instead of being OOM-killed (#1250) 2026-10-07 03:57:39 +02:00
Niko1221 805088d5b6 file tier on Linux: unbuffered reads only when the cache cannot hold a twentieth of the experts it would read (at least 1.5 GiB); the rest read through it (#1194, #1116, #1085) 2026-10-07 03:57:38 +02:00
Niko1221 58b1ea221a docker: with REINSTALL=0 the config on the /data volume replaces a regular file left in /opt/strata (#1244) 2026-10-07 03:57:38 +02:00
Niko1221 1341e01a94 setup: the sm_120 compiler warning covers CUDA 13.2.0 and 13.2.1 only; 13.2.2 (nvcc build 86) is accepted (#968); tests for it and for the per-family ROCm pins 2026-10-07 03:57:38 +02:00
Niko1221 10f27e6e74 setup: the ROCm wheel is chosen per GPU family from what its TheRock index offers (a pinned version, else the newest of the same line or major, with a warning); gfx103X-all takes 7.13.0a20260515 and gfx1151 7.14.0a20260608 (#1103, #1267) 2026-10-07 03:57:38 +02:00
Niko1221andelanonimo832 495ba07679 UPDATE.bat / update.sh: a clone on the pre-2026-10-06 history is moved to origin/main (old commits kept in pre-cleanup-backup) when no tracked file is edited; otherwise the two commands are printed (#1276)
Co-authored-by: elanonimo832 <[email protected]>
2026-10-07 03:57:38 +02:00
Niko1221 6d69cd5032 Merge branch 'f-1402-u' into f-1402 2026-10-07 03:36:42 +02:00
Niko1221 56ae98dde9 --kv-grow: STRATA_KV_GROW_HOLD=1 keeps the K/V as long as a parked conversation (#1128)
kvg_trim took only the short request's cell count, so every short request in
a long conversation's session shrank the K/V to 8192 cells and gave the
expert slots back, and the conversation's next turn grew it again and moved
the experts again (a parked conversation is restored through kvg_ensure).
With STRATA_KV_GROW_HOLD=1 the trim keeps the longest parked conversation's
cells (+256) while it is parked.  Opt-in: the default trims to the request
alone as in 0.1.40, so the K/V gives back its VRAM as before.  Output is
unchanged; only --kv-grow is affected.
2026-10-07 03:34:33 +02:00
Niko1221 53623e56f3 Merge branch 'f-1402-w' into f-1402 2026-10-07 03:09:28 +02:00
xjc10 ddef182709 bench: community report, 2x RX 6900 XT (gfx1030), Ryzen 5 5600X, IQ3_S, 131K (PR #1270)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:56 +02:00
wolffahrer f012ae3001 bench: community report, RTX 5090 Laptop 24 GB, Windows 11, see README (PR #1263)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:54 +02:00
ManfredCh a58c4f3b5d bench: community report, RTX 5090 Laptop 24 GB, UD-IQ4_XS (first NVIDIA measurement) (PR #1245)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:53 +02:00
KalAbaddon ca2618b920 bench: community report, RTX 5080, Ryzen 9 9900X3D, Coder IQ1_M and others (PR #1243)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:53 +02:00
qni-live 93690a8d53 bench: community report, 2x RTX PRO 4500 Blackwell, Swift IQ3_XXS (PR #1226)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:49 +02:00
cezartolvai f44c540365 bench: community report, 2x RTX 2080 Ti (R9), see README (PR #1225)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:48 +02:00
lucapug 27e2e6f6ec bench: community report, RTX 3090 eGPU, 64 GB RAM, IQ2_XS vs IQ3_XXS (PR #1199)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:48 +02:00
apollo-mg 6024082a33 bench: community report, 4x Tesla P100 16 GB (CUDA 12 engine), IQ3_XXS (PR #1192)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:47 +02:00
nytasuk-commits b67da13702 bench: community report, RX 7900 XTX, Windows 11, IQ3_S (PR #1173)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:47 +02:00
CC-David-CC c819f4a89d bench: community report, RTX PRO 6000, Q8, --prefill 8192 (PR #1171)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:46 +02:00
Dmitry-B d79510700c bench: community report, RTX 4090, IQ3_S 140K / IQ3_XXS 200K opt-ins and tool-call hotfix (PR #1158)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:46 +02:00
lechlna000 08e1363a7e bench: community report, 2x Tesla P100 16 GB, Flash-Next IQ3_S, 128K (PR #1157)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:45 +02:00
jase100k c1daa7b1fb bench: community report, 2x RX 7900 GRE (gfx1100), layer split, #848 / #854 (PR #1140)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:44 +02:00
CC-David-CC 8ca475c223 bench: community report, RTX PRO 6000, Q4 / Q8 (MTP and ngram runs) (PR #1137)
Ported from the community PR with the author kept. Raw dumps and logs were trimmed (TRIMMED.md in the folder lists them); the README, summaries and configs are unchanged. Indexed in bench/results/COMMUNITY.md.
2026-10-07 03:04:42 +02:00
Niko1221andgopinath87607 e6ca0a04ba layer split: with the stage trim, auto prices a later stage's weights per candidate range, projections and cudaMalloc granules included (#1238, #880)
With --layer-split auto and STRATA_STAGE_TRIM=1 a later stage's weights load after the search, and the search priced them
as the full arena with no projections, so a small card was over-credited: P40 24 GB + RTX 3070 8 GB picked K=2 (prefill
107 / 112 tok/s at 4K / 32K) where K=16 gives 343 / 654 in the report. The carve is now priced for each candidate range:
the trimmed arena (WeightTable::pool_bytes) plus NativeDense::weight_bytes_for, a walk of load over the GGUF headers that
rounds each matrix to cudaMalloc's 2 MiB granule (+0.8% on the iq3_s pack, high on purpose). After each late stage loads, the
bytes priced are compared with the VRAM it really took and a warning says when they drift.

Only the late-weights arm changes (auto with the trim asked, opt-in); every other run has its weights loaded before the
search, carve_bytes is 0 there, and placements do not move. Ported from 572b607 of #880 onto this tree's loader (layer ranges
as arguments, STRATA_Q8_PACKED, STRATA_HC_Q8 untouched); not run on a mixed pair here.

Co-authored-by: gopinath87607 <[email protected]>
2026-10-07 03:02:56 +02:00
Niko1221 0499f8f7be engine: a Windows HIP engine says when the runtime in use is not the bundled one beside it (#1261)
The report shows the bundled amdhip64_7.dll next to strata.exe and the System32 copy (a different build) in use, with the
prompt GEMMs failing. The import is bound before main, so the engine cannot swap it; it now prints a WARNING naming the
bundled file when the loaded one is another, so the log carries the fact. Nothing else changes. The fix itself (loading the
bundled runtime by full path, or a delay-loaded import) needs a Windows AMD machine to test and is not in this commit.
2026-10-07 03:02:56 +02:00
Niko1221 0f931943c8 engine: a line before each verify-window capture says the free VRAM, and a Windows start with little room says what to try (#1275)
On Windows an engine with 313 MiB of VRAM free after a 15,104-token borrow exited silently inside the verify-window
capture. The cause is not reproduced here (no Windows AMD card), so this only leaves a trace and a hint, and changes
nothing: the capture prints how much VRAM is free before it starts, and a Windows start with less than 512 MiB free
and --prefill auto names --prefill auto:8192 (the reporter's workaround) and --vram-reserve-mib. The chunk choice is
untouched.
2026-10-07 03:02:56 +02:00
Niko1221andzeriyoshi c0c932cf9d cmake: gfx1150 (Radeon 890M, Strix Point) is an unvalidated HIP target, with a docs row (#1217)
The build stopped with a FATAL_ERROR because gfx1150 was in none of the architecture lists, although the device code
(the WMMA guards, sudot4) already covers it. It is now in the unvalidated list: the build warns and goes on. The
report's machine (96 GB, TheRock 7.14.1, IQ3_XXS: decode 17-19 tok/s, prompt about 125-135 tok/s) is in docs/AMD_HIP.md.
Setup's own AMD path is unchanged.

Co-authored-by: zeriyoshi <[email protected]>
2026-10-07 03:02:56 +02:00
Niko1221andcrazyaimachine 6111382bec vision: a JPEG with an EXIF Orientation other than 1 is turned upright before the encoder reads it (#1229)
Phones save portrait photos as landscape JPEGs with an Orientation tag, and the encoder's decoder ignores it, so the
model saw them sideways. Such a JPEG now takes the Pillow route (exif_transpose, then PNG); JPEGs without the tag, PNG,
BMP and GIF still pass through byte for byte, and without Pillow nothing changes. Other formats that carry the tag
are turned too.

Co-authored-by: crazyaimachine <[email protected]>
2026-10-07 03:02:56 +02:00
Niko1221andcrazyaimachine e21dc577a8 batch: with --batch-groups a second request goes to the other pipeline group, and idle slots no longer cost rows in a group's window (#1249)
On a layer split with --batch-groups, two concurrent requests ran at about half speed: the slot choice could put both in
one group, and a group's window carried a row for every slot, active or not. Slot choice now prefers the group with the
fewest busy slots (a held prefix of 512 tokens or more still wins), and a window covers the slots up to its last active
one. Without --batch-groups the slot choice is exactly as before.

Co-authored-by: crazyaimachine <[email protected]>
2026-10-07 03:02:56 +02:00
Niko1221andfukc-gihtub 09e5ab4f6e generate: --layer-split auto does not choose a split whose prompt path cannot start, and says so when none can (#1094, #1160)
The search priced placements by decode time only, so at 512K context it could pick a split where a stage could neither
lend a 512-token prompt chunk nor afford its own buffers, and the engine ran out of VRAM at start while an explicit
--layer-split worked. A placement is now taken only if every stage can lend the last-resort chunk or afford its own
buffers (the serve path's own tests, priced from the same figures); the tie-break for balanced stages stays. A
start-time probe on the live free figures says which card is short.

Changed from the PR: nothing is refused. If no placement can start on paper, the search runs again without the gate
(the choice is then what it was before) and prints what is short and how to make room; the probe only warns.

Co-authored-by: fukc-gihtub <[email protected]>
2026-10-07 03:02:56 +02:00
Chen Cheng 3bcfd8f303 verify: a window graph that finds no VRAM frees older batch slot layouts' graphs instead of failing (#1185)
Every layout of active slots keeps its own graph pair, and with the expert cache sized to the reserve the next window
graph can find no VRAM left; the engine then exited (#997, `parallel` 4 to 8). cudaGraphInstantiate that runs out of
memory now evicts the least recently used batch layouts, other than the one being captured, until it fits; they are
captured again when needed. The start prints how many were freed. Nothing changes while everything fits.
2026-10-07 03:02:56 +02:00
Gabe Ortiz 5bc479d99a serve: opt-in recovery of tool calls the model writes next to the template's form, "tool_call_recovery": true (#1172)
With the key on, a call of a declared tool is the call when the model writes the tool's name as a parameter tag,
a JSON object inside <tool_call>, or a bare <function=NAME> at the start of a line; two calls in one <tool_call>
are two calls. Anything else stays text. Without the key every reply is exactly as before. On the author's corpus of
1,462 agent turns: 92.9% read as intended before, 98.3% with the key. Re-files #966.
2026-10-07 03:02:56 +02:00
Niko1221 725022a011 serve: /v1/responses accepts a JSON text.format together with tools (#782)
Codex's review request sends tools, an additional_tools item and text.format json_schema at once, and got a 400.
Now a tool call passes through as it does without the format (words written before the call go out as a message
ahead of it), and the schema is checked on the turn that ends in text, as json_schema works without tools. With
tools the system directive says the schema is for the final answer only. The chat route's rule is unchanged.
Tests in serve/test_responses.py reproduce the request shape: a call, a valid text answer, and (with jsonschema
installed) an answer that breaks the schema.
2026-10-07 03:02:56 +02:00
hulkbig 423f5893d7 serve: poll the cancel event every 0.5 s while the engine is quiet, so a client that leaves during a long prompt read stops it (#1182)
The wait for the engine's next line was 10 s, so a disconnect during a quiet prefill reached the engine only at the
next heartbeat. Both waits now poll the cancel event every 0.5 s; the HTTP heartbeat stays at 10 s, and an expired
silence deadline is enforced before another heartbeat. serve/test_quiet_cancel.py drives a fake engine that says
nothing until STOP. This is the Python half of #962; the engine half (stop between prompt chunks) is open.
2026-10-07 03:02:56 +02:00
elanonimo832 41e21f5013 engine: document --spec and --spec-min-p in --help (#1198) 2026-10-07 03:02:56 +02:00
elanonimo832 9bbcf5b8e5 docs: --calibrate loads the model more than once, and starts it when it ends (#1197) 2026-10-07 03:02:56 +02:00
xjc10 1db5d255ad tests: hip_prefill_wmma_gemm_parity skips on a card without matrix cores instead of failing (#1148)
RDNA2 and older and gfx906 have no WMMA; the kernel declines every shape there. The test now exits 77, which CMake
reports as a skip.
2026-10-07 03:02:56 +02:00
Avicennasis e01cbf0d3e strata_pack.py build --force: unmake the old pack before writing, so a build that stops part-way is an unfinished build, not a pack (#1176)
--force now removes the old pack's markers (index.txt, manifest.json) before the build writes a byte. A forced build
that stops part-way leaves experts.bin without a manifest, which the next build or setup makes again, instead of the
old markers over partly new experts.
2026-10-07 03:02:55 +02:00
Avicennasis 20ebd1ebf4 serve: a tool's "parameters" / "input_schema" that is not an object is a 400 naming it, not an AttributeError on its first call (#1175)
A string or list as a tool's schema passed the tools check and raised later, in parse_tool_call and the stream parser,
after the 200 and part of the reply. It is now a 400 naming the tool and the field. A missing or null schema still
passes.
2026-10-07 03:02:55 +02:00
Niko1221 48d2e6cabc spec: DraftPolicy measures a lookup window size again when its cost is stale (#1252)
A window size's cost was updated only when that size was chosen, and the re-probe covered only sizes seen fewer than
three times. A size priced high during one slow stretch (a prompt's cache refill) could stay priced out for the life
of the process. A size not measured for 300 rounds is now tried again when a confident lookup could use it, one
round per stretch. Output is unchanged (the policy only picks which drafts to verify); draft_policy_test gains a case
that fails without the change.
2026-10-07 03:02:55 +02:00
rkcth ac52c2c754 mtp: HIP does not create the draft's unused shared-expert branch stream and two events (#1240)
HIP keeps the shared expert on the draft's own stream; only CUDA branches. The stream and events were created and
never used, and the report's R9700 lost prompt speed after the first request.
2026-10-07 03:02:55 +02:00
Niko1221 3b499f174d prefill: the gfx11 fused expert kernel waits after its weight write, and the VGPR-capped variant is off on clang 22 (#1180)
STRATA_PF_FUSED=1 on gfx1100 gave prompts that answered only "!". Two causes, from the report:
- the stage loop wrote the next weight buffer with no barrier after it; one __syncthreads() now follows the write.
- the waves_per_eu(8) variant of native_w11_kernel computes wrong results with clang 22 (ROCm 7.10): there the cap is 1
  (NW_LB; -DSTRATA_W_LB=N overrides it). Earlier compilers keep 8.
Default output is unchanged (the path is opt-in on gfx11).
2026-10-07 03:02:55 +02:00
Niko1221 3449041bb6 Merge branch 'f-1402-y' into f-1402 2026-10-07 02:56:13 +02:00
Niko1221 73e01fa54d sycl: on the xe driver without --stream-experts, --pcie-frac defaults to 0 (and an explicit value gets a warning)
An Arc Pro B70 (xe) page-faults when a kernel reads ordinary pageable host memory (devcoredump: ccs0, FaultType 0,
faulted address in the CPU's mmap range, then Timedout job and device lost). Without --stream-experts the experts
outside the VRAM cache sit in a pageable arena that the PCIe share hands to the GPU. The default share is 0 there
(the CPU computes those misses) with a one-line warning naming --stream-experts, as setup configures it. An explicit
--pcie-frac is kept and warned about. i915 (Arc Alchemist), CUDA and HIP are unchanged.
strata::intel_gpu_driver() reads the driver from sysfs; verify.cpp's i915 test uses it.
The mirror table (host-mirror pointers for the device plan) is filled only from pinned USM aliases under
--stream-experts, so it cannot hand out pageable pointers.
2026-10-07 02:52:41 +02:00
Niko1221 5af104b515 Merge branch 'f-1402-y' into f-1402 2026-10-07 02:42:06 +02:00
Niko1221 26609953a5 tools/batch_test.py: --long-tokens N reads the last prompt from N tokens of filler beside the earlier slots' windows (a prompt loan under --batch) 2026-10-07 02:33:42 +02:00
Niko1221 0a22c46747 verify: a batch's GDN output is quantized to q8_1 under STRATA_QFUSE too (#1139)
STRATA_QFUSE=1 (the gfx1151 arch default) has gdn_step_norm_multi write the
out-projection's q8_1 image itself and skips native_quantize_q8_1.  The batched
recurrence (batch_rec_) calls gdn_step_norm_multi once per slot with no q8_1
destination, so with --batch the projection read stale bytes and every
concurrent answer was corrupt.  The batched case now quantizes as without
QFUSE; the fused image holds the same bytes (gr_parity), so batched output
equals solo.  Single-request paths are unchanged.
2026-10-07 02:33:42 +02:00
wOvANandwin10ogod 7ef55e1959 verify batch: service the doorbell graph per-window (ar_on), not per-process (#1118)
A stage that was 100% resident at init takes the doorbell graph while
a prompt loan/shrink/swap is in flight (ar_off_), and that graph waits
on host doorbells the skipped service never raises: one GPU spins at
100% while the host sits in cudaStreamSynchronize, until the watchdog
fires with the previous stage's stale label. Key the all-resident
service skip in run_slot_rows, batch_poll and service off ar_on(),
as run() already does.

PR #1102 (win10ogod) is the same fix for run_slot_rows and batch_poll;
this one also covers service().

Co-authored-by: win10ogod <[email protected]>
2026-10-07 02:33:42 +02:00
Benjamin Larsson 35435e3d26 verify: the batched K/V append writes a streamed K8V4 state's host copy (#1188, #1169)
#783's batched append of a verify window's cells passed no host copy for the
hybrid K8V4 format (int8 and q4_0 pass st.host, and the per-token path passes
K8V4's two halves).  With --kv-resident, a block that was not resident when its
cells were appended came back from the host copy without them, so the tokens a
window wrote were lost: free generation degenerated into repetition (#1188,
#1135) and parked conversations lost their decode-written turns (#1169).  The
batched path now passes the same host halves as the per-token path.
2026-10-07 02:33:42 +02:00
Niko1221 0ef3be82b9 Merge branch 'f-1402-io' into f-1402 2026-10-07 02:29:34 +02:00
Niko1221 b56d6e3a78 sycl: fix the newline escape in the fdinfo warning (it did not compile) 2026-10-07 02:29:12 +02:00
Niko1221 76e1cd2eef sycl: the draft steps wait for the GPU, not 1024 spins (MTP acceptance 4% -> 62%)
wait_step and the chain wait polled ext_oneapi_empty() through DPCT_CHECK_ERROR, which is always 0, so the
loop left after 1024 spins and read draft tokens the GPU had not written (-1). Same migration bug as the
cudaStreamQuery fix in verify.cpp, missed in mtp.cpp. IQ3_XXS on the A750: 61 of 99 drafts accepted on the
story prompt (the 5070 gets 106 of 166); copy_'s readiness test in verify.cpp had the same flaw.
STRATA_DBG_DRAFT=1 prints each window's drafts and the target's tokens.
2026-10-07 02:25:48 +02:00
Niko1221 9370ea41f9 sycl: drain the other queues before each verify window on an i915 card; STRATA_RING_TIMEOUT_S
On an Arc A750 (i915) a verify window launched behind other GPU work in flight (the previous window's commit,
a prefill) often never started: no GPU breadcrumb, the 20 s ring timeout, the engine ends. queues_wait_and_throw
before the launch removes it (9 of 9 three-request runs pass, almost all failed before). Default on for i915,
off elsewhere (STRATA_WINDOW_SYNC=1/0/2 overrides). STRATA_RING_TIMEOUT_S raises the 20 s doorbell timeout.
2026-10-07 02:25:48 +02:00
Niko1221 93b129ad61 sycl: size from the DRM fdinfo when the driver reports the whole card free (i915); STRATA_CHUNKED_FILL=0 toggle
Level Zero's free-memory query returns the constant total on an Arc A750 (i915); the expert cache and draft head
were sized from the whole card and the first launch failed with out-of-host-memory. device_free_bytes now lowers
the figure by what this process holds (drm-total-local0 / drm-total-vram0 in /proc/self/fdinfo) and says so once.
2026-10-07 02:25:48 +02:00
Niko1221 94013054ba sycl: verify_slot names the first bytes that differ 2026-10-07 02:25:36 +02:00
Niko1221 614678f8e7 sycl: native_router_top10_multi launches the single-token route<512> per token block
Same bits as upstream's one-warp-per-token route_multi (native_multi_parity compares them), and the
same launch shape as the 256-expert multi router.
2026-10-07 02:25:36 +02:00
Niko1221 0d4e82c6d8 sycl: host memory a kernel polls comes from zeMemAllocHost BIAS_UNCACHED; null-allocation checks
On the Arc A750 a kernel never saw the host's doorbell and result-row stores to ordinary host USM
(sycl/probe/doorbell.cpp: only the uncached allocation completes the handshake), so every ring wait
hit its bound and the window read stale rows: NaN logits. verify's mapped staging and the pool's
y_miss rows now use host_malloc_polled (sycl_queue.hpp; needs libze-dev's header, else sycl::malloc_host;
STRATA_HOST_UNCACHED=0 turns it off). IQ3_XXS on the A750 now answers 12*12 = 144.

Also: the expert cache and the MTP draft head check Level Zero's null return for a too-large
allocation (and an Arc A750 needs UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS=1 for a single allocation over
4 GiB); gather_rows no longer asks for a cooperative launch it does not use; the expert-cache
zeroing in generate.cpp is chunked too. Based on the probe in PR #423 (maxfridbe).
2026-10-07 02:25:36 +02:00
Niko1221 6d1cb3a657 sycl: fill big device arenas with chunked kernels, not one queue.memset
One memset of the 25 GiB expert-cache arena runs on the blitter engine and times out on the
Arc Pro B70 (xe: engine memory CAT error, GT reset). big_fill_zero zeroes in 256 MiB kernel
launches; used for the expert cache, the MTP and verify arenas and the KV reset.
2026-10-07 02:25:36 +02:00
Niko1221 02a64abf15 io prefetch: warn (never force) on an AMD iGPU with the adaptive tier and on the staging variant; docs note with the measured table 2026-10-07 02:25:35 +02:00
Niko1221 23a38a0bb7 sycl: re-migrate the port to 0.1.40.1 (dpct 2025.3, 3-way merge per docs/INTEL.md)
The sycl/ copies were at 0.1.39 and did not build against the 0.1.40.x shared
headers (load_experts_gguf ready argument, fused_gr_read_multi bool, Gemm::bf16/native ldx,
13 symbols missing at link). Re-migrated the 5ffcce43 and 0.1.40.1 trees with migrate.sh,
merged with merge_upstream.py, hand-resolved the conflicts (the port's tuned expert and
MMVQ kernels kept; verify, mtp, generate adopt 0.1.40.1 with the port's queue handling).
Stubs: vmm (no driver VMM on Level Zero), --remote-expert-opt. Dropped upstream's
opt-in AMD S26/S27 grouped-expert kernels and STRATA_QFUSE/HC_Q8 reads on SYCL.
build.sh and migrate.sh source setvars.sh outside set -u (they exited silently).
2026-10-07 02:25:35 +02:00
Niko1221 aa659c9eb9 io prefetch: the experts.bin tier fills the page cache from the I/O threads and the kernels read the mapping (no staging, nothing waits); cheaper residency probe
The first version staged every uncached blob and made the layer wait for all of them before any expert computed, which
lost the overlap the kernels' own page faults had (-19% on a cold 16 GB lane).  Now prefetch() hands the layer's uncached
blobs to the I/O threads and returns; the kernels that reach a blob before it lands wait on its pages, the others
compute.  STRATA_IO_PF_STAGE=1 keeps the staging variant.  Residency is probed on four sampled pages, not the whole blob.
2026-10-07 02:21:11 +02:00
Niko1221 9745567e5f file tier: opt-in Linux I/O path (STRATA_IO_PREFETCH=1) and counters that separate drive reads from page-cache hits
The mapped file tier reads experts through page faults. With STRATA_IO_PREFETCH=1 (Linux, experts.bin or the GGUF
in place) a blob whose pages are not cached is read with one pread of the whole blob into a staging buffer, the
predicted experts of the next layer(s) are read ahead on I/O threads (STRATA_IO_PF_THREADS, STRATA_IO_PREFETCH_DEPTH,
STRATA_IO_PF_AHEAD=0 for demand reads only), and a cached blob is still handed out as the mapping. Same bytes, so the
same output; off by default.

Counters: each request now logs the OS's own view of the file tier (/proc/self/io read_bytes and major faults) beside
the engine's expert reads, and with STRATA_IO_STATS=1 or io prefetch a mincore split of cached and uncached bytes,
read-ahead used / unused / skipped, and the decode's wait in the fetches (--stats prints the same).

blob_stable(): a pointer that is not recycled, for the copies that run asynchronously (adaptive swaps, the GPU cache
fills, the prompt stager).
2026-10-07 02:21:11 +02:00
Niko1221 e2de691c7c bench: Strata vs llama.cpp v0.6.0 head-to-head script and results (3060, P100, 5070 one round) 2026-10-07 01:33:18 +02:00
Niko1221 cbd83b25a0 tests: expert_multi_test checks and times the pool's rows path (gate/up rows, re-quantization, down rows) against the whole-expert kernel 2026-10-07 00:24:19 +02:00
benoit lange a782177ee1 serve: GET /metrics in the Prometheus text format with vLLM's metric names (#793)
Asked with Accept: text/plain or ?format=prometheus; every other request keeps the JSON.
Latency histograms, a Grafana dashboard and scrape configs under docs/monitoring.
Only the Prometheus part of #793; the pipelined-window slot allocator needs a 2-GPU check and is left out.
2026-10-07 00:18:37 +02:00
Francesco Albano 5208b320dd cpu experts: AVX2 Q8_K activation quantizer for the native i-quant layers (#851)
Byte-identical to ggml's quantize_row_q8_K (q8k_quant_parity checks it, including the
avx-only run). STRATA_NO_Q8K_AVX2=1 keeps ggml's.
2026-10-07 00:16:07 +02:00
InB4DevOps 6f21c9d912 prefill: overlap routed expert uploads with the shared expert (#789)
Reads the routing ids first and queues the shared expert afterwards; staging
counts real transfers. STRATA_PREFILL_STREAM_AHEAD=0 restores the old schedule.
2026-10-07 00:15:26 +02:00
Niko1221andAnon 293da8d223 engine --gpu: set HIP_VISIBLE_DEVICES on AMD builds and warn when it overrides the environment (#852 follow-up)
Co-authored-by: Anon <[email protected]>
2026-10-07 00:15:26 +02:00
Anon f88e27b396 engine: --gpu selects the visible GPUs without environment variables (#852) 2026-10-07 00:15:26 +02:00
Francesco Albano 13eee452de pipeline-windows beside the asynchronous adaptive tier: copies queued from the decode loop at a card's gap (#905)
--adapt-async 1 no longer drops to the blocking tier with --pipeline-windows 2: the steps that overwrite
something a window in flight may read wait for those windows. STRATA_PIPELINE_ADAPT_ASYNC=0 keeps the
blocking tier there. Opt-in; default output unchanged.
2026-10-07 00:01:18 +02:00
Niko1221 08cdd8bf68 layer split auto: on a tie the balanced placement wins
When every placement holds the whole expert profile the predicted decode window is the same for all of them and the winner was floating-point noise: --trim-stage-weights on four GPUs chose layers 6/30/5/7 and read a 32K prompt at 1,750 tok/s, against 2,790 for 12/12/12/12 with the same decode speed and tokens (4x Radeon AI PRO R9700, IQ3_S). Among placements whose window time agrees to 1e-6 ms the smallest slowest stage is kept.
2026-10-07 00:01:18 +02:00
Niko1221 feccbfe32f R1/R3: tools/research_run.py, docs/RESEARCH_RUNS.md (measured numbers), DETAILS.md strata_prefix section 2026-10-07 00:01:12 +02:00
Niko1221 13d14a4944 R1: a pinned shared prefix for many suffix queries (opt-in pin=N / "strata_prefix")
The engine's GEN line takes pin=N: the first N prompt tokens are a shared read-only prefix (a long document many
short questions follow). The prompt is read in two parts at N and the checkpoint there is pinned: retention never
evicts it, a parked conversation holding it stays parked, the session file keeps it, and a later request that resumes
from it does not park the branch it leaves, so N queries keep one prefix. One pin at a time. Without pin= nothing
changes.

The server takes "strata_prefix" in a chat, messages or responses request: {"messages": K}, {"message": I,
"chars": N} or {"tokens": N}, resolved to the longest common start of the prompt's ids. A prefix that cannot be
marked is said in the log and ignored; a malformed field is a 400.

Tests: conv_cache_test (pinned eviction), conversation_cache_test (a pinned parked conversation is not evicted),
conversation_file_test (the pinned checkpoint is the one saved), serve.test_server.SharedPrefix.
2026-10-07 00:01:12 +02:00
Niko1221 0d2ecef720 docs: sampled drafting (STRATA_SPEC_COUPLED / STRATA_SPEC_PROB) and what was measured 2026-10-06 23:58:41 +02:00
Niko1221 589ccc4701 spec: STRATA_SPEC_PROB gates --spec-min-p on the drawn draft's probability by default (STRATA_SPEC_PROB_GATE=top for q's top probability) 2026-10-06 23:58:41 +02:00
Niko1221 33356f7289 spec: probabilistic draft acceptance for sampled MTP drafts (opt-in STRATA_SPEC_PROB=1)
Speculative rejection sampling for the MTP drafts of a sampled request. The draft layer samples
its draft from the chain's distribution q (its own Philox stream, counter | 1<<61) and writes
q's (id, probability) list; the verify head computes the target's post-chain p per row from the
split sampler's block lists (penalties, top_k, top_p, min_p, temperature; at most 64 entries)
and accepts draft d with min(1, p(d)/q(d)), else samples the residual max(0, p - q) with its
own uniform. Row t writes the draft on an accept and the residual sample (never d) on a reject,
so the host's accept loop and the commit are unchanged. Rows past the MTP drafts (a lookup
chain's tail) and suffix windows are point masses: accept with p(d), residual p without d - no
q never means q = 0 (the llama.cpp #29924 class). Greedy requests never reach the new kernels;
the nine existing sampler kernels keep identical SASS.

- spec_prob.hpp: the rule, host reference; sample_tokens_spec / spec_draft_sample kernels
- STRATA_SPEC_PROB_GATE=pick, STRATA_SPEC_PROB_DT=x, STRATA_SPEC_MIN_TEMP=x: tunables, off
- spec_prob_test (CPU, 2M trials per case, chi-square 99.9%, controls that must fail) and
  spec_verify_parity (GPU verifier vs host reference, token for token)
2026-10-06 23:58:41 +02:00
Niko1221 0ab663fc20 serve: STRATA_SPEC_DEPTH=1 prints the MTP drafts tested and accepted by depth for each request
The per-request totals (drafts accepted / offered) hide where a window ends. With
STRATA_SPEC_DEPTH=1 the serve loop counts, for the windows drafted by the MTP layer, how many
times draft 0, 1, 2 ... was tested (every earlier draft was kept) and how many of those were
kept, and prints one stderr line per request. Off by default; nothing else changes.
2026-10-06 23:58:41 +02:00
Niko1221 efe7839905 setup: CUDA 13.2's nvcc for an RTX 50 card (sm_120) is passed over for an older 13.x, or warned about (#892, #968)
Built on our RTX 5070 with nvcc 13.2.51 and 13.0.88 from the same source: with 13.2 the IQ3_S MMVQ product
(iq_multi_parity) and the IQ2_S / IQ3_S MMQ prompt path (prefill_fused_iq_test) are wrong, relative error 0.5 to 1.0; the
other formats are bitwise equal. setup now takes an installed 13.0 / 13.1 over 13.2 for sm_120 (STRATA_NVCC still wins) and
otherwise warns, saying what to install. Recommends, never refuses.
2026-10-06 16:35:09 +02:00
Niko1221 b540e994b8 prefill: STRATA_DBG_FORCE_SLOW_LAYER=1 exercises the fused-buffer backstop (#954)
With the fused layout (STRATA_PF_FUSED=1, a chunk of 1024 tokens or more) layer 0 is treated as one the fused path does not
take, so a prompt must stop with the backstop's error instead of overflowing the shrunk buffers. The check that
the guard works at a 16,384-token chunk without needing a pack that really has such a layer.
2026-10-06 16:35:09 +02:00
Niko1221 471303d9ab tools: stop_resend_test.py, the stop / edit / resend loop of #879 and #606 as a repeatable check
Streams an answer, stops reading, resends or edits the prompt after 0-0.4 s (or with another request already running)
and checks every answer for real text, not a repeated token. Run the engine with STRATA_DBG_NAN=1 beside it.
2026-10-06 16:35:09 +02:00
Niko1221 e155cf8fc4 engine: a big prompt chunk whose lent slots do not fit in RAM is warned about (#669, #765)
--prefill auto:32768 lends 6,000 cache slots to the prompt path; on 31 GB of RAM only 3,700 of their experts stay in
RAM and the rest are read from the SSD on every chunk (prompts ~3x slower than --prefill auto). The engine now says
so at start, with the counts. A warning, never a cap: with the RAM for them the big chunk is 21-35% faster.
2026-10-06 16:35:09 +02:00
Niko1221 5628e280fb docs: troubleshooting for #781 #831, #649, #961 and #654 2026-10-06 16:35:08 +02:00
Niko1221 e8b53d5d46 serve: a full card with an explicit --expert-cache says so, and what to do (#781, #831)
The LOW free-VRAM line at READY now also names the explicit --expert-cache (its slot count) and suggests a smaller
one or 'auto', which re-checks the free VRAM after the slots are written. Said, never changed.
2026-10-06 16:35:08 +02:00
Niko1221 dfc8f74dda verify: STRATA_DBG_GDN=1 checks the commit kernel's window count (#937)
gdn_step_commit_kernel reads n_keep from device memory. With STRATA_DBG_GDN=1 a count past the window's rows is
printed and the commit skipped, where a gfx1030 reporter sees a memory-aperture violation. A template parameter
keeps the default kernel unchanged.
2026-10-06 16:35:08 +02:00
Niko1221andKrypto-Whitehat 73d3983046 prefill: a backstop for the fused layout's MoE buffers - an error naming the layer, not a hang (#954, #958)
fused_layout() and fused_ring() already keep the shrunk GU/H/Xq buffers and the per-layer expert path in step. If
they ever drift (a layer that takes MMQ or the FP16 path at a chunk of stream_all_min() tokens or more while the
buffers are the fused ones), the prompt now stops with 'the fused layout's MoE buffers are too small for layer N'
instead of writing past the buffer: an illegal access, or a kernel that never returns and shows up as
'prefill mmq: iota' one prompt later. Same check in the SYCL copy.

Co-authored-by: Krypto-Whitehat <[email protected]>
2026-10-06 16:35:08 +02:00
Niko1221 0e773c8624 serve: the AMD engine's environment: a HIP_PATH that points nowhere is dropped, an unwritable TEMP is replaced (#654)
A deleted ROCm build named by HIP_PATH / HIP_DEVICE_LIB_PATH / LLVM_PATH, and a TEMP the comgr JIT cannot
create files in, both crash amdhip64 before the engine prints a line. Only a missing path and an unwritable
folder are touched; a healthy environment passes through unchanged.
2026-10-06 16:35:08 +02:00
Tanay 8e52aecab5 setup.sh: make the venv with uv on NixOS, or with STRATA_UV=1 (#662)
uv's own managed Python loads pip wheels' libraries where Nix's Python cannot. Changed from the PR as the plan asked: uv is used only on NixOS or when STRATA_UV=1 is set (no silent toolchain change on other systems), and the setup.py hunk is left out. Not run on NixOS here. (PR #662)
2026-10-06 15:57:39 +02:00
Niko1221 50ba742bcb serve: a reply that ends inside its thinking can be continued once with the thinking closed, opt-in (#1053)
"reasoning_close_retry": true in the config: when a reply ends on its end-of-turn token still inside <think>, with no answer and no tool call, the thinking is closed once and generation continues (the engine holds the prefix). Off by default; tested both ways and for a reply that already answered.
2026-10-06 15:57:39 +02:00
Fringe210 61d5efab45 Enable --ple-io ram on Windows via VirtualLock
PleTable's Mmap+lock path now calls strata::platform::lock_resident (working-set minimum + VirtualLock, chunked per GiB) instead of refusing to start. A lock failure still falls back to touching the pages with a warning, same as the POSIX path.

Changed as the plan asked: only a full VirtualLock counts (a partial one is unlocked again and said so), a warning when the table is more than half of the RAM, the usage text indent fixed. Compiled on Windows here (MSVC, CUDA 13.0); not run with --ple-io ram yet.

(PR #813)
2026-10-06 15:57:39 +02:00
Niko1221 20b5005c7f docs: setup remembers the data folder; how to choose another one (#1070)
A reinstall put the model on the old drive again because setup keeps the last data folder in its settings file. The troubleshooting table now says where, and that --data-dir, --models-dir, --gguf-dir or deleting the data_dir entry change it.
2026-10-06 15:57:39 +02:00
Niko1221 9d692d8d02 serve: a --resident-budget-gib above the free RAM is a warning at start (#1080)
An explicit budget bigger than the RAM the PC has free (or than its total less ~6 GiB) pushes experts to swap or the file tier; one report went from dead slow to fast by deleting the number. The server now says so at start and goes on (recommend, never force).
2026-10-06 15:57:39 +02:00
Niko1221 13c1fcb628 setup: Visual Studio 2026 is accepted for the build with CUDA 13.3 or newer (#985)
find_vcvars looked for versions [16.0,18.0) only, because CUDA 13.0-13.2 refuse a newer Visual Studio. The range now goes up to 19.0 when the toolkit found is 13.3 or newer (the reporter built with VS 18 and CUDA 13.3); older toolkits stay on 2019 and 2022.
2026-10-06 15:57:39 +02:00
moebear 0d73237632 serve: a Codex compaction can be rendered with its conversation's tools, opt-in (#924)
Codex's compaction request has tools: [], so its prompt shared almost nothing with the cached conversation and was read again from the start. With "codex_compaction_cache": true the tools of that conversation's last prompt are kept (one entry, keyed by session and thread) and used for the compaction prompt only; the parser and the response keep the request's own tools, and a request that would not fit with them is rendered as sent. Changed from the PR as the plan asked: off by default, the mismatch documented, ported on top of the opt-in #923 and tested over HTTP with a tool call emitted in the compaction. (PR #924)
2026-10-06 15:57:39 +02:00
moebear 2db8ae14af serve: Codex's thread-title turn can be answered without the engine, opt-in (#923)
Codex sends a thread-title request beside each user turn (another session, no tools); running it on the engine replaces the one prefix cache. With "codex_thread_titles": true in the config it is answered at once with the user's line cut to 36 characters (usage 0). Changed from the PR as the plan asked: off by default (the title is not model-written), documented, tested both ways. (PR #923)
2026-10-06 15:57:38 +02:00
Aníbal Muñoz Calero d125bef795 serve: lazy-load vision encoder with model
Rebased onto the 0.1.40.1 Vision (work_dir, #480 absolute paths). Added tests for the lazy encoder (nothing starts, restart starts it, a failed start is killed); the --lazy help no longer says text-only. A live smoke with a vision config and --lazy is still to run.

(PR #673)
2026-10-06 15:57:38 +02:00
Niko1221andupdalla-apshir f3019d00b3 serve: the endpoints keep reporting the context while the engine restarts (#351)
Cut down to reporting only, as the plan asked: max_context is 0 until READY, and /v1/status, /v1/models, /props, /health, /slots and /metrics told a client the model had no context for the whole load. They now use the last known context (reported_ctx). Requests are still checked in prepare() as before.

Co-authored-by: updalla-apshir <[email protected]>
2026-10-06 15:57:38 +02:00
liowald 3379596b38 serve: usage and timings across reasoning continuations keep the original request's input and sum every pass's work (#615)
A reasoning-budget continuation (and the reasoning-loop recovery pass) is another native generation, not another API request: each pass keeps its own DONE, the prompt reuse is the first pass's (the generated reasoning is not this request's cached input), and times, tokens and draft counts (offloaded included) are summed. Rebased onto the 0.1.40.1 Service.run. (PR #615)
2026-10-06 15:57:38 +02:00
Niko1221andGuillaume PUTIER ecdc028c73 serve: an empty api_key in the config is a warning at start, not silence (#569)
Cut down from the PR as the plan asked (recommend, never force): an empty --api-key or STRATA_API_KEY is already refused (#213) and a blank config key is refused with #956; an api_key that is present but empty now prints that the server has no key and starts. The setup-side refusal is left out (the final line already says no API key is set).

Co-authored-by: Guillaume PUTIER <[email protected]>
2026-10-06 15:57:38 +02:00
Akido 8d8e165148 tools/vision: portable by default; setup.py and the Dockerfile ask for native (#411 #412 #419)
The 0.1.32 image encoder was configured without -DSTRATA_PORTABLE, so the old OFF default built it native
(MSVC + FindSIMD: /arch:AVX512 on the build machine) and it stopped with 0xC000001D on AVX2-only CPUs.  A build
without flags is now the AVX2 baseline; the builds for the PC they run on (setup.py's two encoder builds, the
Dockerfile) pass -DSTRATA_PORTABLE=OFF and stay native.

- portable also forces GGML_AVX512*, GGML_AVX_VNNI and GGML_AMX_* OFF, so a -D cannot put AVX-512 back in
  (with the old file, -DSTRATA_PORTABLE=ON -DGGML_AVX512=ON gave 10522 zmm instructions; now 0)
- native restores a fresh native configure's GGML_OPENMP / GGML_AVX* when a folder was portable before
- the configure log says which of the two it is
- test: build_engine's encoder build (gpu and cpu) passes -DSTRATA_PORTABLE=OFF

Rebased by hand onto the 0.1.40 setup.py (the Windows MSVC environment of #881 kept). The build script of the ready-made engine zips is not in this repository: it must pass -DSTRATA_PORTABLE=ON for the encoder (the default was OFF until now).

(PR #427)
2026-10-06 15:57:38 +02:00
Joe-Huber 954cb2a2b6 chore(github): issue forms (bug, feature, question) and a pull request template (#401)
Short forms with the required fields, a link to the troubleshooting table, and the pull request template. Taken without CODEOWNERS and without the template's checklist (the changes the plan asked for); blank issues stay allowed.
2026-10-06 15:57:38 +02:00
Zac Chen 64de53b634 docs: the RX 7900 XTX IQ3_S rows in the speed tables (#319)
Prompt and output rates for the AMD RX 7900 XTX (gfx1100) beside the NVIDIA rows, with the measurement note. Only the DETAILS tables are taken: the PR's branch also deleted about 190 lines of current text, and the README tables are two fixed machines (RTX 5070 and RX 9070 XT).
2026-10-06 15:57:38 +02:00
1872183316 5ecc7fea17 setup: --inspect, a GGUF's real quantization and whether Strata runs it, from its headers only (#911)
tools/quantscope.py (from 1872183316/quantscope, MIT OR Apache-2.0) reads only the headers (a file, a folder, a URL, ms: or hf: repositories); tools/strata_inspect.py compares them with data/gguf_fingerprints.json, the files setup installs. Changed from the PR: the size menu is untouched (its bits per weight line would change the golden output; left out), and the untested verdict says that setup still refuses the GGUF names it lists as unsupported, so the two do not contradict each other. (PR #911)
2026-10-06 15:57:38 +02:00
Niko1221and1872183316 4f36b95557 setup: ModelScope as an explicit download source, --source modelscope (#908)
Setup never switches hosts by itself: auto is Hugging Face, and a download that fails says --source modelscope exists (no probe at start). With it, each file is checked against the SHA-256 ModelScope publishes (a wrong one is deleted; a file with no published hash is warned about), the MTP tensors against the pinned checkpoint's own hashes, and setup says that ModelScope serves current files, not our pinned revision. Changed from the PR: the automatic switch and its 10 s probe are gone.

Co-authored-by: 1872183316 <[email protected]>
2026-10-06 15:57:38 +02:00
1872183316 7e2864a3fc calibrate: measure the adaptive expert tier (--adapt-every / --adapt-swaps / --adapt-decay), with 512-token answers (#907)
A new step 5 tries two tier settings (every window, 80 or 160 swaps, decay 0.97) against the engine's own, after the worker count, one restart each; a setting is kept only when it beats the default by 3%. The tier follows a text over some windows, so this step measures 512-token answers. The author measured +7.9% on a DDR3 Xeon with an RTX 4060 Ti. Applied by a 3-way merge onto the 0.1.40 calibrate.py; the check that it keeps the engine default on a fast-RAM PC is still to run on a GPU.
2026-10-06 15:57:38 +02:00
Niko1221and1872183316 98222df476 setup: step 1 shows the GPU's PCIe link and says when it reads fewer lanes than the card has (#912)
The generation card and board both run, the lanes now, and the copy rate that implies for a slow link. Changed from the PR: the fewer-lanes note is worded as a hint (a card or laptop can narrow the link while idle), not a verdict.

Co-authored-by: 1872183316 <[email protected]>
2026-10-06 15:57:38 +02:00
Niko1221 af76b97bff setup: images are allowed with UD-Q4_K_XL, with an untested warning (#967)
The model is the same base model and the same image encoder as UD-IQ4_XS; a user ran it by hand (#967, #971). Setup asks about images for it and warns that we have not run this file with images (no smoke test here: the 111 GB file is not on this PC). MCP's plan and size table follow.
2026-10-06 15:57:38 +02:00
Niko1221 a681cac6a8 serve: /v1/responses accepts Codex's additional_tools input items (#782)
Codex v0.160 sends an input item of type additional_tools, which was refused with a 400. The item is no message; a tools array in it joins the request's own tools, and one without a usable array adds nothing. The item's exact shape is not documented here: only the tools-array form is read.
2026-10-06 15:57:38 +02:00
Guillaume PUTIER 832b2e4f34 gr_parity, ple_parity: wait for the default-stream setup before launching on the test stream
Both tests upload and memset on the legacy default stream, then launch on
a cudaStreamNonBlocking stream, which does not wait for it. cudaMemset is
asynchronous to the host and a pageable cudaMemcpy may return before the
DMA lands, so the kernels can read buffers still being written. A device
sync before each launch closes it, the same fix 0.1.36 made in the fused
tests. Not reproduced on an RTX 5090: 0 failures in 210 runs of gr_parity
before and after.

Only the first commit of the PR (the stream syncs); the second one (a rewrite of ple_parity without the fixtures) is not taken. The same syncs for the HIP tests were not added (no AMD test machine). (PR #568)
2026-10-06 15:57:38 +02:00
Klaus Friedel b8aca3e6fb serve: no automatic --layer-split when the args use --peer-device
With several GPUs in the config's "gpu" the server appended --layer-split auto, which the engine refuses beside
--peer-device (#531), so the peer tier could not be started through the server at all:

    strata generate: --peer-device cannot be combined with --layer-split (use one or the other)

The split (and --split-skip-if-fits) is now added only when --peer-device is not among the args; the start line
names the peer tier instead of a layer split. Tests in serve/test_server.py (PeerDevice), docs/SECOND_GPU.md says
how to configure it.

Ported by hand onto the 0.1.40 engine_args (layer_split_value); the start banner now says the peer tier is in use when the split is not.

(PR #665)
2026-10-06 15:57:38 +02:00
1314521gjy f2158a5cf5 docs: the expert cache size is a budget, and an over-sized one pages on Windows
--expert-cache N is N times the largest blob, compared only with the free VRAM
read before the slots are written; the check that runs once they are written is
auto-only. Under WDDM the free figure can be about a gigabyte high and an
over-committed allocation goes to system memory instead of failing, so only the
speed shows it. Measured at 1,048,576 on a 32 GB 4080 SUPER: 11,631 slots with
0 MiB free ran 13.7 tok/s, 11,178 slots (auto) with 217 MiB free ran 102-122,
and raising --vram-reserve-mib moves the free figure because it is deducted
before the cache is sized.

Also records that the GPU process Shared/Dedicated counters did not separate the
slow run from the fast ones on that machine, and adds a Troubleshooting row.

Read against generate.cpp: the second size check after the slots are written is auto-only (the loop breaks on !auto_cache), and the LOW line is printed at 256 MiB.

(PR #799)
2026-10-06 15:57:38 +02:00
christopherrobertbrooks-tech eefed6aa5e docs: running Strata behind llama-swap
The entry (serve/server.py with llama-swap's port, checkEndpoint /health, healthCheckTimeout), the DNS-rebinding
'Host ... is not allowed' trap and its three fixes, /upstream/<model>/ for Strata's own pages, and a note on the

Read against the 0.1.40 server: allowed_hosts and STRATA_ALLOWED_HOSTS, /health, the effort handling and the 403 text match.

(PR #829)
2026-10-06 15:57:38 +02:00
shefowl e7ec6b1abb strata-vision: flash attention off on the CPU only when ggml has AVX-512
ggml's tiled CPU flash-attention kernel needs the head size (72 in this
encoder) to be a multiple of the vector width: 8 floats with AVX2, 16 with
AVX-512. With AVX-512, which a build from source gets on a CPU that has it,
ggml falls back to a kernel that is several times slower and accumulates in
FP16. The release builds are AVX2, where flash attention is the faster one,
so auto stays the default there.

1024x1024 picture, Ryzen 7 7700X, 8 threads, F16 mmproj: AVX2 8.3-9.9 s
with it, 14.0-15.2 s without; AVX-512 43.5 s with it (26% off the FP32
output), 13.3 s without. --gpu keeps auto, and an explicit --flash-attn
still wins.

Rebased onto the 0.1.40 strata_vision.cpp (the --min-tokens usage line). Only a source build on an AVX-512 CPU changes; the release builds (AVX2) do not. The Zen4 timing is not run here.

(PR #660)
2026-10-06 15:57:38 +02:00
mad9home 3e14442811 docker: GGUF_DIR, RESIDENT_BUDGET_GIB and KV_STREAMING env vars; document the stop timeout
- GGUF_DIR passes setup's --gguf-dir, so a container can use model files the
  host already has instead of downloading them again (#252/#444 cover the
  native setup only)
- RESIDENT_BUDGET_GIB and KV_STREAMING pass --resident-budget-gib and
  --kv-streaming; unset keeps setup's own defaults
- INSTALL.md: a mounted GGUF folder must be writable on the first start
  (the Unsloth SHA-256 result is kept as <shard>.done next to each shard)
- INSTALL.md: releasing tens of GB of page-locked RAM on SIGTERM takes longer
  than Docker's 10 s default; without --stop-timeout the container is killed
  (exit 137)

The INSTALL.md paragraph was added by hand (the PR's numbered list no longer exists); KV_STREAMING maps to setup's current --kv-streaming auto|on|off.

(PR #491)
2026-10-06 15:57:38 +02:00
Yannick Loth ebd9c81c98 native head: report the cudaHostAlloc error, not just "cannot pin N MiB"
The bare message cost a full debugging session: on Linux the failure was a
missing libcuda.so.1 (a host driver library the nix loader cannot see) in a
launch environment, not memory pressure.  cudaGetErrorString names the real
cause, e.g. "CUDA driver version is insufficient for CUDA runtime version".

Ported by hand onto the 0.1.40 native_head.cpp (the PR's hunk context had moved).

(PR #476)
2026-10-06 15:57:38 +02:00
Niko1221 3c6456a2e6 setup: every finish mark is written with a warning on a read-only folder (#570)
The PR guarded the mark of a whole shard; mark() itself now warns and goes on, so the SHA-256 result of a verified Unsloth shard and the shared-shard link on a --gguf-dir share do not stop setup either. The step is repeated on the next run.
2026-10-06 15:57:38 +02:00
Guillaume PUTIER 762e58b5b2 setup: a read-only --gguf-dir no longer stops setup when marking checked shards
A whole shard without its .done mark gets one written next to it. On a
read-only --gguf-dir (a network share) that write raised OSError and setup
stopped. It now warns that the mark cannot be written and goes on.

(PR #570)
2026-10-06 15:57:38 +02:00
jollyroger1480 ccfdc2ebef native dense: run the architecture guard only on the first shard of a split
A splitter may copy general.architecture into EVERY shard of a split while the
qwen4exp.* hyperparameters live only in shard 1 (seen with the Huihui
abliterated re-split of the UD-Q4_K_XL GGUF: general.architecture in shards
2-4, no qwen4exp.* there). NativeDense::load then ran check_architecture on
shard 2 and failed with 'missing qwen4exp.block_count'.

Run the full guard only when the shard is the first of the split (no split.no,
or split.no == 0); later shards take the existing split-metadata consistency
check whether or not they carry the architecture key. Verified end to end:
with this change the engine loads the re-split GGUF (previously: refused),
pack built with iq_pack.py --compat-bf16, coherent greedy output.

(PR #648)
2026-10-06 15:57:38 +02:00
Léon Avic Simmons 536db01af9 expert profile: refuse a header version other than 1, naming both numbers and the file
read_expert_profile read the header's version field and then ignored it
((void) version), so a profile of a later format would have been read as a
version 1 layout: the fields after the version are only known to be
n_layers, n_expert, slots, n_ranked in a version 1 file.  Refuse any other
version right after the magic, before the geometry check, with the version
found, the version this engine reads, and the file path in the message.

tests/core/expert_profile_version_test.cpp (registered like
expert_profile_save_test, CPU only): a version 1 file loads; the same bytes
with the version field at 2 or 0 are refused naming the version and the
file; a version 2 file of another geometry is refused by its version, not
its geometry; back at 1 it loads again.

(PR #632)
2026-10-06 15:57:38 +02:00
eadra 638ab2dd5e tests: prefill_mmq_kquant_test waits for its uploads before the kernels
The inputs and the NaN sentinel go up with cudaMemcpy / cudaMemset on the legacy stream, and the kernels run on a
cudaStreamNonBlocking stream, which does not wait for it: a pageable copy or the memset can land after the kernels
started. On an RTX 4090 (sm_89, 128 SMs) that failed the screen ("unwritten or non-finite output") in 3 of 3 runs,
in a different product each time; with the sync before the kernels, 5 of 5 runs passed. The same fix as the fused
tests' (prefill_fused_moe_test.cpp, prefill_fused_iq_test.cpp).

(PR #532)
2026-10-06 15:57:38 +02:00
CurtisThe b9fdc1cdce serve: a batch request continued in another leg reports its own prompt's reuse, and input_tokens is never 0 (#1013)
A request stopped on the solo path and continued in a slot is sent again as its prompt plus what it generated, and the engine holds all of that: the usage split counted the continuation. The first read's reuse is kept, and Anthropic's input_tokens is at least 1 (Claude Code took 0 as not given and counted the prompt twice: Prompt is too long at about half the context). Rebased onto the 0.1.40.1 server and the stop_sequence field.
2026-10-06 15:57:38 +02:00
Niko1221 adc87ab7c9 mcp: the install plan accepts vision=cpu with the AMD backend on Linux (#990)
strata_install refused every vision value but no with backend=hip, although setup's own --vision cpu reads images on the CPU beside the HIP engine. It now follows setup.hip_vision: cpu on Linux is planned, a GPU encoder (yes, gpu) and cpu on Windows get a message that says what works. The Windows AMD engine in MCP's hardware detection is left as it was (it still reports AMD cards on Linux only).
2026-10-06 15:57:38 +02:00
BGonnermann 4bc3b4ef45 serve: an API key with spaces or characters outside ASCII matches; a blank key is refused at start (#956)
The configured key loses the spaces and line ends a header cannot carry; a key outside ASCII matches as UTF-8 and as Latin-1 (constant-time). Server side only: the web app part of the PR (the UTF-8 header and the key-rejected note) is left out.
2026-10-06 15:57:38 +02:00
elanonimo832 2de5fa0a84 serve: the thinking budget can be shared with other apps through POST /settings (#1035)
reasoning_budget_tokens joins the shared Chat settings (0 means no budget, a whole number of tokens otherwise) and is read after the shared and config defaults are merged, as max_tokens is. Rebased onto the 0.1.40.1 Service.run (force argument).
2026-10-06 15:57:38 +02:00
elanonimo832 e2d9a4bd7f serve: a POST /settings body without "defaults" is a 400 and no longer clears every shared setting (#1031)
req.get("defaults") is None both for an absent key and for an explicit null, and null means clients use their own settings again. A body the endpoint does not recognise cleared them all. {"defaults": null} still clears.
2026-10-06 15:57:38 +02:00
Niko1221andZerschranzer 0b576686f6 serve: /slots reports n_prompt_tokens, and one entry per slot with --batch (#562)
A context meter that divides n_prompt_tokens by n_ctx read 0 % against Strata. The key is the running request's prompt size, kept after it ends. Added to the PR: with --batch /slots lists every slot (the view /metrics has) instead of one entry.

Co-authored-by: Zerschranzer <[email protected]>
2026-10-06 15:57:37 +02:00
midagedev f69ce694e1 serve: /props total_slots is the engine's batch slots (#1004)
With "parallel": N the engine runs N requests at once, and /v1/status already says so. /props still said total_slots 1, so llama.cpp clients that read it refused or serialized concurrent runs. It now reports the same count, at least 1.
2026-10-06 15:57:37 +02:00
Niko1221andIvan Tuzhilkin 111d62ae96 serve: /props reports chat_template_caps so Zed offers tools; a failing template probe never stops the start (#986, #987)
The caps (supports_tools, supports_tool_calls, supports_parallel_tool_calls, supports_system_role, supports_preserve_reasoning) are rendered once at template load. Changed from the PR: every probe error is swallowed (logged at debug) and turns that feature off, so a custom template that fails in an odd way cannot crash the start.

Co-authored-by: Ivan Tuzhilkin <[email protected]>
2026-10-06 15:57:37 +02:00
Niko1221andOscar Rojas be8c29185c serve: decode chunked request bodies on every route that reads one, with a size cap (#893, #894)
A POST behind a relay that sends Transfer-Encoding: chunked had no Content-Length and read as empty (400 No messages provided). _body(), the control-body read and the unread-body drain now decode it (it wins over a Content-Length, as RFC 9112 says). A chunked body is capped at 256 MiB (413; the announced size is checked before a chunk is read) and a malformed one is a 400. Content-Length requests are unchanged. Left out of the PR: nothing; added: the cap, the other body readers, tests.

Co-authored-by: Oscar Rojas <[email protected]>
2026-10-06 15:57:37 +02:00
Niko1221 47aafa9eb5 serve: a request with more than 64 unique images keeps its own encoded files (#1072)
The vision cache unlinked the oldest .sve on every insert past 64, including a file the same request had not read yet (FileNotFoundError). Vision.encode_all() protects the request's earlier files; the cache shrinks back on the next insert. A cache hit now counts as a recent use.
2026-10-06 15:57:37 +02:00
Niko1221 ef6c18a14a serve: a request the engine refuses with ERR answers at once, not after a 300 s wait for a DONE (#1059)
The solo and admission paths read the engine's ERR line, then sent STOP and drained for a DONE that never comes (it holds the control lines meanwhile). The ERR now ends the exchange.
2026-10-06 15:57:37 +02:00
Niko1221 43badeda1f bench: index the three AMD / Intel community folders in bench/results/COMMUNITY.md, list the 6900 XT trim
Rows for #927 (2x RX 6900 XT), #917 (Radeon 8060S, gfx1151) and #955 (Arc Pro B65) in the new COMMUNITY.md index, with notes on what each
build was; the bullets I had added to docs/COMMUNITY_BENCHMARKS.md are removed so the index lives in one place. TRIMMED.md lists the
seven telemetry files left out of the 6900 XT folder.
2026-10-06 14:59:04 +02:00
Niko1221 58b6f5d045 docs: gfx1031 reports (#915, #1027, #1078) and the Windows zip note in AMD_HIP.md
The gfx1031 entry now lists the RX 6700 XT run (#1027) and the RX 6800M run on Windows (#915, #1078), and says the Windows zip
needs gfx1031 in the STRATA_HIP_ARCHS build list, which the default list has from 0.1.40.2 on.
2026-10-06 14:59:04 +02:00
Niko1221 56d77a983f docs: Pascal has community numbers now (P40, P100, V100 + P100); a GCC 12.3 build note (#1028, #1069, #1079, #1074)
NVIDIA_V100.md said sm_60 was not measured. It now points to the two-P40 layer split (#1028), the P100 alone and with a helper card
(#1069) and the V100 + P100 pair on 0.1.40 (#1079), each one reporter's run. OLDER_GPUS.md notes the extra host flag a GCC 12.3 +
nvcc build needed in #1074 (one report).
2026-10-06 14:59:04 +02:00
Niko1221andTim Nevits 98d1c8d4f3 bench: community report, Intel Arc Pro B65 32 GB (SYCL), IQ2_XS (PR #955)
Three fresh engine launches, five fixed tasks at 512 and 7,000 prompt tokens, 640 output tokens: decode medians 40.4-41.3
and 39.8-40.9 tok/s, reference hashes recorded. The six local patches the report built with are kept as data; patches 3, 5 and 6
are ported as source changes in this release, the others were already in 0.1.40. Index line added to COMMUNITY_BENCHMARKS.md.

Co-authored-by: Tim Nevits <[email protected]>
2026-10-06 14:59:04 +02:00
mini 16fd58b8c4 bench: community report, Radeon 8060S (gfx1151) on a Ryzen AI Max+ 395 (PR #917)
Interactive Q2_0 and UD-IQ4_XS chats against llama.cpp Vulkan, then controlled A-E runs of the gfx1151 hipBLASLt tuning table
(250.9 to 528.9 prompt tok/s), dense MMQ (+0.2%) and the shared-expert decode stream (+2.5%, 8 requests). The build is
0.1.39 plus then-unmerged PRs (#895, #820, a tuning table adapted from #755), not the 0.1.40 release; the index line in
COMMUNITY_BENCHMARKS.md says so and notes that 0.1.40 turned the shared-expert stream off on HIP (#826).
2026-10-06 14:59:04 +02:00
Niko1221andxjc10 cc52598ab5 bench: community report, 2x RX 6900 XT (gfx1030), seven configurations (#927)
One card, a layer split and the expert-helper mode, Strata 0.1.39 stock and with then-unmerged #835, #849 and #854, original
Flash-Next IQ3_S at 131,072 context; three runs at 4K, 32K and 128K prompt tokens and six recall checks per configuration.
Index line added to COMMUNITY_BENCHMARKS.md with a note that #835 is opt-in in 0.1.40, so the second set is not the
release default. The one-second telemetry.jsonl files (5 MB) are left out, and the README says so.

Co-authored-by: xjc10 <[email protected]>
2026-10-06 14:59:04 +02:00
Niko1221 958f61f548 hip windows: build_windows.bat's default arch list includes gfx1031 (RX 6700 XT / 6800M)
The ready-made Windows HIP zip had no gfx1031 code, so an RX 6700 XT or 6800M user had to build from source with
STRATA_HIP_ARCHS=gfx1031 (#915, PR #1078; #1027 ran a 6700 XT as gfx1030). setup already knows gfx1031 (AMD_ARCHS, the
wheels index, the PCI id) and the arch gate builds it. This only adds it to the default list the zip is built from.

Prepared, not decided: building the release zip with this list makes it larger (one more arch of rocBLAS kernels and
engine code), and the owner decides whether the 0.1.40.2 zip carries it.
2026-10-06 14:59:04 +02:00
Niko1221andmini f5ab356abd hip gfx1151: keep the sign of Q2_0's negative zero; warn when the auto cache outgrows an APU's host memory (PR #895)
Two pieces of PR #895 that 0.1.40 did not have; the rest of it is in 0.1.40 (gfx1151 in the arch gate, sudot4, setup
and the Strix Halo defaults) or is not taken (the ROCm pin bump for every AMD arch).

- dq_q2_0: the compiler on gfx1151 (ROCm 7.13) folds the half path's negative scale times +0 to +0, as HIP 5.7 did on
  gfx1012. The sign-preserving path now covers gfx1151 too. Changes bits on gfx1151 only (the same value the FP32 and CPU
  paths give); nothing else is touched.
- Linux HIP, --expert-cache auto: an APU's "VRAM" is system RAM, and the device-free figure counts the whole GPU-addressable
  pool, so a cache sized from it can ask the OOM killer for nearly all RAM. The PR capped the cache at the host's free
  memory less 4 GiB; here it only prints a warning with both numbers and leaves the cache as asked
  (recommend, never force). --expert-cache N sets it by hand.

Compile check only on HIP; the run needs a gfx1151 box.

Co-authored-by: mini <[email protected]>
2026-10-06 14:59:04 +02:00
Niko1221andStormace 10f44a98d5 tests/hip: prefill_mmq_parity initializes its output sentinel on the product's stream
A plain hipMemset runs on the null stream, which a non-blocking stream does not wait for; on a Radeon 8060S (gfx1151,
Windows) the memset landed after the MMQ kernel and left only the sentinel. hipMemsetAsync on the same stream orders it.
Taken from PR #919's Windows gfx1151 port; its other pieces are not taken here (the Windows carve-out accounting needs
a Windows gfx1151 run).

Co-authored-by: Stormace <[email protected]>
2026-10-06 14:59:04 +02:00
ada22464d3 docs: AMD verify timeouts under host-memory reclaim, one experimental section (PRs #750 and #920)
Merges two community reports of the same mechanism (GPU queues suspended while the kernel reclaims KFD userptr pages)
into one short section in AMD_HIP.md and one line in TROUBLESHOOTING.md: HSA_USERPTR_FOR_PAGED_MEM=0 (two R9700, ROCm
7.2) and GPU_PINNED_MIN_XFER_SIZE=1048576 with --mmap-experts (RX 6800), each labelled as one machine and not
confirmed, with the MemoryMax versus MemoryHigh note. Condensed from the two PRs.

Co-authored-by: zengyishou <[email protected]>
Co-authored-by: YamineRL <[email protected]>
2026-10-06 14:59:04 +02:00
Niko1221 4372e3a828 setup: accept gfx1102 (RX 7600 / 7600 XT) as an unvalidated AMD card (#938, #942)
setup listed the RX 7600 XT as "not supported" although the arch gate and the Windows zip already build it, and one
RX 7600 XT ran the engine (Swift IQ3_XXS, 146 requests up to 51K context, about 43 tok/s decode, 250-300 tok/s prompt,
unvalidated; docs/AMD_HIP.md has the report. The two setup tests that expected "not supported" now expect it listed.
2026-10-06 14:59:03 +02:00
Niko1221andTim Nevits 55c3178730 sycl: three shutdown and safety fixes from the Arc Pro B65 report (PR #955, patches 3, 5 and 6)
- conversation_snapshot_test: the pinned host buffers are freed with sycl::free (they come from USM, not malloc).
- STRATA_VERIFY_NO_HOST with experts that are neither in VRAM nor mirrored now refuses to start, instead of a warning
  followed by a generation with no safe host fallback.
- On QUIT the direct-access USM host mirror is released (gguf_src.close()) before the process leaves; a failure to
  release is reported and exits 1.

From the Arc Pro B65 bench report (timnevits), where they are shipped as patch files; the other patches there were
already in 0.1.40. SYCL compile check only.

Co-authored-by: Tim Nevits <[email protected]>
2026-10-06 14:58:47 +02:00
Niko1221andTxgom2 811eeaa32b sycl: the native router's synchronization is a subgroup barrier (hang on Arc A770)
Only subgroup row zero of the 32x8 block reaches the router's synchronization in route(); the other seven rows return
before it. A work-group barrier there is divergent, and on an Arc A770 the kernel hung. The surviving row now meets a
subgroup barrier. Also recorded in sycl/tools/fixups.py so a regenerated tree keeps it.

Salvaged from PR #797 (the rest of that draft is not taken: its build fixes were already in 0.1.40). An A770 on i915
reported decode only with this change (23.0 tok/s with MTP, 8.65 without). Needs an Arc owner to test on the current
tree; SYCL compile check only.

Co-authored-by: Txgom2 <[email protected]>
2026-10-06 14:58:47 +02:00
Niko1221andNathan Bradley 3e05230b1d hip: gfx1151 hipBLASLt tuning table (hipBLASLt 1.5.0)
tune_hipblaslt on the gfx1201-100500 table's 32 shapes, on a Ryzen AI Max+ 395 (Radeon 8060S, gfx1151) with ROCm
10.2.0a20260930 (whl-next, hipBLASLt 1.5.0 efafc1c1). 3.9-15x faster than plain hipBLAS per shape; prefill with
STRATA_HIPBLASLT_VERBOSE=1: launches=3138 fallbacks=0.

Only the table is taken from PR #944. Its gfx11 WMMA attention and scorer duplicate the kernels already in 0.1.40, and
its "gfx11" prefix gate would enable gfx1103 and gfx1152, which 0.1.40 leaves out on purpose. To be re-measured on the
gfx1151 box.

Co-authored-by: Nathan Bradley <[email protected]>
2026-10-06 14:58:47 +02:00
joeyjoe02 557c2b67c1 sycl: read doorbell flags with an uncached L1+L3 hint
On an Arc Pro B60 a system-scope atomic load of host USM is still served
from the GPU's cache, so a device spin never sees the host's store and
runs to kSpinMax every time. Read the flag through an annotated_ptr with
an uncached L1+L3 read_hint after an acquire fence instead. Bounded
ping-pong: 263 ms -> 2.4 us per round trip; per-layer waits on a B60
with experts in the host mirror: 8.7-10.5 -> 19.3 tok/s decode.
-DSTRATA_DOORBELL_ATOMIC_LOAD keeps the old load.
2026-10-06 14:58:47 +02:00
Niko1221andkonijiwa110 65ce329917 qsa_select: FP32 tiled block scores below sm_80 (opt-in: STRATA_SELECT_SIMT=1)
Based on konijiwa110's PR #742. Below sm_80 there is no TF32 tensor-core scorer, so the prompt path scores the QSA
blocks on the warp kernel: 5 shuffles per 4 products, ~1 TFLOPS on an RTX 2080 Ti, 24% of a 128K prompt. With
STRATA_SELECT_SIMT=1, qsa_block_scores_tc runs an FP32 tiled GEMM there (rows = query x indexer head, K = 128 through
shared memory, 2 queries x 4 heads x 4 blocks per thread, relu per head summed in registers), ~6.9 TFLOPS by the
author's measurement (128K prompt 1,016 to 1,242 tok/s, 248K 715 to 952 on one 2080 Ti).

The tiled kernel sums in another order than the warp kernel, so its scores are not bitwise equal. The PR made it the
default below sm_80; here it is opt-in, so default output is unchanged until an interleaved A/B of at least 10 pairs
supports a default. The tail block keeps the warp kernel's arithmetic. STRATA_SELECT_OLD=1 still forces the warp kernel.

qsa_select_bench is now also a CUDA test: at 32K, and at 131K with STRATA_QSA_WARP=1 and STRATA_SELECT_SIMT=1 so any
card runs the FP32 tiled scorer (FP64 gate, selections compared with the warp kernel's).

Co-authored-by: konijiwa110 <[email protected]>
2026-10-06 14:58:47 +02:00
xjc10 6126f8dbeb HIP gfx103x: PR #540's attention kernel with 8 cells per step and DPP lane exchanges (bit-exact)
On HIP the default attention kernel's score loop is bound by the LDS pipe on gfx1030: its 12 x 5 __shfl_xor_sync per
cell compile to ds_bpermute_b32 and share that pipe with the shared-memory q reads.  attn_chunk_kernel_pre75 (PR #540,
bit for bit the default kernel) is now built for HIP wave32 (not the gfx906 backend) and run on gfx103x by default:
- NC_ template parameter, cells per step: CUDA 2 as before, HIP 8 (on gfx1030 every step up in NC was faster although
  occupancy fell from NC 1: the q reads bound it, not occupancy)
- DPP_ template parameter: the butterfly's lane exchanges (reduce12, the softmax max / sum) through DPP row_xmask within
  a row and v_permlanex16 across rows instead of ds_bpermute; same values, same pairing order, unchanged bit for bit
- pre75_attn() on HIP: gfx103x per device; STRATA_ATTN_PRE75=0 off, =1 also another wave32 card
CUDA is unchanged: the non-experimental build does not build the kernel, and the template defaults equal the old
constants with shfl_x<false> = __shfl_xor_sync.

RX 6900 XT, IQ3_S, prompt tok/s at 9.4K / 34.7K / 105.8K: 441 / 467 / 460 -> 459 / 494 / 488; with #835's FP16 GEMMs
746 / 917 / 928 -> 801 / 1,022 / 1,041.  Teacher-forced log-probabilities against the default kernel: KL 0, argmax 100%.

bench, docs: the pre75 attention kernel on RX 6900 XT - measurements, exactness, decode A/B; OLDER_GPUS.md notes

bench/results/2026-10-04-rdna2-pre75-attention: the rig, the compiler output (registers, occupancy, ds_bpermute counts)
that motivated NC 8 and DPP, prompt speed on this branch (+4.2 / +5.7 / +6.1% at 9.4K / 34.7K / 105.8K tokens), the
step-by-step NC / DPP series on the FP16 prompt path, the teacher-forced exactness check (KL 0 over 1,578 positions),
a decode A/B (0.3-0.6 ms shorter windows, two rounds), the limits, and the configuration used.

docs/OLDER_GPUS.md: the RX 6800 / 6900 row says this kernel is the default on gfx1030 and names the switch; the A/B
switches line says what STRATA_ATTN_PRE75 does there.
2026-10-06 14:58:47 +02:00
Eli Elgaev 0ff5b4145b qsa_prompt_attn: Volta m8n8k4 kernel - hi and lo halves in separate accumulator chains
Follow-up to #600 (fks's mma.m8n8k4 prompt attention for sm_70).

The kernel splits q (for q.k) and p (for p.v) into FP16 hi + lo halves
and runs both products into the same accumulator. Volta's tensor cores
don't keep full FP32 precision while they accumulate, so once the hi
products are in the accumulator most of the much smaller lo
contribution gets truncated away, and the split buys less than it
should. Giving the lo products their own chain and adding the two in
FP32 at the end fixes that.

qsa_prompt_attn_parity, one V100-SXM2-16GB, 2,048 queries, error vs the
FP64 reference (the FP32 kernel in brackets):

  ctx 8,192    5.88e-6 -> 3.02e-6   (2.14e-6)   11.94 -> 12.20 ms
  ctx 32,768   5.08e-6 -> 2.33e-6   (2.37e-6)   12.20 -> 12.38 ms
  ctx 131,072  5.35e-6 -> 2.75e-6   (1.81e-6)   12.34 -> 12.51 ms

Registers unchanged (166 / 226 / 163 for modes 0 / 1 / 3), no spills.
docs/NVIDIA_V100.md updated for the new error level.
2026-10-06 14:58:47 +02:00
willowmerestudios 3641d2f8dd tools: sm60_dp4a_check (STRATA_DP4A vs byte reference, bit-exact + timing) 2026-10-06 14:58:47 +02:00
Niko1221andAcckiyGerman e874e654fc hip: accept gfx1010 and gfx1011 (RDNA1) in the arch gate
gfx1010 (Navi 10) is measured on an RX 5700 XT: 8 GB card, Coder IQ1_M, 32K
context, prompt 115 / 144 / 149 tok/s and decode 18.0 / 20.3 / 23.3 tok/s at 4K /
16K / 30K prompt tokens, 6 of 6 needle checks at 8K and 30K, 61 of 61 ctest on the
card. Nothing beyond the arch list is needed: the portable scalar dp4a that main
falls to measured 162.7 / 24.3 against 158.3 / 24.0 with the __gfx1012__ SDWA
control (#442) - inside the run-to-run spread - and 16 of 16 HIP tests pass on it.

gfx1011 (Navi 12) is untested on hardware: same RDNA1 ISA, builds with 0 errors on
ROCm 7.14's gfx101X-dgpu wheels.

Two box facts, for whoever picks this up: hipBLASLt 1.4.0 refuses all 26 of the
engine's GEMM shapes on gfx1010 (heuristic_count = 0), so there is no tuning table
and prompts run on plain hipBLAS; and in a PC that also has an AMD iGPU, ROCm's HSA
runtime segfaults in GpuAgent::InitDma while enumerating the pair - below HIP, so
HIP_VISIBLE_DEVICES cannot reach it - and the engine must start with
ROCR_VISIBLE_DEVICES naming the one KFD node (rocminfo for the index).

Co-authored-by: AcckiyGerman <[email protected]>
2026-10-06 14:58:47 +02:00
BlueKingMuch 0af54b6f56 HIP: a failed first configure no longer leaves the kernels at -O0
CMake caches CMAKE_HIP_FLAGS_<CONFIG> while it enables HIP, from the
values its platform files set once the compiler is identified. When
HIP cannot compile CMake's identification source, that configure
caches them empty and then stops; every later configure of the same
build folder keeps the cached values, so the build that finally works
compiles every HIP source without -O, i.e. at -O0, while C and C++ get
-O3 -DNDEBUG. setup configures build-hip every time, so one failed
attempt is enough.

That happened on an RX 6800 (Windows 11, HIP SDK 7.2, CMake 4.0.1):
the first attempt failed on Visual Studio 2026's <cmath>, the next one
passed with the include fix, and the engine (0.1.30 with the Windows
build of #247, --expert-cache 2048) decoded at 0.24-0.27 tok/s with
7.1-7.5 s per decode window. STRATA_VERIFY_PROFILE put the time in
the GPU's own stages (per window: q8+qkv GEMVs 1468 ms and out-proj
849 ms in the DeltaNet layers, the LM head 540 ms); the waits for flags
the CPU raises were 41 ms of its 8.4 s per window. With the flags back:
59-62 ms per window, 28.1-28.3 tok/s.

After enable_language(HIP), hip_backend.cmake gives each empty
per-configuration HIP flag the value CMake computed for the compiler
(CMAKE_HIP_FLAGS_<CONFIG>_INIT), which is what a fresh build folder
gets. Flags with a value, from CMake or the command line, are left
alone; one set empty on purpose is filled too.

Reproduced on Windows (HIP SDK 7.2, CMake 4.0.1) with a first
configure that failed on --rocm-path flags split at the space in
C:/Program Files, and on Linux (CMake 3.28, TheRock's ROCm 7.10 clang)
with one that failed on a --rocm-device-lib-path that does not exist.
After a working second configure in the same folder,
CMAKE_HIP_FLAGS_RELEASE stays empty and native_mmvq.cu compiles
without -O; with this change it gets -O3 -DNDEBUG. A fresh folder gets
-O3 -DNDEBUG either way.

setup.py's engine fingerprint now covers cmake/ as well, so a change
there alone (this one) recompiles a locally built engine on its next
start; that includes NVIDIA engines compiled here.
2026-10-06 14:58:39 +02:00
Niko1221 2497a05c26 hip: opt-in gfx1103 (Radeon 780M iGPU): CMake accepts it, STRATA_EXPERIMENTAL_GFX1103=1 makes setup treat it as a unified-memory AMD card 2026-10-06 14:47:02 +02:00
imanu 569c094521 fix(mtp): --batch-mtp exits on the first admission with a draft-vocab head
The batch slots' drafters share the draft-vocab head but never copied its ggml type; it stayed -1 and the first
admission of --batch-mtp exited with "unsupported native MMVQ GGML type".
2026-10-06 14:17:54 +02:00
Niko1221andzhouxihong1 b4182a8d50 vmm: fall back to cudaGetDriverEntryPoint below CUDA 12.5 (#1071)
cudaGetDriverEntryPointByVersion arrived in CUDA 12.5; the CUDA 12.2-12.4 engine build stopped in vmm.cpp. Same
fallback expert_cache.cpp already carries, and the sticky error of a failed lookup is cleared.

Co-authored-by: zhouxihong1 <[email protected]>
2026-10-06 14:17:54 +02:00
xxDoman f236dbb85c gfx906: the 0.1.40 tree does not build on HIP/gfx906 (three compile errors)
The STRATA_HIP_GFX906 build defines STRATA_HIP_GFX906, not STRATA_USE_HIP, so three
sites that gate on STRATA_USE_HIP still take the CUDA branch and fail to compile:

1. fused_gr.cu: a bare 'return;' in the bool fused_gr_read_multi -> 'return false;'.
2. mtp.cpp: the MTP stream-priority block (cudaDeviceGetStreamPriorityRange) is gated
   on !defined(STRATA_HIP_GFX906) too.
3. vmm.cpp: the <cuda.h> include and the CUDA driver VMM code get the same gate.

All three are inside #if blocks that only the gfx906 build reaches.
2026-10-06 14:17:54 +02:00
aleesposito85 ac11069a97 fix(build): qualify size_t in vmm.hpp (GCC 12 rejects the unqualified name)
vmm.hpp only includes <cstdint> and <vector>, so the unqualified size_t is not
declared on toolchains that do not expose ::size_t through those headers
(GCC 12.2, Debian 12). Qualify it as std::size_t, as expert_cache.hpp does.
2026-10-06 14:17:54 +02:00
Niko1221 7b7f1a57c4 bench: add bench/results/COMMUNITY.md, an index of the community bench folders 2026-10-06 13:19:05 +02:00
Hugua700 1be67e8949 bench: RX 6800M (gfx1031) community report - self-built HIP engine on Windows (#1078)
Bench folder ported from PR #1078 by Hugua700, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:05 +02:00
fioruccione d6c444efdb bench: community report, 2x RTX 4060 Ti 16 GB + Threadripper PRO 3975WX, UD-IQ4_XS at 131K, layer split (#1062)
Bench folder ported from PR #1062 by fioruccione, authorship kept.
Trimmed for 0.1.40.2: 7 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:05 +02:00
sarge18 3f4d44ece2 Community benchmark: 2x Tesla P40 22 GB (Pascal, experimental CUDA 12), single card and layer split (#1028)
Bench folder ported from PR #1028 by sarge18, authorship kept.
Trimmed for 0.1.40.2: 8 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:05 +02:00
GPF cfb4a33495 bench: community report, RX 6700 XT 12 GB (gfx1031), Ryzen 7 5700G, I… (#1027)
Bench folder ported from PR #1027 by GPF, authorship kept.
Trimmed for 0.1.40.2: 6 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:05 +02:00
avarakin 60982e6ad6 Bench: RTX 5060 Ti 16 GB on Ryzen 9 7945HX, IQ3_S (#1016)
Bench folder ported from PR #1016 by avarakin, authorship kept.
Trimmed for 0.1.40.2: 21 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:05 +02:00
lfontanez 09fadfee61 Community benchmark: RTX 4070 Ti SUPER 16 GiB (Shin-BlackMamba, upstream 6f32ec07) (#995)
Bench folder ported from PR #995 by lfontanez, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: bench/results/2026-09-29-speed-0126/README.md, bench/results/2026-09-29-speed-0126/matrix.json, docs/COMMUNITY_BENCHMARKS.md, tools/integrate_speed_0126_shin_blackmamba.py.
2026-10-06 13:19:05 +02:00
Zauberio 18bce49fed bench: community report, RTX 5070 Ti + Ryzen 7 9800X3D on Windows 11, IQ3_S, 5K to 164K prompt tokens (#832)
Bench folder ported from PR #832 by Zauberio, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:05 +02:00
gravitomagnetic f129082f08 bench: community report — RTX 5090, IQ3_S at 1M context (YaRN), agent-style workload (#466)
Bench folder ported from PR #466 by gravitomagnetic, authorship kept.
Trimmed for 0.1.40.2: 1 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:05 +02:00
obiscr 3be9f44cda bench: community report, RTX 5070 Ti + Ryzen 9 5900X, 64 GB DDR4-3600 (IQ2_XS / IQ3_XXS / IQ3_S, six-arm A/B) (#993)
Bench folder ported from PR #993 by obiscr, authorship kept.
Trimmed for 0.1.40.2: 1 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:04 +02:00
1872183316 ffa7a90b74 bench: community report, RTX 4060 Ti + Xeon E5-2673 v3 (DDR3, PCIe 3.0 x8), Q2_0 and IQ3_S, 0.1.39 path vs #706 + #764 + a busier tier (#913)
Bench folder ported from PR #913 by 1872183316, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:04 +02:00
Mitsuasa513 2752f62687 Community benchmark: Tesla V100-SXM2-32GB on Windows (CUDA 12.6 / MSVC) (#902)
Bench folder ported from PR #902 by Mitsuasa513, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:04 +02:00
T-Crypt efa1ea2f91 bench: community report, Flash-Next IQ2_XS vs a dense 27B as coding agents (RTX 4090 + 32 GB RAM) (#882)
Bench folder ported from PR #882 by T-Crypt, authorship kept.
Trimmed for 0.1.40.2: 1 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
christopherrobertbrooks-tech abc802e528 bench: community report, Tesla V100 32 GB + RTX 4070 layer split with 16 GB of RAM (IQ2_XS) (#850)
Bench folder ported from PR #850 by christopherrobertbrooks-tech, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
T-Crypt 2cf6f8a6d7 bench: community report, RTX 4090 + 32 GB RAM at 512K context (IQ2_XS) (#834)
Bench folder ported from PR #834 by T-Crypt, authorship kept.
Trimmed for 0.1.40.2: 4 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
christopherrobertbrooks-tech 3366409f68 bench: community report, Tesla V100 32 GB with 16 GB of RAM (Coder IQ1_M) (#823)
Bench folder ported from PR #823 by christopherrobertbrooks-tech, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
did-technomancer 0b65d593c3 bench: community results for an RX 6800 on Windows (0.1.39 against 0.1.38) (#815)
Bench folder ported from PR #815 by did-technomancer, authorship kept.
Trimmed for 0.1.40.2: 3 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:04 +02:00
1314521gjy 1ce2c0ed35 Community benchmark: Flash-Next IQ3_S on an RTX 4080 SUPER (Windows), with the expert-pool worker sweep (#780)
Bench folder ported from PR #780 by 1314521gjy, authorship kept.
Trimmed for 0.1.40.2: 2 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:04 +02:00
30crows 239a1fb45c bench: community results for an RTX 4090 Laptop GPU (IQ3_S, 128K and 256K) (#757)
Bench folder ported from PR #757 by 30crows, authorship kept.
Trimmed for 0.1.40.2: 4 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:04 +02:00
xyzzing 8d6ae0cd83 Community benchmark: RX 7900 XTX 24 GB — ROCm 7.1.1 vs ROCm 10.2 nightly, serve-path decode (Qwen3.8-Flash-Next IQ3_S) (#745)
Bench folder ported from PR #745 by xyzzing, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
jase100k 7b52a0b62b Community benchmark: RX 7900 GRE (gfx1100, 16 GB), three Flash-Next quants (#740)
Bench folder ported from PR #740 by jase100k, authorship kept.
Trimmed for 0.1.40.2: 3 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
zimuhuan-code eb4e0c4b4f Community benchmark: 2× RTX 3060 12 GB, Qwen3.8-Flash-Next IQ3_S (three prompt sizes, 3 runs each) (#721)
Bench folder ported from PR #721 by zimuhuan-code, authorship kept.
Trimmed for 0.1.40.2: 8 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
pertain99 c9a554bc34 bench: community report, RTX 3090 Ti + 2x Xeon E5-2699 v3 (HP Z840), IQ3_S at 262K (#674)
Bench folder ported from PR #674 by pertain99, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:04 +02:00
Dmitry-B fe63d57a8d bench: community report — RTX 4090, Ryzen 9 7950X, Flash-Next IQ3_S (#624)
Bench folder ported from PR #624 by Dmitry-B, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:04 +02:00
HarukiOtaku 34ee687744 bench: Windows 11 + Radeon AI PRO R9700 (gfx1201), IQ2_XS, 4K to 248K prompt tokens (#618)
Bench folder ported from PR #618 by HarukiOtaku, authorship kept.
Trimmed for 0.1.40.2: 2 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
lijieming 8dd3c4e5a5 bench: add community RTX 4000 Ada 2-GPU vs 3-GPU layer-split results (#521)
Bench folder ported from PR #521 by lijieming, authorship kept.
Trimmed for 0.1.40.2: 2 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
Efs-O 99b9311c1f bench: community results, 2x RTX 5060 Ti 16 GB (PCIe gen3), Strata 0.1.34 (#483)
Bench folder ported from PR #483 by Efs-O, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
Ambolio 263c99b96d bench: community report, RTX 5090 + Ryzen 9 9950X3D, IQ3_S at the full 262K window; prefill auto:32768 A/B; conversation-cache 32 GiB demo (#440)
Bench folder ported from PR #440 by Ambolio, authorship kept.
Trimmed for 0.1.40.2: 4 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
qni-live c69872aa23 bench: community entry, 2x RTX PRO 4500, Swift IQ3_XXS, engine 0.1.30 and 0.1.36 (#418)
Bench folder ported from PR #418 by qni-live, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
pipeob0 822b9857c0 bench: IQ4_XS on a 64 GB PC - pinned arena vs experts.bin mmap vs the RAM budget (#416)
Bench folder ported from PR #416 by pipeob0, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
rwkeyes c888e4a993 bench(gfx1100): 0.1.31 vs 0.1.30 prefill/decode, curated entry + raw trials (#404)
Bench folder ported from PR #404 by rwkeyes, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:04 +02:00
steve8697 75b0554906 Add an RTX 5090 + Ryzen 7 9700X IQ3_S benchmark on engine 0.1.39 (#811)
Bench folder ported from PR #811 by steve8697, authorship kept.
Trimmed for 0.1.40.2: 27 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:03 +02:00
steve8697 c9fc157744 Add an RTX 5090 + Ryzen 7 9700X IQ3_XXS benchmark on engine 0.1.39 (#777)
Bench folder ported from PR #777 by steve8697, authorship kept.
Trimmed for 0.1.40.2: 21 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:03 +02:00
xxDoman c97ad41eb0 bench: community MI50 (gfx906) — Strata 0.1.38 speed and recall (#758)
Bench folder ported from PR #758 by xxDoman, authorship kept.
Trimmed for 0.1.40.2: 21 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:03 +02:00
CYoung83 f546be7e3f Bench: RTX 5090 / 9950X3D (IQ3_S, three-pass A/B) + RTX 2080 Ti / 9900KF (IQ3_XXS, Turing) (#698)
Bench folder ported from PR #698 by CYoung83, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:03 +02:00
Yasei-no-otoko cc86ad90de bench: report Coder IQ1_M on Threadripper 3990X and RX 6900 XT (#628)
Bench folder ported from PR #628 by Yasei-no-otoko, authorship kept.
Trimmed for 0.1.40.2: 30 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/AMD_HIP.md.
2026-10-06 13:19:03 +02:00
R6DJO 9aa0f3e5df Community benchmark: 2x RTX 5060 Ti (16 GB), Xeon E5-2690 v4, IQ3_XXS, engine 0.1.35 (#508)
Bench folder ported from PR #508 by R6DJO, authorship kept.
Trimmed for 0.1.40.2: 40 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:03 +02:00
brenoperucchi 2a7db04226 bench: community report, RTX 5090 + Ryzen 9 5950X (AVX2), UD-Q4_K_XL / IQ3_S / Swift IQ3_XXS on engines 0.1.31-0.1.39 (#433)
Bench folder ported from PR #433 by brenoperucchi, authorship kept.
Trimmed for 0.1.40.2: 2 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:03 +02:00
mlfather 8c15b38be7 bench: community report, RTX PRO 4500 x1/x2 + RTX PRO 4000, Threadripper PRO 3975WX, engine 0.1.31 (#417)
Bench folder ported from PR #417 by mlfather, authorship kept.
Trimmed for 0.1.40.2: 162 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
2026-10-06 13:19:03 +02:00
noahark afee5009d8 docs: community benchmark row - Tesla V100-PCIE-32GB (sm_70 source build, before/after calibration) (#707)
Bench folder ported from PR #707 by noahark, authorship kept.
Trimmed for 0.1.40.2: 0 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:03 +02:00
homeofe aebd0f34d7 bench: community results, RX 9070 XT 16 GB on Windows 11, ready-made AMD engine 0.1.35, IQ3_S (#499)
Bench folder ported from PR #499 by homeofe, authorship kept.
Trimmed for 0.1.40.2: 2 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:03 +02:00
homeofe 707fd49eb4 bench: community results, RTX 2080 Ti 11 GB + Threadripper 3960X, IQ3_S at 262K with KV streaming (#469)
Bench folder ported from PR #469 by homeofe, authorship kept.
Trimmed for 0.1.40.2: 4 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:03 +02:00
shrisha108 219960ef34 bench: community report, 2x TITAN RTX (sm_75), IQ3_S at 262K (#389)
Bench folder ported from PR #389 by shrisha108, authorship kept.
Trimmed for 0.1.40.2: 1 raw dumps/logs/binaries left out (listed in TRIMMED.md of each folder).
Not ported: docs/COMMUNITY_BENCHMARKS.md.
2026-10-06 13:19:03 +02:00
Niko1221 4211e3b338 docs: VRAM_ELASTIC.md (what POST /v1/vram answers, #1034); MULTI_GPU.md: Pascal/Volta cards admitted when named (#1029) 2026-10-06 13:13:50 +02:00
Niko1221 82f46a8c8f serve: a tool call inside a code fence or inline code in the visible answer is text (#1058)
The live smoke showed the same quoted-example problem in the answer: a model that wrote the call format in a
fenced example after </think> got a real tool call. The content parser now uses the fence / inline-code
tracking of the reasoning rescue: an opener inside a ``` or ~~~ fence (tracked across stream chunks) or inside
inline code stays text, the decision depending only on the text before it, so streamed and whole outputs agree.
Openers at the top level fire as before, including one after a sentence on the same line (none of the 37
corpus specimens needed that, but the old tests do and nothing argues against it). A closed call at the end of
an answer cut by max tokens stays a call.

Tests: quoted calls stay text and real ones fire at every chunk split; OpenAI (whole and streamed), Anthropic
and Responses over HTTP.
2026-10-06 11:25:45 +02:00
Niko1221 b6c2e01589 serve: a request that meets a restart in progress gets an EngineDied, not an AttributeError (#1012)
Found in the live smoke: an old request that learns its engine died while restart() has already taken the
process (self.proc is None) failed in death_note() with an AttributeError, and its client got no reply at
all. death_note(), alive() and _send() now read the process once and treat None as a dead engine.

Test: after close() the death note is a sentence, alive() is False and a send raises EngineDied.
2026-10-06 10:36:01 +02:00
Niko1221 3a711ae90b serve: a --batch request whose engine died during its prompt read ends at once instead of after 300 s (#1012)
Found in the live smoke of #1012: a long prompt being admitted when the engine is killed read its own
end-of-engine line (EngineDied), and its cleanup then drained the control lines for up to 300 s for an
end line that had been read already; the client got its 503 only after that. The cleanup now skips the
STOP and the drain when the engine process the request was sent to is gone (the server counts its engine
processes in gen, and a request notes the one it has the control lines of), and the drain polls once a
second and gives up as soon as that engine is gone.

Tests: a long read killed alone, and with short requests arriving meanwhile (it gives way and is admitted):
every request ends in a 503 within seconds, and the next request restarts the engine.
2026-10-06 10:22:06 +02:00
Niko1221 511b646783 setup: a four-part hotfix version cannot break the version code
Versions are read as their first three numbers everywhere; the one place that was not (the version compiled
into an engine binary, `engine=0.1.40.1`) now accepts an optional fourth number. A test pins that a hotfix
tag keeps finding the engine zips: the checkout's v<project version> release first, the latest release second.
2026-10-06 10:03:35 +02:00
Niko1221 57a69eb48f serve: a restart keeps the admission state, so waiting requests no longer hang or fail with ValueError (#1012)
StrataEngine.restart() runs __init__ again, which made a new condition variable, control lock, wait counter
and wait list while requests were still waiting on the old ones: they hung for good, or failed with
`list.remove(x): x not in list`. The admission state is now made once and kept; restart() wakes the waiters
when it ends. A request that waits for the control lines now ends with a clean EngineDied (503, "not sent")
when the engine is gone and nobody is starting it, keeps waiting while a restart is under way, and goes on
with the new engine when it is up. The lock is checked again after it is taken, a slot that frees itself
after a restart keeps the lists of its own process, and the wait counters cannot go below zero.

Tests (fake batch engine behind the real StrataEngine and Service): waiters during a restart are served by
the new engine (2 and 4 slots, same condition variable and lock), get a clean 503 when the engine stays down,
and when the restart itself fails; the next request restarts and is answered; the solo engine is unchanged.
2026-10-06 10:03:35 +02:00
Niko1221 38357f1858 serve: a tool call inside the thinking is a call only when it was an act (#804, #1058)
The #804 rescue turned any declared <tool_call> in the reasoning into a real call, so a call that was only
quoted (in a code fence, mid-sentence, in inline code, followed by more prose) ran, and so did one in a reply
cut by max tokens (47Hunter47's `rm -rf build` case). Now it must start a line (after optional whitespace),
be outside a code fence (``` or ~~~, tracked across chunks) and inline code, close with </tool_call>, and have
only whitespace or more calls after it before the end of the turn or </think>. Such calls wait until then,
and become reasoning text again when the turn ends by length, cancel or error. The decision is the same
streamed or whole. Declared calls beside an undeclared wrapper are still delivered, the wrapper stays
reasoning.

The server tells the parser how the turn ended, logs how many calls were read or kept as text, and counts
the read ones in the metrics totals (tool_calls_from_reasoning).

Tests: 37 recorded and constructed specimens (PR #525's corpus and 47Hunter47's 12 live strandings; 16 act,
21 do not) at chunk sizes 1, 2, 3, 7, random and whole; fence and opener split at every position; OpenAI
(whole and streamed), Anthropic and Responses over HTTP.
2026-10-06 10:03:35 +02:00
Chen Cheng 5b5455982f serve: an engine that exits after an unread ERR line says why (#997 #890)
With "parallel": N, a batch window that fails prints its reason as an ERR
line on stdout and the engine exits with code 1. At that point every
request reads its own slot's lines, so nobody reads that ERR line, and the
server's note fell back to the log's last "strata" line (a stats line in
#997), or to "the usual cause is running out of RAM" without a log.

The pump now keeps an ERR line that was the engine's last stdout line, and
death_note gives it first. test_parallel's fake engine can fail its first
window over two slots: the note names the error.

(cherry picked from commit 9c5066303e)
2026-10-06 09:45:16 +02:00
3933 changed files with 778671 additions and 8634 deletions

No files matched your search

+18
View File
@@ -4,6 +4,24 @@ bench/prompts/** -text
# launchers keep their platform line endings wherever the repository is cloned
*.sh text eol=lf
*.bat text eol=crlf
*.cmd text eol=crlf
*.ps1 text eol=crlf
bin/strata text eol=lf
# copies of real packs' native_experts.txt, parsed byte for byte by tests/core/expert_layout_test.cpp
tests/data/native_experts/** -text
# the quant yardstick's text set: tokenized byte for byte (set.ids.json), never convert its line endings
data/quant_quality/** -text
# the license text is shipped and checked as LF text on every checkout
LICENSE text eol=lf
# container scripts run in Linux: always LF
docker/*.sh text eol=lf
Dockerfile text eol=lf
docker/Dockerfile text eol=lf
docker/compose/* text eol=lf
# Apple tools (codesign, plutil) read these: LF only, wherever the repository is cloned
*.plist text eol=lf
+26
View File
@@ -0,0 +1,26 @@
name: Benchmark
description: Share measurements from your machine, or report a problem with the benchmark tools
title: "[Benchmark] "
labels: ["benchmark"]
body:
- type: markdown
value: |
The title must start with **[Benchmark]**. Keep it, and add the GPU and model after it.
To share a full report, follow the [community benchmark guide](https://github.com/Niko1221/Strata/blob/main/docs/COMMUNITY_BENCHMARKS.md) and open a results-only pull request. The standard protocol is a [draft](https://github.com/Niko1221/Strata/blob/main/docs/BENCHMARK_PROTOCOL.md); existing tools do not implement it yet. Include poor results and failed requests.
- type: textarea
id: what
attributes:
label: What did you measure, or what went wrong
description: Hardware (GPU and VRAM, CPU, RAM, OS), Strata version, model and quantization, the exact command, and the numbers. Paste the report or link the pull request.
validations:
required: true
- type: dropdown
id: reproduction-status
attributes:
label: Reproduction status
options:
- Measured locally
- Unconfirmed (I did not run it)
validations:
required: true
+101
View File
@@ -0,0 +1,101 @@
name: Bug report
description: Something in Strata doesn't work as documented
title: "[Area] "
labels: ["bug"]
body:
- type: markdown
value: |
Thanks for the report. Please read [CONTRIBUTING.md](https://github.com/Niko1221/Strata/blob/main/CONTRIBUTING.md) first.
Search the [open and closed issues](https://github.com/Niko1221/Strata/issues?q=is%3Aissue) first: this may already be tracked, and new evidence belongs in that thread.
Write only what you saw. Do not guess at steps, errors, hardware or severity.
**The title must start with its area:** [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other]. Choose the same one below.
- type: dropdown
id: area
attributes:
label: Area
description: Start the issue title with the same area in square brackets, for example "[Engine] decode stops after 2 minutes". Pick Other if none fits.
options:
- "[Engine] (speed, kernels, models, the server and API)"
- "[GUI] (the desktop app)"
- "[TUI] (the Terminal: strata launch, connect, monitor)"
- "[Benchmark] (measurements and reports)"
- "[Docs] (documentation)"
- "[Other] (anything else, including setup and installation)"
validations:
required: true
- type: dropdown
id: how
attributes:
label: Desktop app, Terminal or source?
options:
- Desktop app (Windows, Linux or Mac)
- Terminal (the `strata` command)
- Source (START-HERE.bat, setup.sh or a build)
- Docker
- Other
validations:
required: true
- type: textarea
id: what
attributes:
label: What happened
description: What you did, what you got, and what you expected instead. Paste the exact error text if there is one.
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Smallest steps or command that reproduce it
description: The shortest list of steps that shows the problem. If you cannot make it happen again, say so here.
validations:
required: false
- type: dropdown
id: reproduction-status
attributes:
label: Reproduction status
options:
- Reproduced locally
- Unconfirmed static analysis (I read code or ran a scanner; I did not see it fail)
validations:
required: true
- type: input
id: version
attributes:
label: Strata version
description: "In the app: Settings > About. In the Terminal: `strata --version`. From source: the About tab of the web page, or `engine` in `GET /v1/status`, or the git commit."
placeholder: "Strata 0.2.1 · engine 0.1.46"
validations:
required: true
- type: input
id: hardware
attributes:
label: Graphics card, memory and system
description: The card and its VRAM, the RAM of the PC, and the operating system.
placeholder: "RTX 5070 12 GB · 64 GB RAM · Windows 11"
validations:
required: true
- type: input
id: model
attributes:
label: Model, size and settings
description: The model and size you started, the context, and whether pictures were on. Write what is relevant.
placeholder: "Qwen3.8-Flash-Next IQ2_XS · 128K context · pictures on"
validations:
required: false
- type: textarea
id: log
attributes:
label: Logs
description: |
The app: Settings > Engine > Diagnostics > View logs > Save logs, then drag the file in. The Terminal: paste the output of `strata doctor`. From source: drag in `strata-<model>.log` from your Strata folder.
Remove API keys, passwords and private text first.
validations:
required: false
- type: markdown
value: |
A couple of extras that help a lot:
- Check the [troubleshooting page](https://github.com/Niko1221/Strata/blob/main/docs/TROUBLESHOOTING.md) first, many reports are already solved there.
+5
View File
@@ -0,0 +1,5 @@
blank_issues_enabled: true
contact_links:
- name: Troubleshooting and known issues
url: https://github.com/Niko1221/Strata/blob/main/docs/TROUBLESHOOTING.md
about: Symptom-to-fix table for crashes, out of memory, hangs, and port problems.
@@ -0,0 +1,41 @@
name: Feature request
description: Suggest something new or an improvement
title: "[Area] "
labels: ["enhancement"]
body:
- type: markdown
value: |
Explain what you want in plain language. For a large feature, a change to a default or a refactor, please agree the scope here first. Read [CONTRIBUTING.md](https://github.com/Niko1221/Strata/blob/main/CONTRIBUTING.md).
**The title must start with its area:** [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other]. Choose the same one below.
- type: dropdown
id: area
attributes:
label: Area
description: Start the issue title with the same area in square brackets, for example "[Engine] decode stops after 2 minutes". Pick Other if none fits.
options:
- "[Engine] (speed, kernels, models, the server and API)"
- "[GUI] (the desktop app)"
- "[TUI] (the Terminal: strata launch, connect, monitor)"
- "[Benchmark] (measurements and reports)"
- "[Docs] (documentation)"
- "[Other] (anything else, including setup and installation)"
validations:
required: true
- type: textarea
id: idea
attributes:
label: What do you want to do?
description: Describe the goal rather than the implementation. Say who needs it and why.
validations:
required: true
- type: textarea
id: workaround
attributes:
label: What do you do today instead?
description: The workaround you use now, if there is one.
validations:
required: false
- type: markdown
value: |
All of this counts: model and architecture support, the server and API, the app and the Terminal, multi-GPU and layer split, images and vision, MCP servers, setup and installation, and performance.
+45
View File
@@ -0,0 +1,45 @@
name: Question
description: Ask how to use Strata, or share a result
title: "[Area] "
labels: ["question"]
body:
- type: markdown
value: |
Not sure how to do something, or want to share numbers from your machine? Ask here. To share a full benchmark, see [CONTRIBUTING.md](https://github.com/Niko1221/Strata/blob/main/CONTRIBUTING.md#share-a-benchmark).
**The title must start with its area:** [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other]. Choose the same one below.
- type: dropdown
id: area
attributes:
label: Area
description: Start the issue title with the same area in square brackets, for example "[Engine] decode stops after 2 minutes". Pick Other if none fits.
options:
- "[Engine] (speed, kernels, models, the server and API)"
- "[GUI] (the desktop app)"
- "[TUI] (the Terminal: strata launch, connect, monitor)"
- "[Benchmark] (measurements and reports)"
- "[Docs] (documentation)"
- "[Other] (anything else, including setup and installation)"
validations:
required: true
- type: textarea
id: question
attributes:
label: Your question
description: What you're trying to do. Include your version, GPU, and OS if it affects the answer.
validations:
required: true
- type: textarea
id: tried
attributes:
label: What you tried
description: What you already did and where you got stuck.
validations:
required: false
- type: textarea
id: numbers
attributes:
label: Results (optional)
description: Only if you ran a benchmark or measured speed. Model, quant, context, and the tok/s you get.
validations:
required: false
+30
View File
@@ -0,0 +1,30 @@
**The title must start with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other], for example "[Engine] fix decode stop on long prompts" or "[GUI] keep the model list after a tab switch".**
<!--
Read CONTRIBUTING.md first. Lead with the problem and the result, not a file-by-file list.
Placeholders below are not test results: replace them with what you observed, or write "none".
-->
## Problem and result
<!-- What triggered this change. How it behaves before and after. -->
## Related discussion
<!-- The issue URL and the maintainer's agreement on scope, when the change needs one (features, refactors, defaults). Write "none" if not applicable. -->
## Approach
<!-- Why this fix. Compatibility effects. Keep unrelated cleanup out of the PR. -->
## Validation
<!-- Exact commands, the revision and the environment tested, what you saw. Keep the three lists apart. -->
**Passed:**
<!-- command, revision/environment, observed result, log -->
**Failed:**
<!-- command and failure, or "none observed" -->
**Not run:**
<!-- check, why, and what stays unverified (for example CUDA, HIP, SYCL or Metal builds you could not do) -->
## Limits
<!-- Known limitations or remaining work. -->
+218
View File
@@ -0,0 +1,218 @@
'use strict';
// Format check for issues and pull requests (see CONTRIBUTING.md and .github/ISSUE_TEMPLATE/*.yml).
// Pure functions only: no network, no filesystem. Used by intake.yml and reopen.yml through reset-bot.js.
const AREAS = ['Engine', 'GUI', 'TUI', 'Benchmark', 'Docs', 'Other'];
const TITLE_RE = new RegExp('^\\[(' + AREAS.join('|') + ')\\] \\S');
const NO_ANSWER = /^(_no response_|none|n\/a|na|-|todo|tbd|\.|\?)$/i;
const NOT_RUN_LABELS = /^\*\*(passed|failed|not run)\s*:?\*\*\s*:?/gim;
// Version the reopened work must be tested on.
const CURRENT_VERSION_RE = /\b0\.2\.\d+|0\.2\.0-beta/i;
// Any version or commit.
const ANY_VERSION_RE = /\bv?\d+\.\d+(\.\d+)*(-[a-z0-9.]+)?\b|\b[0-9a-f]{7,40}\b|\bcommit\b/i;
const HARDWARE_RE = /\b(rtx|gtx|geforce|quadro|tesla|radeon|rx ?\d|instinct|arc|a\d{3,4}|b\d{2,3}|apple|m[1-5]( pro| max| ultra)?|nvidia|amd|intel|gpu|cpu|vram|\d+ ?gb|ram|windows|linux|ubuntu|debian|fedora|arch|macos|mac os|wsl)\b/i;
const REVISION_RE = /0\.2\.\d|beta|\bmain\b|\b[0-9a-f]{7,40}\b/i;
function stripComments(text) {
return String(text || '').replace(/<!--[\s\S]*?-->/g, '');
}
function hasAreaPrefix(title) {
return TITLE_RE.test(String(title || ''));
}
// Issue forms render as "### Label\n\nvalue". Returns { label: value } for those headings.
function parseSections(body) {
const out = {};
let cur = null;
for (const line of stripComments(body).split(/\r?\n/)) {
const m = /^#{1,6}\s+(.+?)\s*#*\s*$/.exec(line);
if (m) {
cur = m[1].trim();
out[cur] = '';
} else if (cur !== null) {
out[cur] += line + '\n';
}
}
for (const k of Object.keys(out)) out[k] = out[k].trim();
return out;
}
function realText(value) {
const t = String(value || '').replace(NOT_RUN_LABELS, '').replace(/\s+/g, ' ').trim();
return NO_ANSWER.test(t) ? '' : t;
}
function findSection(sections, re) {
for (const k of Object.keys(sections)) if (re.test(k)) return { key: k, value: sections[k] };
return null;
}
// Required parts of each issue form: [display name, heading regex, minimum characters of real text].
const FORMS = {
bug: {
detect: /what happened/i,
parts: [
['Area', /^area$/i, 2],
['Desktop app, Terminal or source?', /desktop app, terminal or source/i, 2],
['What happened', /what happened/i, 20],
['Reproduction status', /reproduction status/i, 2],
['Strata version', /strata version/i, 3],
['Graphics card, memory and system', /graphics card|memory and system/i, 3],
],
needsCurrentVersion: 'Strata version',
},
feature: {
detect: /what do you want to do/i,
parts: [
['Area', /^area$/i, 2],
['What do you want to do?', /what do you want to do/i, 20],
],
},
question: {
detect: /your question/i,
parts: [
['Area', /^area$/i, 2],
['Your question', /your question/i, 15],
],
},
benchmark: {
detect: /what did you measure/i,
parts: [
['What did you measure, or what went wrong', /what did you measure/i, 30],
['Reproduction status', /reproduction status/i, 2],
],
needsCurrentVersionInBody: true,
},
};
function checkIssue({ title, body, labels = [], mode = 'intake' }) {
const missing = [];
if (!hasAreaPrefix(title)) {
missing.push('Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other], then a short description. Example: "[Engine] decode stops after 2 minutes".');
}
const text = stripComments(body);
const sections = parseSections(text);
const labelSet = new Set(labels.map((l) => String(l).toLowerCase()));
const needCurrent = mode === 'reopen';
let formKey = null;
for (const [key, f] of Object.entries(FORMS)) {
if (findSection(sections, f.detect)) { formKey = key; break; }
}
if (formKey) {
const form = FORMS[formKey];
for (const [name, re, min] of form.parts) {
const s = findSection(sections, re);
if (!s || realText(s.value).length < min) missing.push('Fill in "' + name + '".');
}
if (needCurrent && form.needsCurrentVersion) {
const s = findSection(sections, /strata version/i);
if (s && realText(s.value) && !CURRENT_VERSION_RE.test(s.value)) {
missing.push('Re-test on Strata 0.2.0-beta and put that version in "' + form.needsCurrentVersion + '". If it no longer happens, say so in a comment instead.');
}
}
if (needCurrent && form.needsCurrentVersionInBody && !CURRENT_VERSION_RE.test(text)) {
missing.push('Say which Strata version you measured on. Re-measure on 0.2.0-beta if you can.');
}
return { ok: missing.length === 0, missing, kind: formKey };
}
// Free-form text (older issues, blank issues).
const plain = realText(text);
const lenient = labelSet.has('enhancement') || labelSet.has('question');
if (plain.length < (lenient ? 40 : 80)) {
missing.push('Describe it in the issue body: what happened, what you expected, and the exact error if there is one. The issue forms (New issue > Bug report) list what to include.');
}
if (!lenient) {
if (!ANY_VERSION_RE.test(text)) missing.push('Add the Strata version (Settings > About, or `strata --version`).');
else if (needCurrent && !CURRENT_VERSION_RE.test(text)) missing.push('Re-test on Strata 0.2.0-beta and name that version in the issue. If it no longer happens, say so in a comment instead.');
if (!HARDWARE_RE.test(text)) missing.push('Add your graphics card with its VRAM, your RAM and your operating system.');
}
return { ok: missing.length === 0, missing, kind: 'free-form' };
}
function checkPull({ title, body, mode = 'intake' }) {
const missing = [];
if (!hasAreaPrefix(title)) {
missing.push('Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other], then a short description. Example: "[Engine] fix decode stop on long prompts".');
}
const text = stripComments(body);
const sections = parseSections(text);
const problem = findSection(sections, /problem|result|summary|what.*change/i);
const validation = findSection(sections, /validation|testing|tests?\b|how.*tested/i);
let valText = '';
if (problem || validation) {
if (!problem || realText(problem.value).length < 20) missing.push('Fill in "Problem and result": what triggered the change, and how it behaves before and after.');
if (!validation || realText(validation.value).length < 20) {
missing.push('Fill in "Validation": the exact commands, the revision and environment tested, what you saw. Keep Passed, Failed and Not run apart.');
} else {
valText = validation.value;
}
} else {
const plain = realText(text);
if (plain.length < 150) {
missing.push('Describe the pull request: the problem and the result. The pull request template shows the parts (Problem and result, Related discussion, Approach, Validation, Limits).');
}
if (!/(problem|result|before|after|fix|because|bug)/i.test(plain)) missing.push('Add "Problem and result": what triggered the change, and how it behaves before and after.');
if (!/(validat|tested|passed|ran |built|build|not run)/i.test(plain)) missing.push('Add "Validation": the exact commands, the revision and environment tested, and what you saw.');
valText = plain;
}
if (mode === 'reopen' && valText && !REVISION_RE.test(valText)) {
missing.push('Say in Validation which revision you tested: rebase on main (or 0.2.0-beta) and re-run your checks.');
}
return { ok: missing.length === 0, missing, kind: 'pull' };
}
function checkItem({ isPull, ...rest }) {
return isPull ? checkPull(rest) : checkIssue(rest);
}
function checklist(missing) {
return missing.map((m) => '- [ ] ' + m).join('\n');
}
const CONTRIB = 'https://github.com/Niko1221/Strata/blob/main/CONTRIBUTING.md';
function intakeComment(isPull, missing) {
return [
'Thanks for the ' + (isPull ? 'pull request' : 'issue') + '. To keep the tracker readable, every item needs an area prefix in the title and the parts of the ' + (isPull ? 'pull request template' : 'issue form') + ' filled in. This one is missing:',
'',
checklist(missing),
'',
'Edit the ' + (isPull ? 'pull request' : 'issue') + ' (title and description) and this note goes away on its own. If it is still missing parts in 5 days it will be closed, which is not a rejection: fix it and comment `/reopen`.',
'',
'Details: [CONTRIBUTING.md](' + CONTRIB + ')',
].join('\n');
}
function reopenMissingComment(isPull, missing) {
return [
'Not reopened yet. Still missing:',
'',
checklist(missing),
'',
'Edit the ' + (isPull ? 'pull request' : 'issue') + ' and comment `/reopen` again. Details: [CONTRIBUTING.md](' + CONTRIB + ')',
].join('\n');
}
function reopenOkComment(isPull) {
return 'Reopened, thank you. A maintainer will pick it up from here.' + (isPull ? ' If it conflicts with main, rebase it first; that speeds up review.' : '');
}
function staleComment(isPull) {
return [
'Closing this ' + (isPull ? 'pull request' : 'issue') + ' because it still did not follow the format after 5 days. That is housekeeping, not a rejection.',
'',
'To bring it back: fix the title (area prefix) and the missing parts, then comment `/reopen`. A check runs and reopens it, or tells you what is still missing. See [CONTRIBUTING.md](' + CONTRIB + ').',
].join('\n');
}
module.exports = {
AREAS, TITLE_RE, hasAreaPrefix, parseSections, stripComments, realText,
checkIssue, checkPull, checkItem, checklist,
intakeComment, reopenMissingComment, reopenOkComment, staleComment,
};
+192
View File
@@ -0,0 +1,192 @@
'use strict';
const test = require('node:test');
const assert = require('node:assert');
const c = require('./item-check');
const bot = require('./reset-bot');
const bugForm = (over = {}) => {
const f = {
'Area': '[Engine] (speed, kernels, models, the server and API)',
'Desktop app, Terminal or source?': 'Terminal (the `strata` command)',
'What happened': 'Decode stops after two minutes with "CUDA error 700". I expected it to continue.',
'Smallest steps or command that reproduce it': '_No response_',
'Reproduction status': 'Reproduced locally',
'Strata version': 'Strata 0.2.0-beta · engine 0.1.45.3',
'Graphics card, memory and system': 'RTX 5070 12 GB · 64 GB RAM · Windows 11',
...over,
};
return Object.entries(f).map(([k, v]) => `### ${k}\n\n${v}`).join('\n\n');
};
test('title prefix', () => {
assert.ok(c.hasAreaPrefix('[Engine] fix decode'));
assert.ok(c.hasAreaPrefix('[Docs] x'));
assert.ok(!c.hasAreaPrefix('Engine fix'));
assert.ok(!c.hasAreaPrefix('[engine] fix'));
assert.ok(!c.hasAreaPrefix('[Engine]'));
assert.ok(!c.hasAreaPrefix('[Engine] '));
assert.ok(!c.hasAreaPrefix('[Area] foo'));
assert.ok(!c.hasAreaPrefix(''));
assert.ok(!c.hasAreaPrefix(undefined));
});
test('filled bug form passes in both modes', () => {
for (const mode of ['intake', 'reopen']) {
const r = c.checkIssue({ title: '[Engine] decode stops', body: bugForm(), labels: ['bug'], mode });
assert.deepStrictEqual(r.missing, []);
assert.strictEqual(r.kind, 'bug');
}
});
test('bug form: missing parts are named', () => {
const r = c.checkIssue({ title: 'decode stops', body: bugForm({ 'Strata version': '_No response_', 'What happened': 'bad' }) });
assert.ok(!r.ok);
assert.ok(r.missing.some((m) => /title/i.test(m)));
assert.ok(r.missing.some((m) => m.includes('What happened')));
assert.ok(r.missing.some((m) => m.includes('Strata version')));
});
test('reopen needs a 0.2 version, intake does not', () => {
const body = bugForm({ 'Strata version': 'Strata 0.1.44' });
assert.ok(c.checkIssue({ title: '[Engine] x y', body, mode: 'intake' }).ok);
const r = c.checkIssue({ title: '[Engine] x y', body, mode: 'reopen' });
assert.ok(!r.ok);
assert.ok(r.missing.some((m) => m.includes('0.2.0-beta')));
});
test('html comments and placeholders do not count', () => {
const r = c.checkIssue({ title: '[Engine] x y', body: bugForm({ 'What happened': '<!-- describe -->' }) });
assert.ok(r.missing.some((m) => m.includes('What happened')));
});
test('feature and question forms', () => {
const f = '### Area\n\n[GUI] (the desktop app)\n\n### What do you want to do?\n\nLet me pin a model to the top of the list in the app.';
assert.ok(c.checkIssue({ title: '[GUI] pin models', body: f, mode: 'reopen' }).ok);
assert.ok(!c.checkIssue({ title: '[GUI] pin models', body: '### Area\n\n[GUI]\n\n### What do you want to do?\n\n_No response_' }).ok);
const q = '### Area\n\n[Other]\n\n### Your question\n\nHow do I move the model folder to another drive?';
assert.ok(c.checkIssue({ title: '[Other] model folder', body: q }).ok);
});
test('free-form issue', () => {
const good = 'Strata 0.2.0-beta on an RTX 3060 12 GB with 32 GB RAM on Linux: the server hangs when I send two requests at once. Expected both to finish.';
assert.ok(c.checkIssue({ title: '[Engine] hang on two requests', body: good, mode: 'reopen' }).ok);
assert.ok(!c.checkIssue({ title: '[Engine] hang', body: 'it crashes', mode: 'intake' }).ok);
const old = good.replace('0.2.0-beta', '0.1.40');
assert.ok(c.checkIssue({ title: '[Engine] hang', body: old, mode: 'intake' }).ok);
assert.ok(!c.checkIssue({ title: '[Engine] hang', body: old, mode: 'reopen' }).ok);
assert.ok(c.checkIssue({ title: '[Other] idea', body: 'It would help if the app remembered my last model between runs, today I pick it again.', labels: ['enhancement'], mode: 'reopen' }).ok);
});
const prBody = (over = {}) => {
const p = {
problem: 'Decode stopped on prompts above 8k because the stop check read the wrong length. Now it continues.',
validation: '**Passed:** `ctest -R decode`, commit abcdef1, RTX 3060, 12 of 12.\n\n**Failed:** none observed\n\n**Not run:** HIP build',
...over,
};
return `## Problem and result\n<!-- hint -->\n${p.problem}\n\n## Related discussion\nnone\n\n## Approach\nFix the length.\n\n## Validation\n${p.validation}\n\n## Limits\nnone`;
};
test('filled PR passes', () => {
assert.deepStrictEqual(c.checkPull({ title: '[Engine] fix stop', body: prBody(), mode: 'reopen' }).missing, []);
});
test('untouched PR template fails', () => {
const tpl = '## Problem and result\n<!-- What triggered -->\n\n## Validation\n<!-- cmds -->\n\n**Passed:**\n<!-- x -->\n\n**Failed:**\n\n**Not run:**\n';
const r = c.checkPull({ title: '[Engine] fix stop', body: tpl });
assert.ok(!r.ok);
assert.ok(r.missing.some((m) => m.includes('Problem and result')));
assert.ok(r.missing.some((m) => m.includes('Validation')));
});
test('PR: empty body and bad title', () => {
const r = c.checkPull({ title: 'fix', body: null });
assert.ok(!r.ok);
assert.ok(r.missing.length >= 2);
});
test('PR reopen needs a revision', () => {
const body = prBody({ validation: '**Passed:** I tried it and it worked for me on my machine, looked fine.' });
assert.ok(c.checkPull({ title: '[Engine] x y', body, mode: 'intake' }).ok);
assert.ok(!c.checkPull({ title: '[Engine] x y', body, mode: 'reopen' }).ok);
});
test('free-form PR', () => {
const body = 'This fixes a crash when the context is above 8k, because the length was read from the wrong field. Before it aborted, after it continues. I built it on CUDA and ran the decode test, passed.';
assert.ok(c.checkPull({ title: '[Engine] fix crash', body }).ok);
assert.ok(!c.checkPull({ title: '[Engine] fix crash', body: 'fixes stuff' }).ok);
});
test('messages', () => {
const m = c.intakeComment(false, ['a', 'b']);
assert.ok(m.includes('- [ ] a') && m.includes('/reopen') && m.includes('CONTRIBUTING.md'));
assert.ok(c.staleComment(true).includes('/reopen'));
});
// handlers with a fake GitHub client
function fake(calls, extra = {}) {
const rest = {
issues: {
getLabel: async () => ({}),
createLabel: async (a) => calls.push(['createLabel', a.name]),
addLabels: async (a) => calls.push(['addLabels', a.labels]),
removeLabel: async (a) => calls.push(['removeLabel', a.name]),
createComment: async (a) => calls.push(['comment', a.body]),
update: async (a) => calls.push(['update', a.state]),
get: async () => ({ data: extra.live }),
listEvents: async () => {},
listForRepo: async () => {},
},
pulls: { get: async () => ({ data: { mergeable_state: 'clean' } }) },
};
return { rest, paginate: async (fn) => (fn === rest.issues.listEvents ? (extra.events || []) : (extra.items || [])) };
}
const core = { info() {} };
const repo = { owner: 'o', repo: 'r' };
test('intake labels and comments once', async () => {
const calls = [];
const item = { number: 1, title: 'bad', body: '', labels: [], user: { type: 'User' }, author_association: 'NONE' };
await bot.intake({ github: fake(calls), context: { repo, payload: { action: 'opened', issue: item } }, core });
assert.deepStrictEqual(calls.map((c) => c[0]), ['addLabels', 'comment']);
calls.length = 0;
item.labels = [{ name: 'needs-format' }];
await bot.intake({ github: fake(calls), context: { repo, payload: { action: 'edited', issue: item } }, core });
assert.deepStrictEqual(calls, []);
});
test('intake clears the label when edited into shape', async () => {
const calls = [];
const item = { number: 1, title: '[Engine] x y', body: bugForm(), labels: [{ name: 'needs-format' }], user: { type: 'User' } };
await bot.intake({ github: fake(calls), context: { repo, payload: { action: 'edited', issue: item } }, core });
assert.deepStrictEqual(calls.map((c) => c[0]), ['removeLabel', 'comment']);
});
test('reopen: ignores strangers, reopens the author when it passes', async () => {
const issue = { number: 5, state: 'closed', user: { login: 'amy' }, labels: [] };
let calls = [];
const comment = { body: '/reopen', user: { login: 'bob', type: 'User' }, author_association: 'NONE' };
await bot.reopen({ github: fake(calls), context: { repo, payload: { issue, comment } }, core });
assert.deepStrictEqual(calls, []);
const live = { number: 5, title: '[Engine] x y', body: bugForm(), labels: [] };
comment.user.login = 'amy';
await bot.reopen({ github: fake(calls, { live }), context: { repo, payload: { issue, comment } }, core });
assert.deepStrictEqual(calls.map((c) => c[0]), ['update', 'removeLabel', 'comment']);
assert.strictEqual(calls[0][1], 'open');
calls = [];
live.title = 'no prefix';
await bot.reopen({ github: fake(calls, { live }), context: { repo, payload: { issue, comment } }, core });
assert.deepStrictEqual(calls.map((c) => c[0]), ['comment']);
assert.ok(calls[0][1].includes('Not reopened'));
});
test('sweep closes only old flagged items', async () => {
const calls = [];
const old = new Date(Date.now() - 6 * 86400000).toISOString();
const fresh = new Date(Date.now() - 1 * 86400000).toISOString();
const mk = (n, created) => ({ number: n, title: 'bad', body: '', labels: [{ name: 'needs-format' }], created_at: created });
const gh = fake(calls, { items: [mk(1, old), mk(2, fresh)] });
await bot.sweep({ github: gh, context: { repo }, core });
assert.deepStrictEqual(calls.map((c) => c[0]), ['comment', 'update']);
});
+164
View File
@@ -0,0 +1,164 @@
'use strict';
// GitHub-side handlers for intake.yml, reopen.yml and the 5-day sweep.
// They take the actions/github-script objects ({ github, context, core }) and never touch PR code:
// everything comes from the event payload and the REST API.
const check = require('./item-check');
const LABEL = 'needs-format';
const STALE_DAYS = 5;
const MAINTAINER = new Set(['OWNER', 'MEMBER', 'COLLABORATOR']);
// Items a maintainer closed on purpose stay closed.
const BLOCK_LABELS = new Set(['wontfix', 'invalid', 'duplicate', 'declined']);
function labelNames(item) {
return (item.labels || []).map((l) => (typeof l === 'string' ? l : l.name));
}
async function ensureLabel(github, context) {
const { owner, repo } = context.repo;
try {
await github.rest.issues.getLabel({ owner, repo, name: LABEL });
} catch (e) {
if (e.status !== 404) throw e;
await github.rest.issues.createLabel({
owner, repo, name: LABEL, color: 'fbca04',
description: 'Title or description does not follow CONTRIBUTING.md yet',
});
}
}
async function removeLabel(github, context, number) {
const { owner, repo } = context.repo;
try {
await github.rest.issues.removeLabel({ owner, repo, issue_number: number, name: LABEL });
} catch (e) {
if (e.status !== 404) throw e;
}
}
function itemFromPayload(context) {
const p = context.payload;
return p.pull_request || p.issue;
}
function isPullItem(item, context) {
return Boolean(context.payload.pull_request || item.pull_request);
}
// issues: opened/edited/reopened, pull_request_target: opened/edited/reopened
async function intake({ github, context, core }) {
const { owner, repo } = context.repo;
const item = itemFromPayload(context);
if (!item || (item.user && item.user.type === 'Bot')) return;
if (MAINTAINER.has(item.author_association) && context.payload.action === 'opened') {
// Maintainer items are not nagged on creation.
return;
}
const isPull = isPullItem(item, context);
const labels = labelNames(item);
const hasLabel = labels.includes(LABEL);
const res = check.checkItem({ isPull, title: item.title, body: item.body, labels, mode: 'intake' });
core.info('check ' + (isPull ? 'PR' : 'issue') + ' #' + item.number + ': ' + (res.ok ? 'ok' : res.missing.join(' | ')));
if (res.ok) {
if (hasLabel) {
await removeLabel(github, context, item.number);
await github.rest.issues.createComment({ owner, repo, issue_number: item.number, body: 'Thanks, this follows the format now.' });
}
return;
}
if (context.payload.action === 'edited' && !hasLabel) return; // edits only matter for items already flagged
if (hasLabel) return; // one note only
await ensureLabel(github, context);
await github.rest.issues.addLabels({ owner, repo, issue_number: item.number, labels: [LABEL] });
await github.rest.issues.createComment({ owner, repo, issue_number: item.number, body: check.intakeComment(isPull, res.missing) });
}
// issue_comment created, body starts with /reopen
async function reopen({ github, context, core }) {
const { owner, repo } = context.repo;
const comment = context.payload.comment;
const issue = context.payload.issue;
if (!/^\s*\/reopen\b/.test(comment.body || '')) return;
if (comment.user.type === 'Bot') return;
const isAuthor = comment.user.login === issue.user.login;
const isMaint = MAINTAINER.has(comment.author_association);
if (!isAuthor && !isMaint) {
core.info('ignored /reopen from ' + comment.user.login + ' (not the author or a maintainer)');
return;
}
const isPull = Boolean(issue.pull_request);
const say = (body) => github.rest.issues.createComment({ owner, repo, issue_number: issue.number, body });
const labels = labelNames(issue);
const wasOpen = issue.state === 'open';
if (labels.some((l) => BLOCK_LABELS.has(l.toLowerCase())) && !isMaint) {
await say('This one was closed on purpose (' + labels.filter((l) => BLOCK_LABELS.has(l.toLowerCase())).join(', ') + '). Please add your new evidence in a comment and a maintainer will decide.');
return;
}
if (wasOpen && !labels.includes(LABEL)) {
core.info('already open, nothing to do');
return;
}
// The bot reads the live item, not the possibly stale event copy.
const live = (await github.rest.issues.get({ owner, repo, issue_number: issue.number })).data;
const res = check.checkItem({ isPull, title: live.title, body: live.body, labels: labelNames(live), mode: 'reopen' });
core.info('reopen check #' + live.number + ': ' + (res.ok ? 'ok' : res.missing.join(' | ')));
if (!res.ok) {
await say(check.reopenMissingComment(isPull, res.missing));
return;
}
if (!wasOpen) {
try {
await github.rest.issues.update({ owner, repo, issue_number: live.number, state: 'open' });
} catch (e) {
await say('The check passed, but GitHub would not reopen it (' + (e.message || e.status) + '). ' + (isPull ? 'If the branch was deleted, restore it or open a new pull request and link this one.' : 'Please open a new issue and link this one.'));
return;
}
}
await removeLabel(github, context, live.number);
let extra = '';
if (isPull) {
try {
const pr = (await github.rest.pulls.get({ owner, repo, pull_number: live.number })).data;
if (pr.mergeable_state === 'dirty') extra = '\n\nIt has conflicts with main now: please rebase it.';
} catch (e) { /* best effort */ }
}
await say(check.reopenOkComment(isPull) + extra);
}
async function labelTime(github, owner, repo, number, fallback) {
const events = await github.paginate(github.rest.issues.listEvents, { owner, repo, issue_number: number, per_page: 100 });
let t = null;
for (const e of events) {
if (e.event === 'labeled' && e.label && e.label.name === LABEL) t = e.created_at;
}
return t || fallback;
}
// scheduled: close items that kept the needs-format label for more than 5 days
async function sweep({ github, context, core }) {
const { owner, repo } = context.repo;
const cutoff = Date.now() - STALE_DAYS * 86400 * 1000;
const items = await github.paginate(github.rest.issues.listForRepo, { owner, repo, state: 'open', labels: LABEL, per_page: 100 });
core.info(items.length + ' open items with ' + LABEL);
for (const item of items) {
const since = new Date(await labelTime(github, owner, repo, item.number, item.created_at)).getTime();
if (since > cutoff) continue;
const isPull = Boolean(item.pull_request);
// Re-check first: an edit that fixed it should have cleared the label already; do not close good items.
const res = check.checkItem({ isPull, title: item.title, body: item.body, labels: labelNames(item), mode: 'intake' });
if (res.ok) {
await removeLabel(github, context, item.number);
continue;
}
await github.rest.issues.createComment({ owner, repo, issue_number: item.number, body: check.staleComment(isPull) });
await github.rest.issues.update({ owner, repo, issue_number: item.number, state: 'closed', state_reason: 'not_planned' });
core.info('closed #' + item.number);
}
}
module.exports = { intake, reopen, sweep, LABEL, STALE_DAYS };
+114
View File
@@ -0,0 +1,114 @@
name: Docker images
# Builds the Strata container images of one release from its engine packages and pushes them to ghcr.io.
# Run by hand only (Actions tab -> Docker images -> Run workflow). Nothing starts it on its own.
#
# For each flavor the job downloads the engine package from the release and checks it against the signed update feed
# (docker/fetch-engines.py: the feed's Ed25519 signature, the archive's SHA-256, the manifest hash, the package signature,
# every file's hash), builds the image with docker/build.sh and, only when "push" is on, pushes
# ghcr.io/niko1221/strata:<version>-<flavor> and ghcr.io/niko1221/strata:latest-<flavor>
# The first run with "push" off builds everything without publishing it (a dry run).
on:
workflow_dispatch:
inputs:
release_tag:
description: 'Release tag whose engine packages go into the images (for example v0.2.1)'
required: true
type: string
flavors:
description: 'Flavors to build, comma separated (empty: every flavor the release has a package for)'
required: false
default: ''
type: string
push:
description: 'Push the images to ghcr.io (off: build only, nothing is published)'
required: true
default: false
type: boolean
latest:
description: 'Also move the latest-<flavor> tags (turn off for a pre-release or a test)'
required: true
default: true
type: boolean
source_ref:
description: 'Git ref of the Dockerfile and the Terminal sources (empty: the release tag)'
required: false
default: ''
type: string
permissions:
contents: read
env:
IMAGE: ghcr.io/niko1221/strata
jobs:
plan:
runs-on: ubuntu-24.04
outputs:
flavors: ${{ steps.list.outputs.flavors }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.source_ref || inputs.release_tag }}
- name: Which flavors does the release have a package for
id: list
env:
WANT: ${{ inputs.flavors }}
TAG: ${{ inputs.release_tag }}
run: |
set -eu
args="--release-tag $TAG --list --skip-missing"
[ -n "$WANT" ] && args="$args --flavors $WANT"
python3 docker/fetch-engines.py $args > list.tsv
cat list.tsv
flavors=$(awk -F'\t' '$3 != "SKIP" {printf "%s\"%s\"", (n++ ? "," : ""), $1}' list.tsv)
[ -n "$flavors" ] || { echo "the release has no engine package for any requested flavor"; exit 1; }
echo "flavors=[$flavors]" >> "$GITHUB_OUTPUT"
image:
needs: plan
runs-on: ubuntu-24.04
timeout-minutes: 120
permissions:
contents: read
packages: write
strategy:
fail-fast: false
max-parallel: 3
matrix:
flavor: ${{ fromJSON(needs.plan.outputs.flavors) }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.source_ref || inputs.release_tag }}
- name: Free disk space (the runner has about 14 GB; a package plus its image needs about 10 GB)
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL || true
docker system prune -af >/dev/null 2>&1 || true
df -h /
- name: Log in to ghcr.io
if: ${{ inputs.push }}
env:
REGISTRY_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REGISTRY_USER: ${{ github.actor }}
run: printf '%s' "$REGISTRY_TOKEN" | docker login ghcr.io -u "$REGISTRY_USER" --password-stdin
- name: Fetch, verify, build (and push when push is on)
env:
TAG: ${{ inputs.release_tag }}
FLAVOR: ${{ matrix.flavor }}
PUSH: ${{ inputs.push && '--push' || '' }}
LATEST: ${{ !inputs.latest && '--no-latest' || '' }}
run: |
docker/publish.sh --release-tag "$TAG" --flavors "$FLAVOR" --image "$IMAGE" $PUSH $LATEST
- name: Summary
if: ${{ always() }}
run: |
{
echo "### ${{ matrix.flavor }}"
echo "- release: ${{ inputs.release_tag }}, pushed: ${{ inputs.push }}"
echo "- image: \`$IMAGE:${INPUT_TAG#v}-${{ matrix.flavor }}\`"
} >> "$GITHUB_STEP_SUMMARY"
env:
INPUT_TAG: ${{ inputs.release_tag }}
+58
View File
@@ -0,0 +1,58 @@
name: Intake
# New issues and pull requests that do not follow CONTRIBUTING.md get the needs-format label and one friendly note.
# Items that keep the label for 5 days are closed by the daily sweep, with a note on how to /reopen.
#
# pull_request_target is used instead of pull_request on purpose: a pull request from a fork gets a read-only
# GITHUB_TOKEN under pull_request, so it could not be labelled or commented on. It is safe here because this
# workflow never checks out or runs the pull request's code. It checks out the default branch only (the bot
# scripts) and reads the title and description from the event payload and the REST API.
on:
issues:
types: [opened, edited]
pull_request_target:
types: [opened, edited]
schedule:
- cron: '17 4 * * *'
workflow_dispatch:
permissions:
contents: read
issues: write
pull-requests: write
concurrency:
group: intake-${{ github.event.issue.number || github.event.pull_request.number || 'sweep' }}
cancel-in-progress: false
jobs:
check:
if: github.event_name == 'issues' || github.event_name == 'pull_request_target'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.repository.default_branch }}
sparse-checkout: .github/scripts
persist-credentials: false
- uses: actions/github-script@v7
with:
script: |
const bot = require('./.github/scripts/reset-bot.js');
await bot.intake({ github, context, core });
sweep:
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.repository.default_branch }}
sparse-checkout: .github/scripts
persist-credentials: false
- uses: actions/github-script@v7
with:
script: |
const bot = require('./.github/scripts/reset-bot.js');
await bot.sweep({ github, context, core });
+38
View File
@@ -0,0 +1,38 @@
name: Reopen
# An author comments /reopen on their closed issue or pull request. The check runs and the item is reopened,
# or the bot replies with what is still missing. GitHub does not let authors reopen items a maintainer closed,
# so this workflow does it for them.
#
# issue_comment always runs the workflow file from the default branch with a write-capable token, which is what
# we need for fork authors too. It never checks out or executes the pull request's code: it reads the live
# title and description through the REST API. Only the author of the item or a maintainer can trigger it.
on:
issue_comment:
types: [created]
permissions:
contents: read
issues: write
pull-requests: write
concurrency:
group: reopen-${{ github.event.issue.number }}
cancel-in-progress: false
jobs:
reopen:
if: startsWith(github.event.comment.body, '/reopen') && github.event.comment.user.type != 'Bot'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.repository.default_branch }}
sparse-checkout: .github/scripts
persist-credentials: false
- uses: actions/github-script@v7
with:
script: |
const bot = require('./.github/scripts/reset-bot.js');
await bot.reopen({ github, context, core });
+21
View File
@@ -0,0 +1,21 @@
name: Scripts test
on:
pull_request:
paths:
- '.github/scripts/**'
- '.github/workflows/intake.yml'
- '.github/workflows/reopen.yml'
permissions:
contents: read
jobs:
node-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
- run: node --test .github/scripts/item-check.test.js
@@ -0,0 +1,65 @@
name: Minimal serving runtime
on:
push:
paths:
- .github/workflows/terminal-minimal-server.yml
- requirements.txt
- strata.py
- strata_cli/**
- strata_control/**
- serve/**
- tests/terminal/test_runtime.py
- tools/minimal_server_smoke.py
- tools/strata_tokenizer.py
pull_request:
paths:
- .github/workflows/terminal-minimal-server.yml
- requirements.txt
- strata.py
- strata_cli/**
- strata_control/**
- serve/**
- tests/terminal/test_runtime.py
- tools/minimal_server_smoke.py
- tools/strata_tokenizer.py
workflow_dispatch:
permissions:
contents: read
jobs:
fake-engine:
name: Linux / runtime requirements only
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v7
with:
persist-credentials: false
- uses: actions/setup-python@v7
with:
python-version: '3.11'
- name: Install only pinned runtime requirements
shell: bash
run: |
python -m venv "$RUNNER_TEMP/strata-runtime-venv"
"$RUNNER_TEMP/strata-runtime-venv/bin/python" -m pip install -r requirements.txt
- name: Check fake-engine serving without terminal packages
shell: bash
run: |
python - <<'PY'
import os
import subprocess
from pathlib import Path
report = Path(os.environ['RUNNER_TEMP']) / 'minimal-server-smoke.json'
python = str(Path(os.environ['RUNNER_TEMP']) / 'strata-runtime-venv' / 'bin' / 'python')
subprocess.run([python, 'tools/minimal_server_smoke.py',
'--report', str(report)], check=True)
print(report.read_text())
PY
- name: Run full server suite without terminal packages
shell: bash
run: |
"$RUNNER_TEMP/strata-runtime-venv/bin/python" -m unittest discover -s serve -p 'test_*.py'
@@ -0,0 +1,115 @@
name: Portable monitor client
on:
push:
paths:
- .github/workflows/terminal-portable-monitor.yml
- packaging/monitor_bundle.py
- strata.py
- strata_cli/**
- strata_control/**
- tools/minimal_cli_smoke.py
- tools/portable_monitor_smoke.py
- tools/portable_monitor_pty_smoke.py
- requirements-monitor.txt
- LICENSE
pull_request:
paths:
- .github/workflows/terminal-portable-monitor.yml
- packaging/monitor_bundle.py
- strata.py
- strata_cli/**
- strata_control/**
- tools/minimal_cli_smoke.py
- tools/portable_monitor_smoke.py
- tools/portable_monitor_pty_smoke.py
- requirements-monitor.txt
- LICENSE
workflow_dispatch:
permissions:
contents: read
jobs:
client:
name: ${{ matrix.os }} / Python ${{ matrix.python }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
python: ['3.10', '3.11']
steps:
- uses: actions/checkout@v7
with:
persist-credentials: false
- uses: actions/setup-python@v7
with:
python-version: ${{ matrix.python }}
- name: Check source CLI without optional packages
shell: bash
run: |
python - <<'PY'
import os
import subprocess
import sys
from pathlib import Path
report = Path(os.environ['RUNNER_TEMP']) / 'minimal-cli-smoke.json'
subprocess.run([sys.executable, 'tools/minimal_cli_smoke.py',
'--report', str(report)], check=True)
print(report.read_text())
PY
- name: Build and smoke test without site packages
shell: bash
run: |
python - <<'PY'
import os
import subprocess
import sys
from pathlib import Path
bundle = Path(os.environ['RUNNER_TEMP']) / 'strata-monitor.pyz'
report = bundle.with_name('monitor-stdlib-smoke.json')
subprocess.run([sys.executable, 'packaging/monitor_bundle.py',
'build', str(bundle)], check=True)
subprocess.run([sys.executable, 'tools/portable_monitor_smoke.py',
'--bundle', str(bundle), '--report', str(report)], check=True)
print(report.read_text())
PY
- name: Install optional Textual dependency
if: matrix.python == '3.11'
run: python -m pip install -r requirements-monitor.txt
- name: Smoke test packaged interactive Overview
if: matrix.python == '3.11'
shell: bash
run: |
python - <<'PY'
import os
import subprocess
import sys
from pathlib import Path
bundle = Path(os.environ['RUNNER_TEMP']) / 'strata-monitor.pyz'
report = bundle.with_name('monitor-interactive-smoke.json')
subprocess.run([sys.executable, 'tools/portable_monitor_smoke.py',
'--bundle', str(bundle), '--interactive',
'--report', str(report)], check=True)
print(report.read_text())
PY
- name: Check POSIX terminal restoration
if: matrix.os == 'ubuntu-latest' && matrix.python == '3.11'
shell: bash
run: |
python - <<'PY'
import os
import subprocess
import sys
from pathlib import Path
bundle = Path(os.environ['RUNNER_TEMP']) / 'strata-monitor.pyz'
report = bundle.with_name('monitor-pty-smoke.json')
subprocess.run([sys.executable, 'tools/portable_monitor_pty_smoke.py',
'--bundle', str(bundle), '--report', str(report)], check=True)
print(report.read_text())
PY
+28
View File
@@ -0,0 +1,28 @@
name: Title area
# Checks that a pull request or issue title starts with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
# It reads the title only: no checkout, no labels, no comments, no write access.
on:
pull_request:
types: [opened, edited, reopened, synchronize]
issues:
types: [opened, edited, reopened]
permissions: {}
jobs:
title:
runs-on: ubuntu-latest
steps:
- name: Check the title
env:
TITLE: ${{ github.event.pull_request.title || github.event.issue.title }}
run: |
case "$TITLE" in
"[Engine] "?*|"[GUI] "?*|"[TUI] "?*|"[Benchmark] "?*|"[Docs] "?*|"[Other] "?*)
echo "Title OK: $TITLE" ;;
*)
echo "::error::The title must start with its area and then a short description. Allowed areas: [Engine], [GUI], [TUI], [Benchmark], [Docs], [Other]. Example: \"[Engine] fix decode stop on long prompts\". Your title: $TITLE"
exit 1 ;;
esac
+24
View File
@@ -48,6 +48,7 @@ build-vision-cuda12.bat
# setup.py outputs (engine, downloads) and release builds
/engine/
/engine-sycl/
/engine-cuda12/
/build-release*/
/dist/
@@ -61,3 +62,26 @@ build-vision-cuda12.bat
/.ccache/
/.rocm-win/
# local Windows HIP packaging output
dist-hip/
# Terminal / Desktop split: local venvs, build output and archives never enter git
.terminal-venv/
.terminal-test-venv/
node_modules/
*.dmg
/release/artifacts/
/apps/host/dist/
/apps/desktop/src-tauri/target/
/apps/desktop/src-tauri/resources/node/*
!/apps/desktop/src-tauri/resources/node/.gitignore
# the Control component (bundled CPython + wheels + strata_control) is built by apps/desktop/scripts/stage-control.py
/apps/desktop/src-tauri/resources/control/*
!/apps/desktop/src-tauri/resources/control/.gitignore
# the pruned DeepSeek Harness (MIT) is built by apps/desktop/scripts/stage-harness.mjs
/apps/desktop/src-tauri/resources/harness/*
!/apps/desktop/src-tauri/resources/harness/.gitignore
# screenshots and checkpoints from the Mac rebuild live outside git (StrataSplit\evidence-archive)
/apps/host/docs/evidence/
/apps/host/docs/checkpoints/
+75 -13
View File
@@ -1,23 +1,85 @@
# AGENTS.md
Strata runs the Qwen3.8-Flash-Next mixture-of-experts model (and its Coder, Swift 1.5 and Unsloth variants) on a
normal PC: one NVIDIA or AMD graphics card plus system RAM, on Windows or Linux. It has a C++/CUDA/HIP engine
(`src/`, `include/`), a Python server with an OpenAI- and Anthropic-compatible API and a web app (`serve/`), and a
one-click installer (`setup.py`, started by `START-HERE.bat` / `setup.sh`).
Strata runs the Qwen3.8-Flash-Next mixture-of-experts model (and its Coder, Swift 1.5, Unsloth, Gyro and NVFP4
variants, and Qwen3.6-35B-A3B) on a normal PC: one NVIDIA or AMD graphics card plus system RAM on Windows or Linux,
or an Apple Silicon Mac. Version 0.2.1, engine 0.1.46. It has three parts:
- **The app** (`apps/desktop`, the Tauri shell; `apps/host`, the Node host and the web UI). It is for people who do
not want a terminal. The installer has the engine inside.
- **The Terminal** (`strata.py`, `strata_cli/`, `strata_control/`): the `strata` command. It sets up a model, serves
it and watches it. Agents use this part.
- **The engine and the server** (`src/`, `include/`, `metal/`, `sycl/` for the C++/CUDA/HIP/SYCL/Metal engine;
`serve/` for the Python server with an OpenAI- and Anthropic-compatible API and a web page). `setup.py`, started by
`START-HERE.bat` or `setup.sh`, is the older one-click installer from source.
## Installing Strata for a user
Follow **[docs/AI_SETUP.md](docs/AI_SETUP.md)**: check the PC, pick the model by RAM, run setup non-interactively,
start and verify the server, and connect the user's apps. Never expose the server beyond `127.0.0.1` without
`--api-key`. As an alternative to shell commands, Strata's MCP server ([docs/MCP_SERVER.md](docs/MCP_SERVER.md))
offers the same steps as tools.
Follow **[docs/AI_SETUP.md](docs/AI_SETUP.md)**. For an agent that has a shell, the short version is:
1. Install the Terminal. It needs no administrator rights.
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh # Linux, Mac
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex # Windows PowerShell
```
On Windows, open a new terminal afterwards so that `strata` is on the PATH.
2. Check what would happen. Nothing is changed:
```sh
strata launch --plan --model q2_0
```
3. Set up and serve without questions. It downloads the engine for the card and the model (tens of GB), starts the
server, prints the addresses and returns. The server keeps running.
```sh
strata launch --model q2_0 --yes --no-tui
```
Useful flags (all of them: `strata launch --help`): `--family`, `--model-path` (files the user already has),
`--data-dir`, `--context 32k|64k|128k`, `--vision off|on|gpu|cpu`, `--parallel N`, `--gpu 0,1`,
`--access local|lan|cloudflare|tailscale`, `--port`, `--api-key-env NAME`, `--new-key`, `--save-preset NAME`,
`--output json`. `strata launch --status` shows what is serving, `strata launch --stop` stops it, and
`strata launch --show-key` prints the API key.
4. Connect the user's apps: `strata connect` lists them, `strata connect <app> --check` sends one prompt and one tool
call the way that app does.
Never expose the server beyond `127.0.0.1` without an API key. `--access lan` makes a key for you. The old route (clone
the repository and run `START-HERE.bat --yes` or `./setup.sh --yes`) still works and is the fallback when the
Terminal cannot run. Strata's MCP server ([docs/MCP_SERVER.md](docs/MCP_SERVER.md)) offers the same steps as tools.
## Working on the code
- How the engine works, every measured number, the API and all settings: [docs/DETAILS.md](docs/DETAILS.md) and
the [paper](docs/paper/Strata-Paper.pdf).
- AMD (HIP) build and validation: [docs/AMD_HIP.md](docs/AMD_HIP.md); multi-GPU: [docs/MULTI_GPU.md](docs/MULTI_GPU.md).
- Setup's own tests run without a GPU or downloads: `python tools/test_setup_<name>.py` (for example
`tools/test_setup_amd.py`, `tools/test_setup_choices.py`).
- Keep the docs' style: plain words, measured numbers with what they were measured on, no claims without a
measurement.
- The Terminal: [docs/TERMINAL.md](docs/TERMINAL.md) and [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). The app,
the packages and the update feed: [packaging/README.md](packaging/README.md) and [packaging/UPDATES.md](packaging/UPDATES.md).
- AMD (HIP) build and validation: [docs/AMD_HIP.md](docs/AMD_HIP.md); multi-GPU: [docs/MULTI_GPU.md](docs/MULTI_GPU.md);
Mac (Metal): [docs/MAC.md](docs/MAC.md).
- Tests: the Terminal and Control tests are in `tests/terminal` (`python -m pytest tests/terminal`). The app's UI
tests run with `npm test` in `apps/host`. Setup's own tests run without a GPU or downloads:
`python tools/test_setup_<name>.py` (for example `tools/test_setup_amd.py`, `tools/test_setup_choices.py`).
- Keep the docs' style: plain words, short sentences, written for non-technical users first. Give measured numbers
with what they were measured on. Make no claim without a measurement.
## Contributing a change or report
The rules are in **[CONTRIBUTING.md](CONTRIBUTING.md)**. Read it before you open an issue or a pull request, and
follow it whether a person or an agent writes the text. The short version:
- Start every issue and pull request title with its area: `[Engine]`, `[GUI]`, `[TUI]`, `[Benchmark]`, `[Docs]` or
`[Other]`, for example `[Engine] fix …`. A check on GitHub fails a pull request whose title has no area.
- Search the open and closed issues and pull requests first, and add to a thread that already covers your point.
- Agree the scope in an issue before a large feature, a refactor or a change to a default.
- One problem per pull request. Lead with the problem and the result, then the approach, then the validation
(passed, failed and not run, kept apart) and the limits.
- A new feature is opt-in, and the default path stays byte-identical to the last release. Say how you checked.
- Build every backend a file touches (CUDA, HIP, SYCL) before asking for review.
- Change a default only where you measured it faster, and show the numbers with what they were measured on.
- Never submit measurements, logs or reproduction steps you did not produce. Mark a finding from reading code as
unconfirmed static analysis.
- A report from hardware the maintainers do not have is welcome. Follow
[docs/COMMUNITY_BENCHMARKS.md](docs/COMMUNITY_BENCHMARKS.md), compare against a same-day run of the build you are
testing, and say what you did not test.
- Open test requests and the hardware that is wanted are listed in [docs/TEST_REQUESTS.md](docs/TEST_REQUESTS.md).
+19
View File
@@ -0,0 +1,19 @@
@echo off
rem Benchmark this Strata install the way it is used and write a report you can share (docs/COMMUNITY_BENCHMARKS.md).
rem It starts the model with your settings (or uses the one already running), takes about 10 minutes and changes nothing.
rem Options: --clients 4 (several requests at once), --long (a 32K prompt), --compare <older results folder>.
rem This is the quick speed check. The standard benchmark (docs/BENCHMARK_PROTOCOL.md), whose results other people can compare with, is `strata bench run`.
setlocal
title Strata - benchmark
cd /d "%~dp0"
if not exist ".venv\Scripts\python.exe" (
echo Strata is not set up here yet: run START-HERE.bat first.
pause
exit /b 1
)
echo Quick speed check. For results other people can compare with (the standard benchmark, 20 to 30 minutes), run: .venv\Scripts\python.exe strata.py bench run
echo.
".venv\Scripts\python.exe" tools\strata_bench.py %*
set "RC=%ERRORLEVEL%"
pause
exit /b %RC%
+805 -29
View File
File diff suppressed because it is too large. Load diff
+122
View File
@@ -0,0 +1,122 @@
# Contributing to Strata
**AI assistance is welcome. The contributor is responsible for understanding the change and supplying real evidence.**
These rules apply to issues, pull requests, benchmark reports, and the agents helping prepare them.
You should be able to explain every changed line and answer review questions about it.
## Before starting
1. Search [issues](https://github.com/Niko1221/Strata/issues) and
[pull requests](https://github.com/Niko1221/Strata/pulls), including closed discussions. Add new evidence to an
existing thread when it covers your problem.
2. Read [AGENTS.md](AGENTS.md) and the documentation for the component you want to change.
3. Get maintainer agreement on scope before substantial features, refactors, architecture changes, or changes to
defaults. Describe the user problem, proposed behavior, and tradeoffs in an issue first. Agreement on scope
does not guarantee a merge. Focused fixes and benchmark reports can go straight to a PR.
4. Keep one problem per PR. Update that PR after feedback; do not open competing attempts for the same change.
Do not resubmit a declined proposal under a different title or through another agent. If new evidence changes
the situation, bring it to the original discussion and let a maintainer decide whether to reopen it.
## Report a problem people can investigate
Use the matching [issue form](https://github.com/Niko1221/Strata/issues/new/choose). A bug report should include:
- The smallest steps or command that reproduce the problem, and the expected and actual behavior.
- Strata version or commit, OS, GPU and VRAM, and relevant model, quantization, and configuration.
- The exact error and relevant logs, with credentials and private content removed.
- Whether you reproduced it yourself. Label a finding based only on reading code or a scanner as
**unconfirmed static analysis**, and explain the suspected path without presenting it as an observed failure.
Do not invent reproduction steps, errors, affected hardware, or severity. If you cannot reproduce something,
say what you inspected and what would verify the hypothesis. Feature requests should explain a concrete user
need and the current workaround. Questions should say what you tried and where you got stuck.
Example structure, with placeholders to replace with actual observations:
```text
Problem: <one observed behavior>
Environment: <commit, OS, GPU/VRAM, model/quant, relevant settings>
Reproduce: <exact command or short sequence>
Expected: <documented or intended behavior>
Actual: <observed behavior and exact error>
Evidence: <sanitized log attachment>
Reproduction status: <reproduced locally OR unconfirmed static analysis>
```
## Titles
Every issue and pull request title starts with its area in square brackets, then a short description: `[Engine]`, `[GUI]`, `[TUI]`,
`[Benchmark]`, `[Docs]` or `[Other]`. For example `[Engine] fix decode stop on long prompts` or `[GUI] keep the model list after a tab
switch`. The issue forms ask for the area and a check on the title fails when the prefix is missing.
## Prepare a focused pull request
Lead with the problem and resulting behavior. Explain why the approach is needed, the important limits, and
any compatibility effects. Link the issue and scope agreement when applicable. A file-by-file narration or
an AI-generated summary of the diff is not a substitute for this explanation.
- Keep unrelated formatting, cleanup, dependency changes, and speculative abstractions out of the PR.
- Add documentation where it changes what a user or developer needs to know. Avoid repeated summaries,
speculative roadmaps, and generated documentation with no actionable information.
- Preserve existing behavior outside the intended change. New features are opt-in; follow the default-path
and performance requirements in [AGENTS.md](AGENTS.md).
- Read the complete diff, including generated files, before submitting. Remove debugging output, secrets,
private paths, and accidental files.
- Prefer a small relevant regression check over tests that only restate the implementation. Documentation-only
changes need link, example, and consistency checks; they do not require GPU builds.
## Show the validation you actually performed
List exact commands, the revision and environment tested, and observed outcomes. Keep **passed**, **failed**,
and **not run** separate. Include relevant output or a log attachment. A command you suggest running is not
evidence that it passed; a CI run on a different revision is not validation of the submitted code.
Build every backend a code change touches, including CUDA, HIP, and SYCL where applicable, before requesting
review. For changes that need hardware execution, report that execution separately from compilation. A mock,
static check, or successful build does not establish hardware correctness. If you lack a required environment,
keep the PR as a draft, identify the missing checks, and ask for targeted help in the same thread. Do not claim
compatibility you have not tested.
Performance claims need before/after runs on the same hardware and workload, with every changed setting
identified. Preserve the individual runs, failures, and variability. Follow the
[community measurement guide](docs/COMMUNITY_BENCHMARKS.md); standard submissions will use the
[benchmark protocol](docs/BENCHMARK_PROTOCOL.md) once it is activated. One fast run does not justify changing a
default or claiming a speedup for other machines.
Example PR body; placeholders are not test results:
```text
Problem and result: <trigger; behavior before and after the change>
Related discussion: <issue URL and scope agreement, when required>
Approach: <why this fix; relevant compatibility effects>
Validation:
Passed: <exact command, revision/environment, observed result, log>
Failed: <exact command and failure, or none observed>
Not run: <check, reason, and what remains unverified>
Limits: <known limitations or remaining work>
```
## Rules for AI-assisted work
Use agents to help investigate, implement, and explain work that you can review. Before submitting, verify
their claims against the source, actual command output, or a cited primary reference.
Never submit fabricated measurements, invented test results, fake logs, or copied claims you have not checked.
Do not turn a repository scan into a batch of speculative issues or PRs. Consolidate related findings, establish
their effect, and submit the useful evidence. Do not present an agent's confidence or another model's approval
as validation. Keep review replies concise and address the requested change or evidence directly.
Maintainers may request a smaller change, return a PR to draft, or close a submission that lacks evidence,
duplicates existing work, or ignores agreed scope. A smaller, understandable contribution with honest limits
is easier to review and maintain.
## Share a benchmark
A results-only PR should contain the human report, raw measurements, sanitized configuration, and supporting
logs. Keep it separate from engine changes and include poor results and failed requests.
The [current community benchmark guide](docs/COMMUNITY_BENCHMARKS.md) remains available for measurements made
with today's tools. The [new protocol](docs/BENCHMARK_PROTOCOL.md) defines the draft standard and future approval
workflow. Its draft status must remain visible: existing reports are not automatically standard or approved.
+89 -101
View File
@@ -1,115 +1,103 @@
# syntax=docker/dockerfile:1
# Strata, built FROM SOURCE (NVIDIA CUDA 13). The engine is compiled here, packaged as an engine component package and then
# installed into the same runtime image as the ready-made ones (docker/Dockerfile): same Terminal, same entrypoint, same /data volume.
#
# Strata: Qwen3.8-Flash-Next on NVIDIA GPUs (RTX 30/40/50, 12+ GB VRAM; two or
# three cards can share one model, 8 GB each - docs/MULTI_GPU.md).
# Most people want the ready-made images instead (nothing compiled): docs/DOCKER.md.
#
# The engine is compiled during docker build, so the first container start only
# downloads the model (~70 GB) and starts the server. docker build has no GPU,
# so the CUDA architectures are fixed here instead of read from nvidia-smi: the
# engine is a fat binary with a cubin per listed arch, and the runtime picks the
# one matching your card. Narrow CUDA_ARCHITECTURES to your card for a faster
# build; a card outside the set needs a rebuild with its own arch.
# docker build -t strata:src . # RTX 20/30/40/50 (compiles for each, ~25 min)
# docker build -t strata:src --build-arg CUDA_ARCHITECTURES=86 . # one card only (faster); 86 = RTX 30, 89 = RTX 40, 120 = RTX 50
# docker run --rm --gpus all -p 127.0.0.1:8080:8080 --ulimit memlock=-1 --shm-size 2g --stop-timeout 120 \
# -v strata-data:/data -e MODEL=IQ3_XXS -e API_KEY=change-me-please-123 strata:src
#
# Build:
# docker build -t strata .
# docker build -t strata --build-arg CUDA_ARCHITECTURES=89 . # RTX 40 only
# docker build has no GPU, so the CUDA architectures are given here instead of read from nvidia-smi. The engine is a fat binary with
# a machine-code part per listed architecture (plus PTX for the newest, which later cards can JIT), and the runtime picks the one
# matching your card. Narrow CUDA_ARCHITECTURES to your card for a faster build. Native code for the CPU that builds the image is NOT
# used (STRATA_PORTABLE=ON), so the image runs on any x86-64 CPU with AVX2.
#
# Run (host needs an NVIDIA driver >= 580 and nvidia-container-toolkit):
# docker run --rm --gpus all \
# -p 8080:8080 \
# --ulimit memlock=-1 \
# -v strata-data:/data \
# -e MODEL=IQ2_XS \
# strata
#
# Setup choices are env vars, read by docker-entrypoint.sh: FAMILY, MODEL, CONTEXT,
# VISION (no | yes | cpu), KV (int8 | q4_0 | k8v4), GPU (one card) or GPUS ("0,2"
# or "all", with LAYER_SPLIT), LOW_RAM (auto | on | off), HOST, PORT, API_KEY.
#
# Only the model files, the prepared pack, the MTP layer and the install config
# live in the /data volume; the engine is part of the image. Strata loads 32-62 GB
# into RAM, so a capped container needs -e LOW_RAM=on: setup.py reads the RAM from
# /proc/meminfo, which here is the host's total, not the container's limit. Add an
# API key before exposing the port to a network: -e API_KEY=<secret>. Pass
# -e REINSTALL=1 to change the model settings later.
#
# --gpus all on a host with two usable cards: setup takes both (the layer split is
# its recommended default). Pin one card with -e GPU=0, or name them with
# -e GPUS=0,2. A volume set up for one card switches to the pair on its first start
# on a two-card host unless GPU or GPUS pins it. LOW_RAM=on runs on one card.
# Plain legacy-builder features only (no heredocs, no --mount). AMD (HIP) and Intel (SYCL) source builds: docs/AMD_HIP.md, docs/INTEL.md.
FROM nvidia/cuda:13.0.0-devel-ubuntu24.04
# STRATA_EXECV=1: setup.py replaces itself with the server, so the server is PID 1
# and docker stop's SIGTERM reaches it (see setup.start). Normal Linux starts, which
# don't set it, keep spawning the server as a child.
ENV DEBIAN_FRONTEND=noninteractive PYTHONUNBUFFERED=1 LANG=C.UTF-8 STRATA_EXECV=1
ARG CUDA_IMAGE=nvidia/cuda:13.0.0-devel-ubuntu24.04
ARG RUNTIME_BASE=ubuntu:24.04
# ---- 1. compile the engine and make its component package
FROM ${CUDA_IMAGE} AS engine-build
ENV DEBIAN_FRONTEND=noninteractive PYTHONUNBUFFERED=1 PATH=/venv/bin:$PATH
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential ca-certificates curl git libatomic1 libgomp1 \
python3 python3-pip python3-venv unzip \
build-essential ca-certificates git libgomp1 libatomic1 python3 python3-pip python3-venv unzip \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /opt/strata
WORKDIR /src
COPY . .
# RTX 20 (75), RTX 30 (86), RTX 40 (89), RTX 50 (120), plus 80 for A-series. CMakeLists
# refuses anything below 75. BUILD_VISION=0 skips the image encoder build.
ARG CUDA_ARCHITECTURES=75;80;86;89;120
ARG CUDA_ARCHITECTURES="75;80;86;89;120"
ARG BUILD_VISION=1
ARG ENGINE_VERSION=
RUN python3 -m venv /venv && /venv/bin/pip install --no-cache-dir -r requirements.txt -r requirements-build.txt
# llama.cpp at the pinned commit (ggml for the engine, mtmd for pictures), exactly as setup.py fetches it
RUN /venv/bin/python -c "import setup; print(setup.get_llama_cpp())"
RUN set -e; \
real=""; for a in $(echo "$CUDA_ARCHITECTURES" | tr ';,' ' '); do real="$real;${a}-real"; done; real=${real#;}; \
last=$(echo "$CUDA_ARCHITECTURES" | tr ';,' ' ' | awk '{print $NF}'); \
/venv/bin/cmake -G Ninja -S . -B build -DCMAKE_BUILD_TYPE=Release -DSTRATA_ENABLE_CUDA=ON -DSTRATA_BUILD_TESTS=OFF \
-DSTRATA_NATIVE_EXPERTS=ON -DSTRATA_PORTABLE=ON -DSTRATA_Q6K_EXPERTS=ON -DSTRATA_MMQ_KQUANTS=ON \
"-DCMAKE_CUDA_ARCHITECTURES=${real};${last}-virtual" -DSTRATA_GGML_DIR=/src/third_party/llama.cpp; \
/venv/bin/cmake --build build --target strata strata_kq -j "$(nproc)"; \
if [ "$BUILD_VISION" = "1" ]; then \
/venv/bin/cmake -G Ninja -S tools/vision -B build-vision -DCMAKE_BUILD_TYPE=Release -DLLAMA_DIR=/src/third_party/llama.cpp \
-DSTRATA_VISION_CUDA=ON -DSTRATA_PORTABLE=ON -DGGML_CUDA_NCCL=OFF "-DCMAKE_CUDA_ARCHITECTURES=${real};${last}-virtual"; \
/venv/bin/cmake --build build-vision --target strata-vision -j "$(nproc)"; \
fi
RUN set -e; \
mkdir -p /stage /engine; \
cp build/strata /stage/; \
[ -f build/libstrata_kq.so ] && cp build/libstrata_kq.so /stage/ || true; \
[ -f build-vision/bin/strata-vision ] && cp build-vision/bin/strata-vision /stage/ || true; \
ver="${ENGINE_VERSION:-$(sed -n 's/^[[:space:]]*project([^)]*VERSION[[:space:]]*\([0-9][0-9.]*\).*/\1/p' CMakeLists.txt | head -1)}"; \
echo "{\"version\": \"$ver\", \"source\": \"docker build\", \"archs\": \"$CUDA_ARCHITECTURES\", \"cuda\": \"13.0\", \"portable\": true}" > /stage/BUILD.json; \
python3 packaging/collect_libs.py --binary build/strata --search /usr/local/cuda/lib64 --out /stage --also 'libcublasLt.so*'; \
archs=""; for a in $(echo "$CUDA_ARCHITECTURES" | tr ';,' ' '); do archs="$archs --arch sm_$a"; done; \
last=$(echo "$CUDA_ARCHITECTURES" | tr ';,' ' ' | awk '{print $NF}'); \
python3 packaging/package_engine.py --src /stage --backend cuda --version "$ver" --out /engine $archs --arch "compute_$last" \
--min-driver nvidia=580.65 --variant cuda13 --platform linux-x86_64 --build-id "docker-source-${ver}-cuda13.0" --include SYSTEM-LIBS.txt --include 'libstrata_kq*' \
--include strata-vision; \
for d in /engine/*/; do python3 packaging/package_engine.py --verify "$d"; python3 packaging/binary_deps.py "$d"; done
RUN python3 -m venv .venv \
&& .venv/bin/pip install --no-cache-dir --upgrade pip \
&& .venv/bin/pip install --no-cache-dir -r requirements.txt \
&& chmod +x setup.sh docker-entrypoint.sh
# llama.cpp at the pinned commit, then the engine and the image encoder, built
# exactly the way setup.py builds them. BUILD.json is what setup.py reads to
# decide whether an engine is current: source=local with a matching src hash
# means the first start reuses it instead of recompiling.
RUN .venv/bin/python - <<'PYEOF'
import json, os, pathlib, shutil
import setup
llama = setup.get_llama_cpp()
nvcc, _ = setup.find_nvcc()
arch = os.environ.get("CUDA_ARCHITECTURES", "75;80;86;89;120").strip().strip('"').replace(",", ";")
vision = "gpu" if os.environ.get("BUILD_VISION", "1") == "1" else "none"
setup.cmake_build(setup.ROOT, setup.ROOT / "build", "strata",
["-DSTRATA_ENABLE_CUDA=ON", "-DSTRATA_BUILD_TESTS=OFF",
f"-DCMAKE_CUDA_ARCHITECTURES={arch}", f"-DCMAKE_CUDA_COMPILER={nvcc}",
f"-DSTRATA_GGML_DIR={llama}"], None, "build-strata.bat")
if vision != "none":
setup.cmake_build(setup.ROOT / "tools" / "vision", setup.ROOT / "build-vision", "strata-vision",
[f"-DLLAMA_DIR={llama}", "-DSTRATA_VISION_CUDA=ON",
f"-DCMAKE_CUDA_ARCHITECTURES={arch}", f"-DCMAKE_CUDA_COMPILER={nvcc}"], None, "build-vision.bat")
eng = setup.ROOT / "engine"
eng.mkdir(exist_ok=True)
shutil.copy2(setup.ROOT / "build" / setup.EXE, eng / setup.EXE)
if vision != "none":
shutil.copy2(setup.ROOT / "build-vision" / "bin" / setup.VEXE, eng / setup.VEXE)
bindir = pathlib.Path(nvcc).parent
meta = {"source": "local", "version": setup.source_version(),
"archs": [int(a.split("-")[0]) for a in arch.split(";") if a.split("-")[0].isdigit()], "vision": vision,
"cuda_dirs": [str(d) for d in (bindir, bindir / "x64", bindir.parent / "lib64") if d.is_dir()],
"src": setup.source_hash(setup.ENGINE_SOURCES),
"vision_src": setup.source_hash(setup.VISION_SOURCES) if vision != "none" else None}
(eng / "BUILD.json").write_text(json.dumps(meta, indent=1))
PYEOF
# the cmake trees are build-time only; the engine itself is what the container needs
RUN rm -rf build build-vision
# ---- 2. the Terminal (the release archive, built here)
FROM ${RUNTIME_BASE} AS terminal
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends python3 ca-certificates && rm -rf /var/lib/apt/lists/*
COPY . /src/
RUN python3 /src/packaging/package_terminal.py --out /out --cache /cache \
&& mkdir -p /opt/strata \
&& tar -xzf /out/strata-terminal-*-linux-x86_64.tar.gz -C /opt/strata \
&& mv /opt/strata/strata-terminal-* /opt/strata/terminal \
&& /opt/strata/terminal/bin/strata --version
# llama.cpp at the commit setup.py pins: fetched here, so the first start does not depend on GitHub. Only gguf-py (the model preparation
# tools) and the file setup.py looks for (ggml/CMakeLists.txt) are kept: 170 MB become a few
RUN cd /opt/strata/terminal && STRATA_CONTROL_INSTALL=1 STRATA_INSTALL_DIR=/opt/strata/seed PYTHONNOUSERSITE=1 \
/opt/strata/terminal/python/bin/python3 -c "import setup; print(setup.get_llama_cpp())" \
&& rm -rf /opt/strata/seed/third_party/*.zip* \
&& cd /opt/strata/seed/third_party/llama.cpp \
&& find . -mindepth 1 -maxdepth 1 ! -name gguf-py ! -name ggml ! -name 'LICENSE*' -exec rm -rf {} + \
&& find ggml -mindepth 1 -depth ! -type d ! -name CMakeLists.txt -delete && find ggml -mindepth 1 -type d -empty -delete \
&& du -sh /opt/strata/seed
# ---- 3. the runtime image (the same as docker/Dockerfile's, FLAVOR=cuda)
FROM ${RUNTIME_BASE}
ARG STRATA_VERSION=dev
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends tini ca-certificates libgomp1 libatomic1 libnuma1 zlib1g \
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/*
COPY --from=terminal /opt/strata/terminal /opt/strata/terminal
COPY --from=terminal /opt/strata/seed /opt/strata/seed
COPY --from=engine-build /engine/ /opt/strata/components/engine/
COPY docker/entrypoint.sh docker/healthcheck.sh /opt/strata/
COPY docker/strata-wrapper.sh /usr/local/bin/strata
RUN chmod 0755 /opt/strata/entrypoint.sh /opt/strata/healthcheck.sh /usr/local/bin/strata
ENV STRATA_COMPONENTS_DIR=/opt/strata/components STRATA_INSTALL_DIR=/data/install XDG_STATE_HOME=/data/state \
XDG_CONFIG_HOME=/data/config HOME=/data/home PORT=8080 LANG=C.UTF-8 PYTHONUNBUFFERED=1 PYTHONNOUSERSITE=1 STRATA_IN_DOCKER=1 \
NVIDIA_VISIBLE_DEVICES=all NVIDIA_DRIVER_CAPABILITIES=compute,utility
VOLUME ["/data"]
EXPOSE 8080
# /health is answered before the API key gate, so it works with or without one.
# The port only opens after the model loads (1-3 minutes, longer on a first run),
# so the start period is generous: a too short one marks a still-loading container
# unhealthy and a restart policy would kill it mid-download.
HEALTHCHECK --interval=30s --timeout=5s --start-period=600s --retries=3 \
CMD curl -fs "http://127.0.0.1:${PORT:-8080}/health" || exit 1
ENTRYPOINT ["./docker-entrypoint.sh"]
LABEL org.opencontainers.image.title="Strata" com.strata.flavor="cuda" com.strata.build="from-source" org.opencontainers.image.version="${STRATA_VERSION}"
HEALTHCHECK --interval=30s --timeout=8s --start-period=2h --retries=3 CMD ["/opt/strata/healthcheck.sh"]
STOPSIGNAL SIGTERM
ENTRYPOINT ["/usr/bin/tini", "--", "/opt/strata/entrypoint.sh"]
CMD ["run"]
+343 -161
View File
@@ -1,222 +1,404 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
</picture>
</p>
[English](README.md) · [简体中文](README.zh-CN.md) · [日本語](README.ja.md) · **Deutsch** · [Français](README.fr.md) · [Español](README.es.md) · [Português](README.pt-BR.md)
<p align="center"><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <b>Deutsch</b> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a></p>
<p align="center"><b>Ein KI-Modell mit 125 Milliarden Parametern auf deinem eigenen Gaming-PC</b><br>
NVIDIA- oder AMD-Grafikkarte (ab 12 GB) · Windows oder Linux · kostenlos und Open Source</p>
<h3 align="center">Ein Spitzenmodell auf deinem eigenen Gaming-PC</h3>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="Ein Voxel-Pagodengarten, den das Modell in Strata geschrieben hat, läuft im Browser"></a><br>
<p align="center">Unterstützt NVIDIA, AMD, Intel · Windows, Linux, Mac · kostenlos und Open Source</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>Ein Voxel-Pagodengarten aus einem einzigen Prompt, auf einer RTX 5070 mit Strata (IQ3_S, 128K Kontext) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">ganzes Video (49 s)</a></sub></p>
Strata lässt **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** auf einem normalen PC laufen.
Das ist ein großes, kluges KI-Modell, das sonst meist einen Server braucht. Es chattet, schreibt Code, versteht
Bilder und arbeitet mit deinen Apps und Coding-Agents zusammen. Nichts verlässt deinen PC.
Strata lässt **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** laufen, ein Modell mit 125
Milliarden Parametern, für das man sonst einen Server braucht, auf einem normalen PC. Es chattet, schreibt Code, liest
Bilder und arbeitet mit deinen Apps und Coding-Agents zusammen. Deine Chats und Dateien bleiben auf deinem PC.
## Wie schnell ist es?
**Neu hier?** Sieh dir an, [was du brauchst](#was-du-brauchst), und [installiere dann die App](#option-a-die-strata-app).
Wir haben es auf zwei ganz normalen Gaming-PCs gemessen. Ein Token ist etwa ¾ eines Wortes.
- **Läuft auf einem Gaming-PC.** 32 GB Arbeitsspeicher (RAM) und Grafikspeicher zusammen reichen für den Anfang.
- **Passt zu deinen Werkzeugen.** Strata spricht die OpenAI- und die Anthropic-API, deshalb verbinden sich Claude Code, Codex CLI, OpenCode und andere Agents damit.
- **Liest Bilder.** Das ist standardmäßig eingeschaltet.
- **Kostenlos.** MIT-Lizenz, kein Konto, keine Cloud.
- **Privat.** Keine Telemetrie. Was Strata herunterlädt, wann es das Internet nutzt und wie es Downloads prüft, steht in [docs/SAFETY.md](docs/SAFETY.md).
- **Schreibt Antworten:** wie schnell die Antwort in einem kurzen Chat erscheint. 60 Tokens pro Sekunde sind
schneller, als du lesen kannst.
- **Liest deinen Prompt:** wie schnell es aufnimmt, was du schickst (hier ein Dokument, Code oder Chatverlauf mit
32K Tokens).
<table>
<tr><th>NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM</th><th>AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM</th></tr>
<tr><td>
| Größe | Schreibt Antworten | Liest deinen Prompt |
| --- | ---: | ---: |
| **Q2_0** | 94 Tokens/s | 2,650 Tokens/s |
| **IQ2_XS** | 79 Tokens/s | 2,090 Tokens/s |
| **IQ3_XXS** | 62 Tokens/s | 1,750 Tokens/s |
| **IQ3_S** | 53 Tokens/s | 1,620 Tokens/s |
| **Coder** | 55 Tokens/s | 2,180 Tokens/s |
</td><td>
| Größe | Schreibt Antworten | Liest deinen Prompt |
| --- | ---: | ---: |
| **Q2_0** | 60 Tokens/s | 1,160 Tokens/s |
| **IQ2_XS** | 52 Tokens/s | 1,110 Tokens/s |
| **Coder** | 44 Tokens/s | 1,420 Tokens/s |
</td></tr>
</table>
NVIDIA: Q2_0 mit Engine 0.1.36, die anderen Zeilen mit 0.1.26 (4K Antworten, 32K Prompts). Die vollständigen
Tabellen stehen in [DETAILS.md](docs/DETAILS.md#speed-measured). Eine Karte mit mehr VRAM ist schneller: Eine
RTX 3090 (24 GB) sollte etwa 100-140 Tokens pro Sekunde schreiben. Lange Chats und andere Karten:
[Tempo jedes Modells](docs/MODELS.md#how-fast-is-each-size), [Ergebnisse aus der Community](docs/COMMUNITY_BENCHMARKS.md).
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
<sub>Strata ist kostenlos. Wenn es auf deinem PC gut läuft, hält ein Kaffee die Arbeit daran am Laufen.</sub></p>
Version 0.2.1 · Engine 0.1.46
## Was du brauchst
| | |
**Du brauchst mindestens 32 GB Arbeitsspeicher (RAM) und Grafikspeicher (VRAM) zusammen.** Bei einem Mac oder einem Ryzen AI Max
ist das der gemeinsame Speicher. Mit mehr Speicher läuft ein größeres oder ein schnelleres Modell.
- Ein PC mit 32 GB und einer Grafikkarte mit 12-16 GB führt den Coder aus.
- Mit 64 GB RAM laufen alle Größen.
| Dein Computer | Details |
| --- | --- |
| **Grafikkarte** | **NVIDIA** GeForce RTX 20, 30, 40 oder 50 Serie, oder **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 oder RX 6800 / 6900 Serie. Sie braucht **12 GB VRAM oder mehr**. |
| **RAM** | 32 GB oder mehr. Dein RAM entscheidet, [welches Modell](#welches-modell-soll-ich-nehmen) passt. Mit 64 GB laufen alle Größen. |
| **Festplatte** | Etwa 80 GB frei. Nimm wenn möglich eine SSD: Der erste Start geht dann viel schneller. |
| **System** | Windows 10 / 11 oder Linux und ein aktueller Grafiktreiber von NVIDIA oder AMD. |
| **NVIDIA** GeForce RTX 20, 30, 40, 50 Serie (12 GB oder mehr) | Installiere die App oder das Terminal. |
| **AMD** Radeon RX 6800 und neuer, Radeon AI PRO R9700 (12 GB oder mehr) | [AMD-Karten](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md), [Intel-Engine](docs/INTEL.md) |
| **Apple-Silicon-Mac** (M1 oder neuer, macOS 15 oder neuer) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max (Strix Halo)** und andere Systeme mit gemeinsamem Speicher | [Strix Halo](docs/STRIX_HALO.md) |
| Ältere Karten und Prozessoren | [Ältere GPUs](docs/OLDER_GPUS.md), [Ältere CPUs](docs/INSTALL.md#older-cpus-experimental) |
Alles andere richtet der Installer ein. Zwei oder drei Karten können sich das Modell teilen ([Multi-GPU](docs/MULTI_GPU.md)).
Außerdem nötig:
Experimentell, von Community-Mitgliedern auf ihren eigenen Rechnern geschrieben und getestet:
- etwa 80 GB freier Speicherplatz (mit einer SSD startet es beim ersten Mal viel schneller)
- Windows 10 / 11, Linux oder macOS
- ein aktueller Grafiktreiber
- **Ältere Grafikkarten** (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT): [Ältere GPUs](docs/OLDER_GPUS.md).
- **Intel Arc**, unter Linux aus dem Quellcode gebaut: [Intel Arc](docs/INTEL_ARC.md).
- **AMD Ryzen AI Max (Strix Halo)**, unter Linux aus dem Quellcode gebaut: [Strix Halo](docs/STRIX_HALO.md).
- **Ältere Prozessoren ohne AVX2**: Sie funktionieren, aber langsam. [Ältere CPUs](docs/INSTALL.md#older-cpus-experimental).
Zwei oder drei Karten desselben Herstellers können sich ein Modell teilen ([Multi-GPU](docs/MULTI_GPU.md)). Strata teilt ein
Modell nie auf NVIDIA- und AMD-Karten auf. Die vollständige Liste: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
Die vollständige Liste: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
## Installation
## Installieren
Such dir eine Option aus. Alle starten dieselbe Engine und nutzen dieselben Modelle.
### Lass deine KI es einrichten
### Option A: die Strata-App
Nutzt du einen KI-Coding-Assistenten (Claude Code, Cursor, Codex, GitHub Copilot, ...)? Füge dort diesen Text ein:
Der einfachste Weg. Die App bringt die Engine mit: installieren, ein Modell wählen, chatten. Lade die Datei für dein
System und deine Grafikkarte von der [Releases-Seite](https://github.com/Niko1221/Strata/releases/latest) herunter. Du bist
unsicher, welche du brauchst? Lies den Absatz unter der Tabelle.
| System | Deine Grafikkarte | Datei zum Herunterladen |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX, Treiber 580 oder neuer | [`Strata-0.2.1-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda13.exe) |
| | NVIDIA, ältere Karte oder Treiber älter als 580 | [`Strata-0.2.1-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda12.exe) |
| | AMD Radeon RX 9000 Serie oder Radeon AI PRO R9700 | [`Strata-0.2.1-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip.exe) |
| | AMD Radeon RX 7000 Serie oder Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX, Treiber 580 oder neuer | [`Strata-0.2.1-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda13.AppImage) |
| | NVIDIA, ältere Karte oder Treiber älter als 580 | [`Strata-0.2.1-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon RX 9000 Serie oder Radeon AI PRO R9700 | [`Strata-0.2.1-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx9070.AppImage) |
| | AMD Radeon RX 7000 Serie | [`Strata-0.2.1-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-strixhalo.AppImage) |
| **Mac** (Apple Silicon, macOS 15 oder neuer) | eingebaut | [`Strata-0.2.1-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-mac.dmg) |
**Wenn du unsicher bist:** Bei NVIDIA nimmst du `cuda13`, wenn dein Treiber Version 580 oder neuer ist, sonst `cuda12`.
`cuda12` ist für ältere Karten und ältere Treiber. Die Dateien `hip` und `amd` sind für AMD-Karten. Für andere AMD-Karten
(RX 6000) gibt es noch keine fertige App: siehe [AMD-Karten](docs/AMD_HIP.md).
Danach:
- **Windows:** Starte die `.exe`. Sie wird für deinen Benutzer installiert und braucht keine Administratorrechte. Wenn
Windows "Der Computer wurde durch Windows geschützt" meldet, wähle **Weitere Informationen** und dann **Trotzdem
ausführen** (der Installer ist noch nicht signiert).
- **Linux:** Mach die Datei ausführbar (`chmod +x`) und starte sie. Sie braucht FUSE 2: `sudo apt install libfuse2`
(Ubuntu 22.04) oder `libfuse2t64` (Ubuntu 24.04 und neuer).
- **Mac:** Öffne die `.dmg` und zieh Strata in den Ordner "Programme".
Auf dem ersten Bildschirm liest die App deinen PC aus und empfiehlt ein passendes Modell. Drück auf Installieren. Das Modell
ist ein großer Download (etwa 40-70 GB) und wird nach einer Unterbrechung dort fortgesetzt, wo er aufgehört hat. Wenn du
das Fenster schließt, beenden sich Strata und die Engine.
### Option B: das Terminal (eine Zeile)
Das Terminal ist der Befehl `strata`, gedacht für Skripte, Server und KI-Agents.
Linux und Mac:
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
```
Windows (PowerShell):
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
Das Skript prüft deinen PC, lädt das Terminal herunter, kontrolliert die SHA-256-Prüfsumme und entpackt es in deinen
Home-Ordner. Es braucht keine Administratorrechte. Öffne unter Windows danach ein neues Terminalfenster. Dann startest du
Strata:
```sh
strata launch
```
Es stellt dir eine Frage nach der anderen: Modell, Kontext, Bilder, wer sich verbinden darf. Danach lädt es die Engine für
deine Grafikkarte herunter (signiert und vor der Nutzung probeweise gestartet), dazu das Modell, und beginnt zu laufen.
Für Skripte und KI-Agents gibt es Flags:
```sh
strata launch --model q2_0 --yes --no-tui
```
Wenn du das Installationsskript vor dem Ausführen lesen willst, lade es zuerst mit `curl -fsSL <address> -o install.sh`
herunter, lies es und führe dann `sh install.sh` aus. Alle Bildschirme und Flags: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md).
### Option C: deine KI richtet es ein
Füge das in Claude Code, Cursor, Codex, GitHub Copilot oder einen anderen Coding-Assistenten ein:
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
Er prüft deine Grafikkarte, deinen RAM und deine Festplatte und wählt das passende Modell. Dann installiert und
startet er es und sagt dir, wie du deine Apps verbindest. KI-Tools können Strata auch über seinen
Der Assistent prüft deine Grafikkarte, deinen Arbeitsspeicher und den Speicherplatz, wählt das passende Modell, installiert
und startet es und sagt dir, wie du deine Apps verbindest. KI-Werkzeuge können Strata auch über den
[MCP-Server](docs/MCP_SERVER.md) installieren, starten und stoppen.
### Oder mach es selbst
Du willst es lieber selbst bauen? Siehe [Aus dem Quellcode bauen](#aus-dem-quellcode-bauen).
[Lade Strata herunter](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) und entpacke es (oder nutze
`git clone`). **Windows:** Doppelklick auf **`START-HERE.bat`**. **Linux:** Führe im Strata-Ordner **`./setup.sh`** aus.
## Modelle
Die Schritte sind für NVIDIA und AMD gleich. Der Installer erkennt deine Karte und richtet die passende Engine
dafür ein. Er stellt dir ein paar Fragen:
Die App und `strata launch` empfehlen ein Modell passend zu deinem Speicher. Es ist ein einziges Modell in mehreren
Größen, mehr oder weniger stark komprimiert. Kleinere Größen sind schneller. Größere Größen sind etwas klüger.
- welches Modell und welche Größe,
- wie viel Kontext (wie viel Text sich das Modell merkt),
- ob es Bilder lesen soll.
### Wenig Speicher? Probier Gyro
Drück jedes Mal Enter, um die empfohlene Antwort zu nehmen. Dann lädt er das Modell herunter (etwa 70 GB) und
startet es. Bricht der Download ab, starte ihn einfach neu: Er macht dort weiter, wo er aufgehört hat. Dein Browser
öffnet die Strata-App unter `http://127.0.0.1:8080`.
[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) (von agentionai) ist die kleinste Möglichkeit, dieses Modell auszuführen.
> **Während das Modell startet, kann dein PC 1-3 Minuten langsam sein oder nicht reagieren** (beim ersten Mal am
> längsten). Strata lädt 35-55 GB in deinen RAM und reserviert einen Teil davon für die Grafikkarte. Das ist normal.
> Warte und schließ das Fenster nicht. Im Fenster siehst du, was Strata gerade tut.
| Größe | Benötigter Speicher (RAM + VRAM) | Download | Was es ist |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27,6 GB | 58,5 GB | Für PCs, auf denen nichts anderes läuft. Die Qualität ist geringer als bei IQ2_XS. |
| **Gyro-M** | 35 GB | 92,0 GB | Ungefähr die Qualität von IQ3_XXS bei etwa 20 % weniger GPU-Speicher. Es schreibt Antworten auf einer 12-GB-Karte und auf einer R9700 17-21 % langsamer als Q2_0. |
**Beim nächsten Mal** startest du wieder `START-HERE.bat` (oder `./setup.sh`). Es startet sofort und lädt nichts
doppelt herunter. Schließ das Fenster, um das Modell zu stoppen. `UPDATE.bat` (`./update.sh`) aktualisiert Strata,
ohne es zu starten. Updates, Docker, mehrere Karten, wo die Dateien landen und alle Optionen:
[docs/INSTALL.md](docs/INSTALL.md).
Gyro läuft auf NVIDIA- und AMD-Karten (bei AMD experimentell), nicht auf Intel Arc und nicht auf dem Mac. Details und Messwerte:
[docs/GYRO.md](docs/GYRO.md). Starte es mit `strata launch --model gyro-s`.
## Welches Modell soll ich nehmen?
### Die anderen Größen
Der Installer empfiehlt eins passend zu deinem RAM. Dasselbe Modell gibt es in mehreren Größen, mehr oder weniger
stark komprimiert. Kleinere Größen sind schneller. Größere sind etwas klüger.
| Dein RAM | Nimm | Warum |
| Dein Speicher (RAM + VRAM) | Nimm | Hinweise |
| --- | --- | --- |
| **32 GB** | **Coder** | passt in 32 GB und ist für Code gemacht (mit einer 24-GB-Karte laufen auch Q2_0 und IQ2_XS) |
| **48 GB** | **IQ2_XS** (oder Q2_0, das schnellste) | die größeren Größen passen nicht |
| **64 GB** | **IQ2_XS** (empfohlen) oder IQ3_XXS / IQ3_S | alle Größen passen; IQ3_S ist das beste und das langsamste |
| **96 GB oder mehr** | **IQ3_S** oder Unsloths UD-IQ4_XS (~4 Bit) | Platz für die größten Größen, auch wenn alles andere offen ist |
| **32 GB** | **Gyro-S**, oder den **[Coder](docs/MODELS.md#coder)** für Code | Der Coder braucht 32 GB RAM und behält die Hälfte der Experten, deshalb ist er außerhalb von Code schwächer |
| **40 GB** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS (39,2 GB), oder **Gyro-M** (35 GB) | Swift denkt kürzer nach, bevor es antwortet. Gyro-M kommt der Qualität von IQ3_XXS nahe, schreibt Antworten aber 17-21 % langsamer als Q2_0 |
| **48 GB** | **IQ3_XXS** (47,0 GB) | Langsamer, besser |
| **64 GB** | **IQ3_S** (54,8 GB), das Original oder Swift 1.5 | Am langsamsten, am besten. Es erreicht in den veröffentlichten Tests das volle Modell |
| **96 GB oder mehr** | **UD-IQ4_XS** von Unsloth (etwa 72 GB RAM) bis **Q8_0** (etwa 136 GB), 4 bis 8 Bit | Näher am vollen Modell. Bei weniger RAM liest Strata einen Teil von der SSD, das ist viel langsamer |
- **[Coder](docs/MODELS.md#coder):** eine Version für Code, bei der die Hälfte der Experten entfernt wurde. Sie
erreicht 91 % des SWE-bench-Verified-Werts des vollen Modells (von ihren Autoren gemessen) und passt in 32 GB RAM.
Außerhalb von Code ist sie schwächer, auch bei chinesischem und anderem CJK-Text (#438). Nimm dafür Q2_0, IQ2_XS
oder IQ3_S, die alle Experten behalten.
- **[Swift 1.5](docs/MODELS.md#swift-15):** ein Fine-Tune, das viel kürzer nachdenkt, bevor es antwortet. Du
bekommst die Antwort früher, bei etwa gleicher Qualität.
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** die ~4-Bit-Version von Unsloth, in der Qualität
zwischen IQ3_S und UD-Q4_K_XL. 94 GB Download. Mit weniger als ~80 GB RAM liest Strata einen Teil davon während
der Antwort von der SSD, dort ist es also langsamer (eine NVMe-SSD hilft).
- **[Unsloth UD-Q4_K_XL](docs/MODELS.md#unsloth-ud-q4_k_xl-experimental)** (experimentell): am nächsten am vollen
Modell. Aber Strata liest das meiste davon während der Antwort von der SSD, deshalb schreibt es auf einem PC mit
64 GB nur 7-8.5 Tokens/s.
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** Das richtest du von Hand
ein. Es steht nicht im Menü des Installers.
Auch mit weniger RAM, als eine Zeile nennt, kann es funktionieren: Strata wechselt von selbst in seinen
[Modus für wenig RAM](docs/MODELS.md#a-big-graphics-card-and-little-ram). Rechne mit weniger Tempo, mit einer kleinen
Grafikkarte mit deutlich weniger.
Größen, Downloads und was wohin passt: [docs/MODELS.md](docs/MODELS.md). Um später ein weiteres Modell
hinzuzufügen, starte `SETUP.bat` (Linux: `./setup.sh --setup`).
Weitere Auswahl:
## So benutzt du es
- **[Swift 1.5](docs/MODELS.md#swift-15)** ist eine Feinabstimmung, die kürzer nachdenkt, bevor sie antwortet, bei etwa gleicher Qualität und gleichem RAM-Bedarf.
- **[NVFP4](docs/NVFP4.md)** ist NVIDIAs 4-Bit-Version für NVIDIA-Karten. Sie braucht etwa 80 GB RAM, um alle Experten im RAM zu halten.
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** ist ein kleineres, anderes Modell, das Bilder liest und mehrere Anfragen gleichzeitig beantworten kann.
- **[OrcaRouters Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** richtest du von Hand ein.
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="Der Monitor-Tab der Strata-App neben einem Coding-Agent"><br>
<sub>Der <b>Monitor</b> der Strata-App (links), während ein Coding-Agent den Pagodengarten aus dem Video schreibt (rechts)</sub></p>
Alle Größen, Downloads und was wo läuft: [docs/MODELS.md](docs/MODELS.md).
- **Im Browser:** Öffne `http://127.0.0.1:8080`. Dort gibt es **Chat**, einen Live-**Monitor** für das Modell und
deine GPU/CPU/RAM und **About** mit den Einstellungen und Adressen.
- **Deine Apps und Coding-Agents:** Füge einen „OpenAI-kompatiblen“ Anbieter mit der Basis-URL
**`http://127.0.0.1:8080/v1`** hinzu. Jeder API-Key und jeder Modellname funktioniert.
- Apps, die die API von Anthropic nutzen: `http://127.0.0.1:8080/v1/messages` (Claude Code:
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`).
- Codex CLI und andere Apps, die die OpenAI Responses API nutzen: `/v1/responses`
([Einrichtung](docs/DETAILS.md#the-responses-api-and-codex-cli)).
- **Nachdenken:** Wähle **aus, niedrig, mittel oder hoch** im Chat-Menü oder unter „Reasoning Effort“ in deiner
App. Aus ist am schnellsten. Hoch ist am besten für schwierige Fragen.
- **Bilder:** Antworte im Setup mit Ja auf „Images?“. Klick dann im Chat auf **Picture** oder hänge Bilder in deiner
App an. AMD-Karten lesen Bilder unter Linux über den Prozessor; unter Windows geht das noch nicht.
- **Vom Handy oder einem anderen PC:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`. Setz immer einen
Key.
- **Eine Anfrage nach der anderen:** Standardmäßig beantwortet Strata eine Anfrage, die anderen warten. Um mehrere
gleichzeitig zu beantworten, setz `"parallel": 2` ([BATCHING.md](docs/BATCHING.md)). Auf einer 12-GB-Karte wird
dadurch jede Antwort langsamer.
- **Lange Prompts:** Strata liest die erste Nachricht eines Chats komplett, etwa 1 Minute pro 30,000 Tokens.
Folgenachrichten starten in Sekunden.
## Wie schnell ist es?
Mehr: [wo deine Chats gespeichert werden](docs/INSTALL.md#where-things-are-stored), [die API](docs/DETAILS.md#using-it).
Gemessen mit einer Grafikkarte mit 12 GB (RTX 5070, Ryzen 5 7600, 64 GB RAM). Ergebnisse anderer Karten findest du hier:
[Tempo jeder Größe](docs/MODELS.md#how-fast-is-each-size), [Ergebnisse der Community](docs/COMMUNITY_BENCHMARKS.md).
## Etwas ist schiefgelaufen?
| Modell | Antworten erscheinen mit | Liest deinen Prompt mit |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 Tokens/s | `████████████` 2.650 Tokens/s |
| **IQ2_XS** | `██████████░░` 79 Tokens/s | `█████████░░░` 2.090 Tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 Tokens/s | `████████░░░░` 1.750 Tokens/s |
| **IQ3_S** | `███████░░░░░` 53 Tokens/s | `███████░░░░░` 1.620 Tokens/s |
| **Coder** | `███████░░░░░` 55 Tokens/s | `██████████░░` 2.180 Tokens/s |
- **Mein PC ist beim ersten Start von Strata eingefroren.** Das ist normal, während das Modell lädt. Warte und
schließ das Fenster nicht. Nach 10 Minuten immer noch eingefroren? Starte den PC neu, schließ andere Programme und
versuch es noch einmal, oder nimm eine kleinere Größe.
- **Es hat beim Herunterladen oder Installieren aufgehört.** Starte `START-HERE.bat` (oder `./setup.sh`) noch
einmal. Es macht dort weiter, wo es aufgehört hat.
- **Es ist sehr langsam und die Festplattenlampe blinkt ständig, oder es meldet „the engine stopped
unexpectedly“.** Dein PC hat nicht genug freien RAM. Schließ andere Programme (Browser brauchen viel) oder nimm
eine kleinere Größe (Q2_0 oder IQ2_XS).
- **Es meldet, dass Port 8080 schon belegt ist.** Strata läuft bereits. Such nach seinem Fenster.
Ein Token ist etwa ¾ eines Wortes, 60 Tokens pro Sekunde sind also schneller, als du lesen kannst. "Liest deinen Prompt"
beschreibt, wie schnell Strata ein langes Dokument, Code oder einen Chatverlauf aufnimmt.
Weitere Probleme und ihre Lösungen: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Kommst du nicht weiter?
Öffne ein [Issue](https://github.com/Niko1221/Strata/issues) und hänge `strata-<model>.log` aus dem Strata-Ordner
an. Du hast ein Sicherheitsproblem gefunden? Melde es vertraulich: [SECURITY.md](SECURITY.md).
Gemessen mit einer AMD-Karte mit 16 GB (RX 9070 XT, Ryzen 9 3900X, 47 GB RAM):
## Wie funktioniert es?
| Modell | Antworten erscheinen mit | Liest deinen Prompt mit |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 Tokens/s | `█████░░░░░░░` 1.160 Tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 Tokens/s | `█████░░░░░░░` 1.110 Tokens/s |
| **Coder** | `██████░░░░░░` 44 Tokens/s | `██████░░░░░░` 1.420 Tokens/s |
Modelle wie dieses laufen normalerweise auf Servern mit Hunderten Gigabyte Grafikspeicher. Deine Grafikkarte hat
12-24 GB. Strata bringt das Modell trotzdem unter, indem es **die Arbeit auf deinen ganzen PC verteilt**. Stell dir
eine Küche vor: Was du ständig brauchst, bleibt auf der Arbeitsfläche, der Rest wartet in der Vorratskammer.
Die Balken haben in beiden Tabellen denselben Maßstab. Eine Karte mit mehr VRAM ist schneller. Eine RTX 3090 (24 GB) sollte
etwa 100-140 Tokens pro Sekunde schreiben. Die Engine-Version hinter jeder Zahl steht in [DETAILS.md](docs/DETAILS.md#speed-measured).
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="Die 24,576 Experten des Modells: die meistgenutzten auf der Grafikkarte, alle im RAM, eine Nachschlagetabelle auf der SSD"></p>
## Benutzung
- **Das Modell ist ein Team aus 24,576 kleinen Spezialisten („Experten“).** Jedes Wort braucht nur 10 davon.
- **Deine Grafikkarte** hält die paar tausend Experten, die am häufigsten gebraucht werden. **Dein RAM** hält alle,
und **dein Prozessor** rechnet gleichzeitig am Rest. **Deine SSD** hält eine große Nachschlagetabelle.
### In der App
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="Ein kleiner Helfer rät die nächsten Wörter; das große Modell prüft alle auf einmal und behält die richtigen"></p>
Die App hat einen **Chat**, einen **Monitor** für das Modell sowie für GPU, CPU und RAM, eine Seite **Modelle** und die
**Einstellungen**. Der Chat kann in einem Ordner auf deinem PC arbeiten. Im Modus **Manuell** fragt er nach, bevor er eine
Datei ändert oder einen Befehl ausführt. Mit `/loop 10m <what to do>` wiederholst du eine Nachricht alle 10 Minuten, solange
der Chat geöffnet ist.
- **Raten, dann prüfen:** Ein kleiner Helfer rät die nächsten paar Wörter. Das große Modell prüft sie alle auf
einmal. Du bekommst dieselbe Antwort, nur 1.6-1.8x schneller.
- **Lange Texte werden in großen Stücken gelesen** (bis zu 8,192 Tokens auf einmal), mit über 1,000 Tokens pro
Sekunde.
Der Chat kann das Web mit DuckDuckGo (oder Bing) durchsuchen, ohne Konto. Unter **Einstellungen, Tools** kannst du einen
Suchschlüssel hinzufügen. Die Suche ist die einzige Funktion, die etwas von deinem PC sendet (die Suchbegriffe).
Die längere Erklärung: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Jeder Teil und seine Zahlen:
Um zwei Modelle auszuführen, schalte **Mehrere Modelle ausführen** ein und gib jedem Modell eine eigene Karte desselben
Herstellers.
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>Ein Chat in der Strata-App: Der Agent hat einen kleinen Voxel-Pagodengarten als eine HTML-Datei geschrieben, und die Seite öffnet sich in der Vorschau neben dem Gespräch</sub></p>
### Im Terminal
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub><code>strata monitor</code>, während das Modell antwortet: Geschwindigkeit, GPU, Speicher und die letzten Anfragen</sub></p>
```sh
strata launch # guided screens: model, context, pictures, access, then serve
strata launch --model q2_0 --yes --no-tui # no questions: serve, print the address, return
strata launch --plan --model q2_0 # show what would happen; nothing changes
strata launch --status # what is being served now
strata launch --stop # stop it
strata monitor --url http://127.0.0.1:8080 # watch speed, GPU, memory and requests
strata update --check # is there a new engine? (strata update installs it)
strata doctor # check the installed engine files
```
`strata --help` listet alles auf. Starten und Bereitstellen laden nichts herunter. Nur die Installation, `strata launch`
(wenn du ein Modell wählst) und `strata update` tun das. Um Strata von deinem Handy oder einem anderen PC aus zu erreichen,
nutze `strata launch --access lan`. Der Befehl erzeugt einen API-Schlüssel und startet in einem Netzwerk nicht ohne einen.
Ein Browser öffnet sich nur, wenn du `--open` angibst. Anleitung: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). Der ganze Client: [docs/TERMINAL.md](docs/TERMINAL.md).
### Mit KI-Agents
Ein Agent installiert Strata mit dem Einzeiler aus Option B und führt
`strata launch --model NAME --yes --no-tui` aus. Die Schritte, Flags und Exit-Codes stehen in [docs/AI_SETUP.md](docs/AI_SETUP.md),
die Regeln für das Repository in [AGENTS.md](AGENTS.md).
### Andere Apps verbinden
| App-Typ | Basis-URL |
| --- | --- |
| OpenAI-kompatibel (die meisten Apps und Agents) | `http://127.0.0.1:8080/v1` (beliebiger API-Schlüssel, beliebiger Modellname) |
| Anthropic-API (Claude Code und andere) | `http://127.0.0.1:8080`, ohne `/v1`. Diese Apps hängen `/v1/messages` selbst an. Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| OpenAI Responses API (Codex CLI) | `http://127.0.0.1:8080/v1/responses` ([Einrichtung](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` gibt die genauen Einstellungen für OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent und OpenClaw aus.
`strata connect opencode --write` schreibt die Datei nach einer Rückfrage und legt eine Sicherung an.
### Bilder
Bilder sind für Modelle, die sie lesen können, standardmäßig eingeschaltet. Hänge ein Bild im Chat an oder füge es ein, oder
sende es aus deiner App. Der Bildleser läuft auf dem Prozessor, wenn deine Grafikkarte weniger als 24 GB hat (einige Sekunden
pro Bild, ohne Speicher der Karte zu belegen), und ab 24 GB auf der Grafikkarte. Die Regel gilt für NVIDIA und AMD gleich.
Du kannst Bilder pro Modell ausschalten oder den Leser mit `--vision off`, `--vision gpu` oder `--vision cpu` wählen.
Die Zahlen hinter der Regel: Auf einer Radeon AI PRO R9700 brauchte der Bildleser der Karte 57 ms für ein Bild mit 260 Tokens,
der des Prozessors 3,1 s. Auf Karten mit 16 GB (Tesla P100 und RTX A4000, Linux) kostete der Bildleser der Karte 3-5 % des
Texttempos, weil er etwa 1,4 GB belegt, die sonst das Modell nutzen würde. Auf dem Mac liest die GPU, egal wie viel Speicher er hat (0,45 s für ein Bild mit 260 Tokens auf einem M4 mit 32 GB, etwa 1,2 GB des gemeinsamen
Speichers); der Prozessor braucht dort etwa 50 s pro Bild und liest deshalb nur mit `--vision cpu`.
### Gut zu wissen
- Das Denkniveau (**aus, niedrig, mittel oder hoch**) stellst du im Chat-Menü oder unter "reasoning effort" in deiner App ein. Aus ist am schnellsten.
- Strata beantwortet standardmäßig eine Anfrage nach der anderen. Für mehrere gleichzeitig nutze `--parallel N` in `strata launch` oder
`"parallel": 2` in der Konfiguration ([BATCHING.md](docs/BATCHING.md)). Auf einer Karte mit 12 GB wird dadurch jede Antwort langsamer.
- Strata liest die erste Nachricht eines Chats komplett, etwa 1 Minute pro 30.000 Tokens. Folgenachrichten starten in Sekunden.
- Wo deine Chats gespeichert werden und die ganze API: [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored), [docs/DETAILS.md](docs/DETAILS.md#using-it).
## Aus dem Quellcode bauen
<details>
<summary>Windows, Linux und Mac aus dem Repository</summary>
[Lade Strata herunter](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) und entpacke es, oder nutze `git clone`.
**Windows:** Doppelklick auf `START-HERE.bat`. **Linux:** Führe `./setup.sh` im Strata-Ordner aus.
**Mac (Apple Silicon, macOS 15 oder neuer):** Installiere einmal die Command Line Tools von Apple mit `xcode-select --install`
(Xcode selbst brauchst du nicht). Öffne das Terminal im Strata-Ordner und führe `./setup.sh` aus. Das Skript baut die
Mac-Engine auf deinem Mac (etwa 7 Minuten), stellt dieselben Fragen, lädt das Modell herunter und startet es. Beim nächsten
Mal startet `./setup.sh` sofort. Welche Größe zu deinem Mac passt: [docs/MAC.md](docs/MAC.md).
Der Installer erkennt deine Karte und richtet die passende Engine ein. Er fragt, welches Modell und welche Größe du willst,
wie viel Kontext (wie viel Text sich das Modell merkt) und ob es Bilder lesen soll. Drück Enter für die empfohlene Antwort.
Dann lädt er das Modell herunter (etwa 70 GB) und startet es. Wenn der Download abbricht, führe das Skript erneut aus, dann
geht es weiter. Dein Browser öffnet die Strata-Seite unter `http://127.0.0.1:8080`.
> Während das Modell startet, kann dein PC 1-3 Minuten langsam sein oder nicht reagieren (beim ersten Mal am längsten). Strata
> lädt 35-55 GB in deinen RAM und sperrt einen Teil davon für die Grafikkarte. Warte ab und schließe das Fenster nicht.
Starte beim nächsten Mal wieder `START-HERE.bat` (oder `./setup.sh`). Es startet sofort und lädt nichts doppelt herunter.
Schließ sein Fenster, um das Modell zu beenden. `UPDATE.bat` (`./update.sh`) aktualisiert Strata, ohne es zu starten.
Aktualisieren, Docker, mehrere Karten, wohin die Dateien kommen und alle Optionen: [docs/INSTALL.md](docs/INSTALL.md).
Um deinen PC zu messen, führe `BENCH.bat` (`./bench.sh`) aus. Das dauert 5-10 Minuten und schreibt einen Bericht, den du teilen
kannst ([so geht's](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).
**Optional: mehr Tempo auf einer Karte, wenn das ganze Modell in den RAM passt.** Zwei Einstellungen zum Einschalten von
RubenTorney (#1904, #1905). Sie helfen, wenn du eine Grafikkarte nutzt und dein RAM alle Experten aufnimmt, zum Beispiel bei
Q2_0 oder IQ3_XXS auf einem PC mit 64 GB. Öffne `strata-<model>.json` im Strata-Datenordner ([wo](docs/INSTALL.md#where-things-are-stored)) und ergänze:
- `"--adapt-async", "1"` in der Liste `"args"`. Der Expertencache wird dann in einem Hilfsthread aktualisiert, statt die
Antwort anzuhalten. Gemessen auf einer RTX 5070 (Windows, Q2_0): +4-5 % beim Decodieren, +7-10 % bei Code und langen
Dokumenten. Die Antworten können sich leicht von der Standardeinstellung unterscheiden und bleiben von Lauf zu Lauf gleich.
- `"STRATA_PIN_BY_NEED": "1"` im Block `"env"` (Windows). Windows kann nur etwa die Hälfte des RAM für die Grafikkarte sperren,
und diese Einstellung sperrt zuerst die Experten, die die Karte am dringendsten braucht. Gemessen auf der RTX 5070: +4,4 % beim Lesen des Prompts.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
Starte Strata nach der Änderung neu. Zum Rückgängigmachen entfernst du die beiden Einträge.
</details>
## Wie funktioniert das?
Modelle wie dieses laufen normalerweise auf Servern mit Hunderten Gigabyte Grafikspeicher. Deine Grafikkarte hat 12-24 GB.
Strata bringt das Modell auf deinen PC, indem es die Arbeit auf den ganzen Rechner verteilt. Stell dir eine Bibliothek vor.
- Das Modell ist ein Team aus 24.576 kleinen Spezialisten ("Experten"), wie 24.576 Nachschlagewerke. Für jedes Wort braucht es nur 10 davon.
- **Deine Grafikkarte ist der Schreibtisch.** Er ist klein und schnell. Die etwa 4.000 am häufigsten genutzten Bücher liegen
dort aufgeschlagen.
- **Dein RAM sind die Regale im Lesesaal.** Alle 24.576 Bücher stehen dort. Braucht ein Wort ein Buch, das nicht auf dem
Schreibtisch liegt, holt dein Prozessor es sofort aus dem Regal, während die Karte weiterarbeitet.
- **Deine SSD ist das Archiv.** Sie enthält eine Nachschlagetabelle von 29 GB. Das Modell liest pro Wort nur wenige Zeilen
daraus, deshalb muss sie nicht im RAM liegen.
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
Ein kleiner Helfer rät außerdem die nächsten Wörter. Das große Modell prüft sie alle auf einmal und behält die richtigen. Du
bekommst dieselbe Antwort, 1,6-1,8-mal früher. Lange Texte werden in großen Stücken gelesen, bis zu 8.192 Tokens auf einmal,
mit über 1.000 Tokens pro Sekunde.
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
Die ausführliche Erklärung: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Jeder Teil mit seinen Zahlen:
[die Details](docs/DETAILS.md#how-it-works) und das [Paper](docs/paper/Strata-Paper.pdf).
## Etwas ist schiefgegangen?
- **Mein PC ist beim ersten Start von Strata eingefroren.** Strata lädt das Modell in den RAM. Warte ab und schließe das
Fenster nicht. Nach 10 Minuten immer noch eingefroren? Starte den PC neu, schließe andere Programme und versuch es noch
einmal, oder wähle eine kleinere Größe.
- **Es ist beim Herunterladen oder Installieren stehen geblieben.** Starte die App oder `strata launch` noch einmal, oder
führe `START-HERE.bat` (oder `./setup.sh`) erneut aus. Es macht dort weiter, wo es aufgehört hat.
- **Es ist sehr langsam, die Festplattenanzeige blinkt ständig, oder es meldet "the engine stopped unexpectedly".** Dein PC
hat nicht genug freien RAM. Schließe andere Programme (Browser brauchen viel) oder wähle eine kleinere Größe wie Q2_0 oder IQ2_XS.
- **Es meldet, dass es keine Engine für diesen PC gibt.** Öffne in der App **Einstellungen, Engine** und drücke **Engine
aktualisieren**. Führe im Terminal `strata update` aus. `strata doctor` zeigt, welche Engine-Dateien installiert sind.
- **Es meldet, dass Port 8080 schon belegt ist.** Strata läuft bereits. Such das Fenster.
Weitere Probleme und Lösungen: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Du kommst nicht weiter? Öffne ein
[Issue](https://github.com/Niko1221/Strata/issues). Hänge in der App die Datei aus **Einstellungen, Engine, Diagnose, Protokolle anzeigen, Protokolle speichern** an. Füge im Terminal die Ausgabe von `strata doctor` ein. Hänge beim Start aus dem Quellcode die
Datei `strata-<model>.log` aus dem Strata-Ordner an. Ein Sicherheitsproblem meldest du bitte privat: [SECURITY.md](SECURITY.md).
## Danksagung und Lizenz
Das Modell ist [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) vom Qwen-Team. Komprimiert
wurde es von [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5)
Das Modell ist [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) vom Qwen-Team. Komprimiert wurde es von
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5), agentionai (Gyro)
und Unsloth. Strata nutzt Teile von [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). Alle Danksagungen:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata ist Open Source unter der [MIT-Lizenz](LICENSE). Ein
paar Teile und alle Modelle haben eigene Lizenzen ([welche](docs/HOW_IT_WORKS.md#license)).
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata ist Open Source unter der [MIT-Lizenz](LICENSE). Einige Teile und
alle Modelle haben eigene Lizenzen ([welche](docs/HOW_IT_WORKS.md#license)).
## Mitmachen
Du hast einen Fehler gefunden oder willst eine Änderung oder einen Benchmark einsenden? Lies zuerst [CONTRIBUTING.md](CONTRIBUTING.md).
Dort steht, was dazugehört, damit andere deine Arbeit nachprüfen können.
## Strata unterstützen
Strata ist kostenlos und Open Source. Wenn es dir nützt, kannst du die Entwicklung unterstützen:
Strata ist kostenlos und steht unter der MIT-Lizenz. Ich baue es und teste es auf jeder Hardware, die ich bekommen kann. Wenn es auf deinem PC gut läuft, hilft mir ein Kaffee,
weiter neue Karten zu testen und das zu beheben, was ihr meldet. Es besteht kein Druck. Strata zu nutzen und zu melden, was dir auffällt, hilft genauso viel.
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
Du möchtest Strata sponsern? Schreib mir über [GitHub](https://github.com/Niko1221).
+341 -154
View File
@@ -1,216 +1,403 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
</picture>
</p>
[English](README.md) · [简体中文](README.zh-CN.md) · [日本語](README.ja.md) · [Deutsch](README.de.md) · [Français](README.fr.md) · **Español** · [Português](README.pt-BR.md)
<p align="center"><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <b>Español</b> · <a href="README.pt-BR.md">Português</a></p>
<p align="center"><b>Ejecuta un modelo de IA de 125 mil millones de parámetros en tu propio PC gaming</b><br>
Tarjeta gráfica NVIDIA o AMD (12 GB o más) · Windows o Linux · libre y de código abierto</p>
<h3 align="center">Ejecuta un modelo de última generación en tu propio PC gaming</h3>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="Un jardín con una pagoda de vóxeles que escribió el modelo de Strata, funcionando en el navegador"></a><br>
<p align="center">Compatible con NVIDIA, AMD, Intel · Windows, Linux, Mac · libre y de código abierto</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>Un jardín con una pagoda de vóxeles, hecho con un solo prompt en una RTX 5070 con Strata (IQ3_S, contexto de 128K) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">vídeo completo (49 s)</a></sub></p>
Strata ejecuta **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** en un PC normal. Es un
modelo de IA grande e inteligente que normalmente necesita un servidor. Conversa, escribe código, entiende imágenes
y funciona con tus aplicaciones y agentes de programación. Nada sale de tu PC.
Strata ejecuta **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**, un modelo de 125 mil millones
de parámetros que normalmente necesita un servidor, en un PC normal. Conversa, escribe código, lee imágenes y funciona
con tus aplicaciones y tus agentes de programación. Tus chats y tus archivos se quedan en tu PC.
## ¿Qué velocidad tiene?
**¿Eres nuevo?** Mira [lo que necesitas](#lo-que-necesitas) y luego [instala la aplicación](#opción-a-la-aplicación-strata).
Lo medimos en dos PC gaming normales. Un token equivale más o menos a ¾ de una palabra.
- **Funciona en un PC gaming.** Con 32 GB entre RAM y memoria gráfica ya puedes empezar.
- **Funciona con tus herramientas.** Habla las API de OpenAI y de Anthropic, así que Claude Code, Codex CLI, OpenCode y otros agentes se conectan a él.
- **Lee imágenes.** Viene activado por defecto.
- **Gratis.** Licencia MIT, sin cuenta, sin nube.
- **Privado.** Sin telemetría. Lo que Strata descarga, cuándo usa internet y cómo comprueba las descargas está en [docs/SAFETY.md](docs/SAFETY.md).
- **Escribe respuestas:** lo rápido que aparece la respuesta en un chat corto. 60 tokens por segundo es más rápido de lo que puedes leer.
- **Lee tu prompt:** lo rápido que procesa lo que le envías (aquí, un documento, código o historial de chat de 32K tokens).
Versión 0.2.1 · motor 0.1.46
<table>
<tr><th>NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM</th><th>AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM</th></tr>
<tr><td>
## Lo que necesitas
| Tamaño | Escribe respuestas | Lee tu prompt |
| --- | ---: | ---: |
| **Q2_0** | 94 tokens/s | 2,650 tokens/s |
| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
| **Coder** | 55 tokens/s | 2,180 tokens/s |
**Necesitas al menos 32 GB entre RAM y memoria gráfica (VRAM).** En un Mac o un Ryzen AI Max, es la memoria unificada.
Con más memoria puedes usar un modelo más grande o más rápido.
</td><td>
- Un PC de 32 GB con una tarjeta de 12 a 16 GB ejecuta el Coder.
- Con 64 GB de RAM funcionan todos los tamaños.
| Tamaño | Escribe respuestas | Lee tu prompt |
| --- | ---: | ---: |
| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
| **Coder** | 44 tokens/s | 1,420 tokens/s |
</td></tr>
</table>
NVIDIA: Q2_0 con el motor 0.1.36, las demás filas con 0.1.26 (respuestas de 4K, prompts de 32K). Las tablas
completas están en [DETAILS.md](docs/DETAILS.md#speed-measured). Una tarjeta con más VRAM es más rápida: una
RTX 3090 (24 GB) debería escribir unos 100-140 tokens por segundo. Chats largos y otras tarjetas:
[velocidad de cada modelo](docs/MODELS.md#how-fast-is-each-size), [resultados de la comunidad](docs/COMMUNITY_BENCHMARKS.md).
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
<sub>Strata es gratis. Si funciona bien en tu PC, un café ayuda a seguir trabajando en él.</sub></p>
## Qué necesitas
| | |
| Tu ordenador | Detalles |
| --- | --- |
| **Tarjeta gráfica** | **NVIDIA** GeForce serie RTX 20, 30, 40 o 50, o **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 o serie RX 6800 / 6900. Necesita **12 GB de VRAM o más**. |
| **RAM** | 32 GB o más. Tu RAM decide [qué modelo](#qué-modelo-elijo) cabe. Con 64 GB funcionan todos los tamaños. |
| **Disco** | Unos 80 GB libres. Usa un SSD si puedes: el primer arranque es mucho más rápido. |
| **Sistema** | Windows 10 / 11 o Linux, y un controlador gráfico actual de NVIDIA o AMD. |
| **NVIDIA** GeForce RTX serie 20, 30, 40, 50 (12 GB o más) | Instala la aplicación o la Terminal. |
| **AMD** Radeon RX 6800 y posteriores, Radeon AI PRO R9700 (12 GB o más) | [Tarjetas AMD](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md), [motor Intel](docs/INTEL.md) |
| **Mac con Apple Silicon** (M1 o posterior, macOS 15 o posterior) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max (Strix Halo)** y otros sistemas con memoria unificada | [Strix Halo](docs/STRIX_HALO.md) |
| Tarjetas y procesadores antiguos | [GPU antiguas](docs/OLDER_GPUS.md), [CPU antiguas](docs/INSTALL.md#older-cpus-experimental) |
El instalador prepara todo lo demás. Dos o tres tarjetas pueden repartirse el modelo ([multi-GPU](docs/MULTI_GPU.md)).
También hace falta:
Experimental, escrito y probado por miembros de la comunidad en sus propios equipos:
- unos 80 GB de disco libre (con un SSD el primer arranque es mucho más rápido)
- Windows 10 / 11, Linux o macOS
- un controlador gráfico actual
- **Tarjetas gráficas antiguas** (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT): [Older GPUs](docs/OLDER_GPUS.md).
- **Intel Arc**, compilado desde el código fuente en Linux: [Intel Arc](docs/INTEL_ARC.md).
- **AMD Ryzen AI Max (Strix Halo)**, compilado desde el código fuente en Linux: [Strix Halo](docs/STRIX_HALO.md).
- **Procesadores antiguos sin AVX2**: funcionan, pero despacio. [Older CPUs](docs/INSTALL.md#older-cpus-experimental).
La lista completa: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
Dos o tres tarjetas de la misma marca pueden compartir un modelo ([multi-GPU](docs/MULTI_GPU.md)). Strata nunca reparte un
modelo entre tarjetas NVIDIA y AMD. Lista completa: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
## Instalación
### Deja que tu IA lo instale
Elige una opción. Todas arrancan el mismo motor y usan los mismos modelos.
¿Usas un asistente de programación con IA (Claude Code, Cursor, Codex, GitHub Copilot, ...)? Pega esto en él:
### Opción A: la aplicación Strata
La forma más fácil. La aplicación lleva el motor dentro: la instalas, eliges un modelo y chateas. Descarga el archivo para
tu sistema y tu tarjeta gráfica desde la [página de versiones](https://github.com/Niko1221/Strata/releases/latest). ¿No
sabes cuál elegir? Mira el párrafo que hay debajo de la tabla.
| Sistema | Tu tarjeta gráfica | Archivo que descargar |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX, controlador 580 o posterior | [`Strata-0.2.1-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda13.exe) |
| | NVIDIA, tarjeta más antigua o controlador anterior a 580 | [`Strata-0.2.1-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda12.exe) |
| | AMD Radeon serie RX 9000 o Radeon AI PRO R9700 | [`Strata-0.2.1-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip.exe) |
| | AMD Radeon serie RX 7000 o Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX, controlador 580 o posterior | [`Strata-0.2.1-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda13.AppImage) |
| | NVIDIA, tarjeta más antigua o controlador anterior a 580 | [`Strata-0.2.1-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon serie RX 9000 o Radeon AI PRO R9700 | [`Strata-0.2.1-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx9070.AppImage) |
| | AMD Radeon serie RX 7000 | [`Strata-0.2.1-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-strixhalo.AppImage) |
| **Mac** (Apple Silicon, macOS 15 o posterior) | integrada | [`Strata-0.2.1-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-mac.dmg) |
**Si no estás seguro:** con NVIDIA, elige `cuda13` si la versión de tu controlador es la 580 o posterior, y `cuda12` en caso
contrario. `cuda12` es para tarjetas y controladores más antiguos. Los archivos `hip` y `amd` son para tarjetas AMD. Las
demás tarjetas AMD (RX 6000) todavía no tienen una aplicación ya preparada: mira
[Tarjetas AMD](docs/AMD_HIP.md).
Después:
- **Windows:** ejecuta el `.exe`. Se instala para tu usuario y no necesita permisos de administrador. Si Windows muestra
"Windows protegió tu PC", elige **Más información** y luego **Ejecutar de todas formas** (el instalador todavía no está
firmado).
- **Linux:** haz el archivo ejecutable (`chmod +x`) y lánzalo. Necesita FUSE 2: `sudo apt install libfuse2` (Ubuntu 22.04)
o `libfuse2t64` (Ubuntu 24.04 y posteriores).
- **Mac:** abre el `.dmg` y arrastra Strata a Aplicaciones.
En la primera pantalla, la aplicación analiza tu PC y recomienda un modelo adecuado. Pulsa Instalar. El modelo es una
descarga grande (unos 40 a 70 GB) y, si se interrumpe, continúa donde se quedó. Al cerrar la ventana, Strata y el motor se
detienen.
### Opción B: la Terminal (una línea)
La Terminal es el comando `strata`, pensado para scripts, servidores y agentes de IA.
Linux y Mac:
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
```
Windows (PowerShell):
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
El script analiza tu PC, descarga la Terminal, verifica su SHA-256 y la descomprime en tu carpeta personal. No necesita
permisos de administrador. En Windows, abre una ventana de terminal nueva después. Luego arranca Strata:
```sh
strata launch
```
Te hace una pregunta cada vez: modelo, contexto, imágenes, quién puede conectarse. Después descarga el motor para tu tarjeta
gráfica (firmado y arrancado como prueba antes de usarlo) y el modelo, y empieza a servir. Para scripts y agentes de IA, usa
opciones:
```sh
strata launch --model q2_0 --yes --no-tui
```
Si quieres leer el script de instalación antes de ejecutarlo, descárgalo primero con `curl -fsSL <address> -o install.sh`,
léelo y luego ejecuta `sh install.sh`. Todas las pantallas y opciones: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md).
### Opción C: que lo instale tu IA
Pega esto en Claude Code, Cursor, Codex, GitHub Copilot u otro asistente de programación:
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
Revisa tu tarjeta gráfica, tu RAM y tu disco, y elige el modelo que cabe. Luego lo instala, lo arranca y te explica
cómo conectar tus aplicaciones. Las herramientas de IA también pueden instalar, arrancar y detener Strata con su
El asistente revisa tu tarjeta gráfica, tu RAM y tu disco, elige el modelo adecuado, lo instala y lo arranca, y te explica
cómo conectar tus aplicaciones. Las herramientas de IA también pueden instalar, arrancar y detener Strata mediante su
[servidor MCP](docs/MCP_SERVER.md).
### O hazlo tú mismo
¿Prefieres compilarlo tú? Mira [Compilar desde el código fuente](#compilar-desde-el-código-fuente).
[Descarga Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) y descomprímelo (o usa `git clone`).
**Windows:** haz doble clic en **`START-HERE.bat`**. **Linux:** ejecuta **`./setup.sh`** en la carpeta de Strata.
## Modelos
Los pasos son los mismos para NVIDIA y AMD. El instalador detecta tu tarjeta y prepara el motor adecuado para ella.
Te hace unas pocas preguntas:
La aplicación y `strata launch` recomiendan un modelo según tu memoria. Es un solo modelo en varios tamaños, más o menos
comprimido. Los tamaños pequeños son más rápidos. Los grandes son algo más inteligentes.
- qué modelo y qué tamaño,
- cuánto contexto (cuánto texto tiene presente el modelo),
- si debe entender imágenes.
### ¿Poca memoria? Prueba Gyro
Pulsa Enter cada vez para usar la respuesta recomendada. Después descarga el modelo (unos 70 GB) y lo arranca. Si la
descarga se detiene, vuelve a ejecutarlo: sigue donde lo dejó. Tu navegador abre la aplicación de Strata en
`http://127.0.0.1:8080`.
[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) (de agentionai) es la forma más pequeña de ejecutar este modelo.
> **Mientras el modelo arranca, tu PC puede ir lento o dejar de responder durante 1-3 minutos** (más la primera vez).
> Strata carga 35-55 GB en tu RAM y reserva una parte para la tarjeta gráfica. Es normal. Espera y no cierres la
> ventana. La ventana muestra lo que está haciendo Strata.
| Tamaño | Memoria que necesita (RAM + VRAM) | Descarga | Qué es |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27,6 GB | 58,5 GB | Para PC donde no cabe nada más. La calidad es menor que la de IQ2_XS. |
| **Gyro-M** | 35 GB | 92,0 GB | Calidad parecida a IQ3_XXS con un 20 % menos de memoria de GPU. Escribe las respuestas un 17-21 % más despacio que Q2_0 en una tarjeta de 12 GB y en una R9700. |
**La próxima vez**, vuelve a ejecutar `START-HERE.bat` (o `./setup.sh`). Arranca enseguida y no descarga nada dos
veces. Cierra su ventana para detener el modelo. `UPDATE.bat` (`./update.sh`) actualiza Strata sin arrancarlo.
Actualizaciones, Docker, varias tarjetas, dónde se guardan los archivos y todas las opciones: [docs/INSTALL.md](docs/INSTALL.md).
Gyro funciona en tarjetas NVIDIA y AMD (en AMD es experimental), no en Intel Arc ni en el Mac. Detalles y mediciones:
[docs/GYRO.md](docs/GYRO.md). Arráncalo con `strata launch --model gyro-s`.
## ¿Qué modelo elijo?
### Los demás tamaños
El instalador te recomienda uno según tu RAM. El mismo modelo viene en varios tamaños, más o menos comprimidos. Los
tamaños pequeños son más rápidos. Los grandes son algo más inteligentes.
| Tu RAM | Elige | Por qué |
| Tu memoria (RAM + VRAM) | Elige | Notas |
| --- | --- | --- |
| **32 GB** | **Coder** | cabe en 32 GB y está hecho para programar (con una tarjeta de 24 GB también funcionan Q2_0 e IQ2_XS) |
| **48 GB** | **IQ2_XS** (o Q2_0, el más rápido) | los tamaños más grandes no caben |
| **64 GB** | **IQ2_XS** (recomendado), o IQ3_XXS / IQ3_S | caben todos los tamaños; IQ3_S es el mejor y el más lento |
| **96 GB o más** | **IQ3_S**, o el UD-IQ4_XS de Unsloth (~4 bits) | espacio para los tamaños más grandes con todo lo demás abierto |
| **32 GB** | **Gyro-S**, o el **[Coder](docs/MODELS.md#coder)** para código | El Coder necesita 32 GB de RAM y conserva la mitad de los expertos, así que es más flojo fuera del código |
| **40 GB** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS (39,2 GB), o **Gyro-M** (35 GB) | Swift piensa menos tiempo antes de responder. Gyro-M se acerca a la calidad de IQ3_XXS, pero escribe las respuestas un 17-21 % más despacio que Q2_0 |
| **48 GB** | **IQ3_XXS** (47,0 GB) | Más lento, mejor |
| **64 GB** | **IQ3_S** (54,8 GB), el original o Swift 1.5 | El más lento, el mejor. Iguala al modelo completo en las pruebas publicadas |
| **96 GB o más** | **UD-IQ4_XS** de Unsloth (unos 72 GB de RAM) hasta **Q8_0** (unos 136 GB), de 4 a 8 bits | Más cerca del modelo completo. Con menos RAM, Strata lee una parte desde el SSD, que es mucho más lento |
- **[Coder](docs/MODELS.md#coder):** una versión para programar a la que se le quitó la mitad de los expertos. Alcanza
el 91% de la puntuación SWE-bench Verified del modelo completo (medido por sus autores) y cabe en 32 GB de RAM. Es
más flojo fuera del código, también con chino y otros textos CJK (#438). Para eso, elige Q2_0, IQ2_XS o IQ3_S, que
conservan todos los expertos.
- **[Swift 1.5](docs/MODELS.md#swift-15):** un ajuste fino que piensa mucho menos tiempo antes de responder. Recibes
la respuesta antes, con más o menos la misma calidad.
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** la versión de ~4 bits de Unsloth, entre IQ3_S y
UD-Q4_K_XL en calidad. Una descarga de 94 GB. Con menos de ~80 GB de RAM, Strata lee una parte desde el SSD
mientras responde, así que ahí es más lento (un SSD NVMe ayuda).
- **[Unsloth UD-Q4_K_XL](docs/MODELS.md#unsloth-ud-q4_k_xl-experimental)** (experimental): el más parecido al modelo
completo. Pero Strata lee la mayor parte desde el SSD mientras responde, así que solo escribe 7-8.5 tokens/s en un
PC con 64 GB.
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** se instala a mano. No está en
el menú del instalador.
Con menos RAM de la que indica una fila también puede funcionar: Strata pasa solo a su
[modo de poca RAM](docs/MODELS.md#a-big-graphics-card-and-little-ram). Espera que sea más lento, y mucho más lento con una
tarjeta gráfica pequeña.
Tamaños, descargas y qué cabe dónde: [docs/MODELS.md](docs/MODELS.md). Para añadir otro modelo más adelante, ejecuta
`SETUP.bat` (Linux: `./setup.sh --setup`).
Más opciones:
## Cómo usarlo
- **[Swift 1.5](docs/MODELS.md#swift-15)** es un ajuste fino que piensa menos tiempo antes de responder, con una calidad y un uso de RAM casi iguales.
- **[NVFP4](docs/NVFP4.md)** es la versión de 4 bits de NVIDIA para tarjetas NVIDIA. Necesita unos 80 GB de RAM para tener todos los expertos en RAM.
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** es un modelo más pequeño y distinto que lee imágenes y puede atender varias peticiones a la vez.
- **[El Uncensored IQ3_XXS de OrcaRouter](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** se configura a mano.
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="La pestaña Monitor de la aplicación de Strata junto a un agente de programación"><br>
<sub>El <b>Monitor</b> de la aplicación de Strata (izquierda) mientras un agente de programación escribe el jardín de la pagoda del vídeo (derecha)</sub></p>
Todos los tamaños, descargas y lo que cabe en cada equipo: [docs/MODELS.md](docs/MODELS.md).
- **En el navegador:** abre `http://127.0.0.1:8080`. Tiene **Chat**, un **Monitor** en vivo del modelo y de tu
GPU/CPU/RAM, y **About** con los ajustes y las direcciones.
- **Tus aplicaciones y agentes de programación:** añade un proveedor "compatible con OpenAI" con la URL base
**`http://127.0.0.1:8080/v1`**. Sirve cualquier clave de API y cualquier nombre de modelo.
- Aplicaciones que usan la API de Anthropic: `http://127.0.0.1:8080/v1/messages` (Claude Code:
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`).
- Codex CLI y otras aplicaciones que usan la API Responses de OpenAI: `/v1/responses`
([configuración](docs/DETAILS.md#the-responses-api-and-codex-cli)).
- **Razonamiento:** elige **off, low, medium o high** en el menú del chat o en el "reasoning effort" de tu aplicación.
Off es lo más rápido. High es lo mejor para preguntas difíciles.
- **Imágenes:** responde que sí a "Images?" en la instalación. Luego haz clic en **Picture** en el chat, o adjunta
imágenes en tu aplicación. Las tarjetas AMD leen imágenes en Linux a través del procesador; en Windows todavía no.
- **Desde tu móvil u otro PC:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`. Pon siempre una clave.
- **Una petición cada vez:** por defecto Strata responde una petición y las demás esperan. Para responder varias a la
vez, pon `"parallel": 2` ([BATCHING.md](docs/BATCHING.md)). En una tarjeta de 12 GB, cada respuesta va más lenta.
- **Prompts largos:** Strata lee entero el primer mensaje de un chat, más o menos 1 minuto por cada 30,000 tokens. Los
mensajes siguientes empiezan en segundos.
## ¿Qué velocidad tiene?
Más: [dónde se guardan tus chats](docs/INSTALL.md#where-things-are-stored), [la API](docs/DETAILS.md#using-it).
Medido con una tarjeta gráfica de 12 GB (RTX 5070, Ryzen 5 7600, 64 GB de RAM). Resultados con otras tarjetas:
[velocidad de cada tamaño](docs/MODELS.md#how-fast-is-each-size), [resultados de la comunidad](docs/COMMUNITY_BENCHMARKS.md).
## ¿Algo salió mal?
| Modelo | Las respuestas aparecen a | Lee tu prompt a |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2.650 tokens/s |
| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2.090 tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1.750 tokens/s |
| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1.620 tokens/s |
| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2.180 tokens/s |
- **Mi PC se congeló la primera vez que arrancó Strata.** Es normal mientras carga el modelo. Espera y no cierres la
ventana. ¿Sigue congelado después de 10 minutos? Reinicia el PC, cierra otros programas y vuelve a intentarlo, o
elige un tamaño más pequeño.
- **Se detuvo durante la descarga o la instalación.** Vuelve a ejecutar `START-HERE.bat` (o `./setup.sh`). Sigue
donde se detuvo.
- **Va muy lento y la luz del disco no deja de parpadear, o dice "the engine stopped unexpectedly".** Tu PC no tiene
suficiente RAM libre. Cierra otros programas (los navegadores usan mucha), o elige un tamaño más pequeño (Q2_0 o
IQ2_XS).
- **Dice que el puerto 8080 ya está en uso.** Strata ya está en marcha. Busca su ventana.
Un token equivale a unos ¾ de palabra, así que 60 tokens por segundo es más rápido de lo que puedes leer. "Lee tu prompt"
es lo rápido que Strata procesa un documento largo, código o un historial de chat.
Más problemas y sus soluciones: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). ¿Sigues atascado? Abre un
[issue](https://github.com/Niko1221/Strata/issues) y adjunta `strata-<model>.log` de la carpeta de Strata. ¿Has
encontrado un problema de seguridad? Infórmalo en privado: [SECURITY.md](SECURITY.md).
Medido con una tarjeta AMD de 16 GB (RX 9070 XT, Ryzen 9 3900X, 47 GB de RAM):
| Modelo | Las respuestas aparecen a | Lee tu prompt a |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1.160 tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1.110 tokens/s |
| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1.420 tokens/s |
Las barras usan la misma escala en las dos tablas. Una tarjeta con más VRAM es más rápida. Una RTX 3090 (24 GB) debería
escribir unos 100-140 tokens por segundo. La versión del motor detrás de cada cifra está en [DETAILS.md](docs/DETAILS.md#speed-measured).
## Uso
### En la aplicación
La aplicación tiene un **Chat**, un **Monitor** del modelo y de tu GPU, CPU y RAM, una página de **Modelos** y los
**Ajustes**. El chat puede trabajar en una carpeta de tu PC. En modo **Manual** pregunta antes de cambiar un archivo o
ejecutar un comando. Escribe `/loop 10m <what to do>` para repetir un mensaje cada 10 minutos mientras el chat esté abierto.
El chat puede buscar en la web con DuckDuckGo (o Bing), sin cuenta. Puedes añadir una clave de búsqueda en **Ajustes,
Herramientas**. La búsqueda es la única función que envía algo fuera de tu PC (las palabras que buscas).
Para ejecutar dos modelos, activa **Ejecutar varios modelos** y dale a cada modelo su propia tarjeta de la misma marca.
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>Un chat en la app de Strata: el agente escribió un pequeño jardín de pagoda de vóxeles en un solo archivo HTML, y la página se abre en la vista previa junto a la conversación</sub></p>
### En la Terminal
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub><code>strata monitor</code> mientras el modelo responde: velocidad, GPU, memoria y las últimas solicitudes</sub></p>
```sh
strata launch # guided screens: model, context, pictures, access, then serve
strata launch --model q2_0 --yes --no-tui # no questions: serve, print the address, return
strata launch --plan --model q2_0 # show what would happen; nothing changes
strata launch --status # what is being served now
strata launch --stop # stop it
strata monitor --url http://127.0.0.1:8080 # watch speed, GPU, memory and requests
strata update --check # is there a new engine? (strata update installs it)
strata doctor # check the installed engine files
```
`strata --help` lo lista todo. Arrancar y servir no descargan nada. Solo lo hacen la instalación, `strata launch` (cuando
eliges un modelo) y `strata update`. Para llegar a Strata desde tu móvil u otro PC, usa `strata launch --access lan`. Crea
una clave de API y no arranca en una red sin ella. No se abre ningún navegador salvo que añadas `--open`. Guía:
[docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). El cliente completo: [docs/TERMINAL.md](docs/TERMINAL.md).
### Con agentes de IA
Un agente instala Strata con el comando de una línea de la opción B y ejecuta
`strata launch --model NAME --yes --no-tui`. Los pasos, las opciones y los códigos de salida están en [docs/AI_SETUP.md](docs/AI_SETUP.md),
y las reglas del repositorio en [AGENTS.md](AGENTS.md).
### Conecta tus otras aplicaciones
| Tipo de aplicación | URL base |
| --- | --- |
| Compatible con OpenAI (la mayoría de aplicaciones y agentes) | `http://127.0.0.1:8080/v1` (cualquier clave de API, cualquier nombre de modelo) |
| API de Anthropic (Claude Code y otros) | `http://127.0.0.1:8080`, sin `/v1`. Estas aplicaciones añaden `/v1/messages` por su cuenta. Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| API OpenAI Responses (Codex CLI) | `http://127.0.0.1:8080/v1/responses` ([configuración](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` muestra los ajustes exactos para OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent y OpenClaw.
`strata connect opencode --write` escribe el archivo después de preguntar y guarda una copia de seguridad.
### Imágenes
Las imágenes vienen activadas por defecto en los modelos que pueden leerlas. Adjunta o pega una imagen en el chat, o envíala
desde tu aplicación. El lector de imágenes funciona en el procesador cuando tu tarjeta gráfica tiene menos de 24 GB (unos
segundos por imagen, sin quitar memoria a la tarjeta) y en la tarjeta gráfica a partir de 24 GB. La regla es la misma para
NVIDIA y AMD. Puedes desactivar las imágenes por modelo, o elegir el lector con `--vision off`, `--vision gpu` o
`--vision cpu`.
Las cifras detrás de la regla: en una Radeon AI PRO R9700, el lector de la tarjeta tardó 57 ms en una imagen de 260 tokens y
el del procesador 3,1 s. En tarjetas de 16 GB (Tesla P100 y RTX A4000, Linux), el lector de la tarjeta costó un 3-5 % de
velocidad en texto, porque ocupa unos 1,4 GB que de otro modo usaría el modelo. En un Mac lee la GPU, tenga la memoria que tenga (0,45 s para una imagen de 260 tokens en un M4 de 32 GB, unos 1,2 GB de la memoria compartida);
el procesador tarda unos 50 s por imagen, así que solo lee con `--vision cpu`.
### Conviene saber
- Elige el nivel de razonamiento (**desactivado, bajo, medio o alto**) en el menú del chat o en "reasoning effort" de tu aplicación. Desactivado es lo más rápido.
- Strata responde a una petición a la vez por defecto. Para varias a la vez, usa `--parallel N` en `strata launch` o
`"parallel": 2` en la configuración ([BATCHING.md](docs/BATCHING.md)). En una tarjeta de 12 GB, esto hace más lenta cada respuesta.
- Strata lee entero el primer mensaje de un chat, unos 1 minuto por cada 30.000 tokens. Los mensajes siguientes empiezan en segundos.
- Dónde se guardan tus chats y la API completa: [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored), [docs/DETAILS.md](docs/DETAILS.md#using-it).
## Compilar desde el código fuente
<details>
<summary>Windows, Linux y Mac desde el repositorio</summary>
[Descarga Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) y descomprímelo, o haz `git clone`.
**Windows:** haz doble clic en `START-HERE.bat`. **Linux:** ejecuta `./setup.sh` en la carpeta de Strata.
**Mac (Apple Silicon, macOS 15 o posterior):** instala una vez las Command Line Tools de Apple con `xcode-select --install`
(no hace falta Xcode). Abre la Terminal en la carpeta de Strata y ejecuta `./setup.sh`. El script compila el motor para Mac
en tu Mac (unos 7 minutos), hace las mismas preguntas, descarga el modelo y lo arranca. La próxima vez, `./setup.sh` lo
arranca enseguida. Qué tamaño cabe en tu Mac: [docs/MAC.md](docs/MAC.md).
El instalador detecta tu tarjeta y prepara el motor adecuado. Pregunta qué modelo y qué tamaño quieres, cuánto contexto
(cuánto texto mantiene el modelo en mente) y si debe leer imágenes. Pulsa Enter para aceptar la respuesta recomendada.
Luego descarga el modelo (unos 70 GB) y lo arranca. Si la descarga se detiene, vuelve a ejecutar el script y continúa. Tu
navegador abre la página de Strata en `http://127.0.0.1:8080`.
> Mientras el modelo arranca, tu PC puede ir lento o dejar de responder durante 1-3 minutos (más tiempo la primera vez).
> Strata carga 35-55 GB en tu RAM y bloquea una parte para la tarjeta gráfica. Espera y no cierres la ventana.
La próxima vez, ejecuta de nuevo `START-HERE.bat` (o `./setup.sh`). Arranca enseguida y no descarga nada dos veces. Cierra
su ventana para detener el modelo. `UPDATE.bat` (`./update.sh`) actualiza Strata sin arrancarlo. Actualización, Docker,
varias tarjetas, dónde van los archivos y todas las opciones: [docs/INSTALL.md](docs/INSTALL.md).
Para medir tu PC, ejecuta `BENCH.bat` (`./bench.sh`). Tarda 5-10 minutos y escribe un informe que puedes compartir
([cómo](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).
**Opcional: más velocidad en una tarjeta cuando todo el modelo cabe en la RAM.** Dos ajustes que se activan a mano, de
RubenTorney (#1904, #1905). Ayudan cuando usas una sola tarjeta gráfica y tu RAM guarda todos los expertos, por ejemplo con
Q2_0 o IQ3_XXS en un PC de 64 GB. Abre `strata-<model>.json` en la carpeta de datos de Strata ([dónde](docs/INSTALL.md#where-things-are-stored)) y añade:
- `"--adapt-async", "1"` a la lista `"args"`. Así la caché de expertos se actualiza en un hilo auxiliar en vez de detener la
respuesta. Medido en una RTX 5070 (Windows, Q2_0): +4-5 % en decodificación, +7-10 % en código y documentos largos. Las
respuestas pueden diferir un poco de las del ajuste por defecto y son iguales de una ejecución a otra.
- `"STRATA_PIN_BY_NEED": "1"` en el bloque `"env"` (Windows). Windows solo puede bloquear aproximadamente la mitad de la RAM
para la tarjeta gráfica, y este ajuste bloquea primero los expertos que la tarjeta necesita antes. Medido en la RTX 5070: +4,4 % en lectura del prompt.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
Reinicia Strata después del cambio. Para deshacerlo, quita las dos entradas.
</details>
## ¿Cómo funciona?
Los modelos como este suelen funcionar en servidores con cientos de gigabytes de memoria gráfica. Tu tarjeta gráfica
tiene 12-24 GB. Strata hace que el modelo quepa **repartiendo el trabajo por todo tu PC**. Piensa en una cocina: lo
que usas todo el rato se queda en la encimera, y el resto espera en la despensa.
Los modelos como este suelen ejecutarse en servidores con cientos de gigabytes de memoria gráfica. Tu tarjeta gráfica tiene
12-24 GB. Strata hace que el modelo quepa repartiendo el trabajo por todo tu PC. Imagina una biblioteca.
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="Los 24,576 expertos del modelo: los más usados en la tarjeta gráfica, todos en la RAM y una tabla de consulta en el SSD"></p>
- El modelo es un equipo de 24.576 pequeños especialistas ("expertos"), como 24.576 libros de consulta. Cada palabra necesita solo 10 de ellos.
- **Tu tarjeta gráfica es el escritorio.** Es pequeño y rápido. Los unos 4.000 libros que más se usan están abiertos encima.
- **Tu RAM son las estanterías de la sala de lectura.** Los 24.576 libros están ahí. Cuando una palabra necesita un libro que
no está en el escritorio, tu procesador lo coge enseguida de la estantería mientras la tarjeta sigue trabajando.
- **Tu SSD es el archivo.** Guarda una tabla de consulta de 29 GB. El modelo lee unas pocas filas por palabra, así que no
hace falta que esté en la RAM.
- **El modelo es un equipo de 24,576 pequeños especialistas ("expertos").** Cada palabra solo necesita 10 de ellos.
- **Tu tarjeta gráfica** guarda los pocos miles de expertos que más se usan. **Tu RAM** los tiene todos, y **tu
procesador** trabaja con el resto al mismo tiempo. **Tu SSD** guarda una gran tabla de consulta.
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="Un pequeño ayudante adivina las siguientes palabras; el modelo grande las comprueba todas a la vez y se queda con las correctas"></p>
Un pequeño ayudante también adivina las siguientes palabras. El modelo grande las comprueba todas a la vez y se queda con
las correctas. Obtienes la misma respuesta, entre 1,6 y 1,8 veces antes. Los textos largos se leen en trozos grandes, de
hasta 8.192 tokens a la vez, a más de 1.000 tokens por segundo.
- **Adivinar y comprobar:** un pequeño ayudante adivina las siguientes palabras. El modelo grande las comprueba todas
a la vez. Recibes la misma respuesta, 1.6-1.8x antes.
- **Los textos largos se leen en trozos grandes** (hasta 8,192 tokens cada vez), a más de 1,000 tokens por segundo.
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
La explicación larga: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Cada parte y sus números:
La explicación más larga: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Cada parte con sus cifras:
[los detalles](docs/DETAILS.md#how-it-works) y el [artículo](docs/paper/Strata-Paper.pdf).
## ¿Algo ha ido mal?
- **Mi PC se congeló la primera vez que arrancó Strata.** Carga el modelo en la RAM. Espera y no cierres la ventana. ¿Sigue
congelado después de 10 minutos? Reinicia el PC, cierra otros programas e inténtalo de nuevo, o elige un tamaño más pequeño.
- **Se detuvo durante la descarga o la instalación.** Abre de nuevo la aplicación o `strata launch`, o vuelve a ejecutar
`START-HERE.bat` (o `./setup.sh`). Continúa donde se quedó.
- **Va muy lento, la luz del disco parpadea sin parar, o dice "the engine stopped unexpectedly".** Tu PC no tiene bastante
RAM libre. Cierra otros programas (los navegadores gastan mucha) o elige un tamaño más pequeño, como Q2_0 o IQ2_XS.
- **Dice que no hay motor para este PC.** En la aplicación, abre **Ajustes, Motor** y pulsa **Actualizar motor**. En la
Terminal, ejecuta `strata update`. `strata doctor` muestra qué archivos del motor están instalados.
- **Dice que el puerto 8080 ya está en uso.** Strata ya se está ejecutando. Busca su ventana.
Más problemas y soluciones: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). ¿Sigues atascado? Abre una
[incidencia](https://github.com/Niko1221/Strata/issues). En la aplicación, adjunta el archivo de **Ajustes, Motor,
Diagnóstico, Ver registros, Guardar registros**. En la Terminal, pega la salida de `strata doctor`. Si lo ejecutas desde el
código fuente, adjunta `strata-<model>.log` de la carpeta de Strata. Para un problema de seguridad, avísanos en privado:
[SECURITY.md](SECURITY.md).
## Créditos y licencia
El modelo es [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), del equipo de Qwen. Lo
comprimieron [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5)
El modelo es [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), del equipo de Qwen. Lo comprimieron
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5), agentionai (Gyro)
y Unsloth. Strata usa partes de [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). Todos los créditos:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata es de código abierto con la [licencia MIT](LICENSE).
Algunas partes y cada modelo tienen sus propias licencias ([cuáles](docs/HOW_IT_WORKS.md#license)).
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata es de código abierto con la [licencia MIT](LICENSE). Algunas
partes y todos los modelos tienen sus propias licencias ([cuáles](docs/HOW_IT_WORKS.md#license)).
## Apoya a Strata
## Contribuir
Strata es libre y de código abierto. Si te resulta útil, puedes apoyar su desarrollo:
¿Has encontrado un error, o quieres enviar un cambio o un benchmark? Lee primero [CONTRIBUTING.md](CONTRIBUTING.md). Explica
qué incluir para que otros puedan comprobar tu trabajo.
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>
## Apoya Strata
Strata es gratis y tiene licencia MIT. Lo construyo y lo pruebo en el hardware que consigo. Si te funciona bien en tu PC, un café me ayuda
a seguir probando tarjetas nuevas y a arreglar lo que nos cuentas. Sin presión. Usarlo y contar lo que encuentras ayuda igual de mucho.
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
¿Te interesa patrocinar Strata? Escríbeme a través de [GitHub](https://github.com/Niko1221).
+344 -160
View File
@@ -1,221 +1,405 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
</picture>
</p>
[English](README.md) · [简体中文](README.zh-CN.md) · [日本語](README.ja.md) · [Deutsch](README.de.md) · **Français** · [Español](README.es.md) · [Português](README.pt-BR.md)
<p align="center"><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <b>Français</b> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a></p>
<p align="center"><b>Faites tourner un modèle d'IA de 125 milliards de paramètres sur votre propre PC de jeu</b><br>
Carte graphique NVIDIA ou AMD (12 Go ou plus) · Windows ou Linux · gratuit et open source</p>
<h3 align="center">Faites tourner un modèle de pointe sur votre propre PC de jeu</h3>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="Un jardin de pagode en voxels écrit par le modèle de Strata, qui tourne dans le navigateur"></a><br>
<p align="center">Compatible avec NVIDIA, AMD, Intel · Windows, Linux, Mac · gratuit et open source</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>Un jardin de pagode en voxels, généré en une seule requête sur une RTX 5070 avec Strata (IQ3_S, contexte de 128K) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">vidéo complète (49 s)</a></sub></p>
Strata fait tourner **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** sur un PC normal. C'est
un grand modèle d'IA, très capable, qui demande d'habitude un serveur. Il discute, écrit du code, lit des images et
fonctionne avec vos applications et vos agents de code. Rien ne quitte votre PC.
Strata fait tourner **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**, un modèle de 125 milliards
de paramètres qui demande normalement un serveur, sur un PC ordinaire. Il discute, écrit du code, lit des images et
fonctionne avec vos applications et vos agents de code. Vos conversations et vos fichiers restent sur votre PC.
## Quelle est sa vitesse ?
**Nouveau ici ?** Regardez [ce qu'il vous faut](#ce-quil-vous-faut), puis [installez l'application](#option-a--lapplication-strata).
Nous l'avons mesurée sur deux PC de jeu ordinaires. Un token correspond à environ ¾ d'un mot.
- **Tourne sur un PC de jeu.** 32 Go de RAM et de mémoire graphique réunis suffisent pour commencer.
- **Fonctionne avec vos outils.** Il parle les API d'OpenAI et d'Anthropic, donc Claude Code, Codex CLI, OpenCode et d'autres agents peuvent s'y connecter.
- **Lit les images.** C'est activé par défaut.
- **Gratuit.** Licence MIT, pas de compte, pas de cloud.
- **Privé.** Aucune télémétrie. Ce que Strata télécharge, quand il utilise internet et comment il vérifie les téléchargements est expliqué dans [docs/SAFETY.md](docs/SAFETY.md).
- **Écrit les réponses :** la vitesse à laquelle la réponse s'affiche dans une courte discussion. 60 tokens par seconde, c'est plus rapide que votre lecture.
- **Lit votre prompt :** la vitesse à laquelle il absorbe ce que vous envoyez (ici un document, du code ou un historique de discussion de 32K tokens).
<table>
<tr><th>NVIDIA : RTX 5070 (12 Go), Ryzen 5 7600, 64 Go de RAM</th><th>AMD : RX 9070 XT (16 Go), Ryzen 9 3900X, 47 Go de RAM</th></tr>
<tr><td>
| Taille | Écrit les réponses | Lit votre prompt |
| --- | ---: | ---: |
| **Q2_0** | 94 tokens/s | 2,650 tokens/s |
| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
| **Coder** | 55 tokens/s | 2,180 tokens/s |
</td><td>
| Taille | Écrit les réponses | Lit votre prompt |
| --- | ---: | ---: |
| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
| **Coder** | 44 tokens/s | 1,420 tokens/s |
</td></tr>
</table>
NVIDIA : Q2_0 avec le moteur 0.1.36, les autres lignes avec 0.1.26 (réponses de 4K, prompts de 32K). Les tableaux
complets sont dans [DETAILS.md](docs/DETAILS.md#speed-measured). Une carte avec plus de VRAM va plus vite : une
RTX 3090 (24 Go) devrait écrire environ 100-140 tokens par seconde. Longues discussions et autres cartes :
[vitesse de chaque modèle](docs/MODELS.md#how-fast-is-each-size), [résultats de la communauté](docs/COMMUNITY_BENCHMARKS.md).
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
<sub>Strata est gratuit. S'il tourne bien sur votre PC, un café aide à poursuivre le travail.</sub></p>
Version 0.2.1 · moteur 0.1.46
## Ce qu'il vous faut
| | |
**Il vous faut au moins 32 Go de RAM et de mémoire graphique (VRAM) réunis.** Sur un Mac ou un Ryzen AI Max, c'est la
mémoire unifiée. Plus de mémoire permet un modèle plus gros ou plus rapide.
- Un PC de 32 Go avec une carte de 12 à 16 Go fait tourner le Coder.
- Avec 64 Go de RAM, toutes les tailles fonctionnent.
| Votre ordinateur | Détails |
| --- | --- |
| **Carte graphique** | **NVIDIA** GeForce RTX série 20, 30, 40 ou 50, ou **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 ou série RX 6800 / 6900. Il faut **12 Go de VRAM ou plus**. |
| **RAM** | 32 Go ou plus. Votre RAM décide [quel modèle](#quel-modèle-choisir-) tient. Avec 64 Go, toutes les tailles tournent. |
| **Disque** | Environ 80 Go libres. Prenez un SSD si possible : le premier démarrage est bien plus rapide. |
| **Système** | Windows 10 / 11 ou Linux, avec un pilote graphique NVIDIA ou AMD à jour. |
| **NVIDIA** GeForce RTX série 20, 30, 40, 50 (12 Go ou plus) | Installez l'application ou le Terminal. |
| **AMD** Radeon RX 6800 et plus récent, Radeon AI PRO R9700 (12 Go ou plus) | [Cartes AMD](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md), [moteur Intel](docs/INTEL.md) |
| **Mac Apple Silicon** (M1 ou plus récent, macOS 15 ou plus récent) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max (Strix Halo)** et autres systèmes à mémoire unifiée | [Strix Halo](docs/STRIX_HALO.md) |
| Cartes et processeurs plus anciens | [GPU anciens](docs/OLDER_GPUS.md), [CPU anciens](docs/INSTALL.md#older-cpus-experimental) |
L'installateur s'occupe de tout le reste. Deux ou trois cartes peuvent se partager le modèle ([multi-GPU](docs/MULTI_GPU.md)).
Il faut aussi :
Expérimental, écrit et testé par des membres de la communauté sur leurs propres machines :
- environ 80 Go d'espace disque libre (un SSD accélère beaucoup le premier démarrage)
- Windows 10 / 11, Linux ou macOS
- un pilote graphique récent
- **Cartes graphiques plus anciennes** (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT) : [Older GPUs](docs/OLDER_GPUS.md).
- **Intel Arc**, compilé depuis les sources sous Linux : [Intel Arc](docs/INTEL_ARC.md).
- **AMD Ryzen AI Max (Strix Halo)**, compilé depuis les sources sous Linux : [Strix Halo](docs/STRIX_HALO.md).
- **Processeurs plus anciens sans AVX2** : ça marche, mais lentement. [Older CPUs](docs/INSTALL.md#older-cpus-experimental).
La liste complète : [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
Deux ou trois cartes de la même marque peuvent se partager un modèle ([multi-GPU](docs/MULTI_GPU.md)). Strata ne répartit
jamais un modèle entre des cartes NVIDIA et AMD. Liste complète : [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
## Installation
### Laissez votre IA l'installer
Choisissez une option. Elles lancent toutes le même moteur et utilisent les mêmes modèles.
Vous utilisez un assistant de code IA (Claude Code, Cursor, Codex, GitHub Copilot, ...) ? Collez-lui ceci :
### Option A : l'application Strata
Le plus simple. L'application contient le moteur : installez-la, choisissez un modèle, discutez. Téléchargez le fichier
correspondant à votre système et à votre carte graphique depuis la [page des versions](https://github.com/Niko1221/Strata/releases/latest).
Vous ne savez pas lequel prendre ? Lisez le paragraphe sous le tableau.
| Système | Votre carte graphique | Fichier à télécharger |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX, pilote 580 ou plus récent | [`Strata-0.2.1-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda13.exe) |
| | NVIDIA, carte plus ancienne ou pilote antérieur à 580 | [`Strata-0.2.1-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda12.exe) |
| | AMD Radeon série RX 9000 ou Radeon AI PRO R9700 | [`Strata-0.2.1-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip.exe) |
| | AMD Radeon série RX 7000 ou Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX, pilote 580 ou plus récent | [`Strata-0.2.1-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda13.AppImage) |
| | NVIDIA, carte plus ancienne ou pilote antérieur à 580 | [`Strata-0.2.1-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon série RX 9000 ou Radeon AI PRO R9700 | [`Strata-0.2.1-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx9070.AppImage) |
| | AMD Radeon série RX 7000 | [`Strata-0.2.1-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-strixhalo.AppImage) |
| **Mac** (Apple Silicon, macOS 15 ou plus récent) | intégrée | [`Strata-0.2.1-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-mac.dmg) |
**Si vous hésitez :** pour NVIDIA, prenez `cuda13` si la version de votre pilote est 580 ou plus récente, sinon `cuda12`.
`cuda12` convient aux cartes plus anciennes et aux pilotes plus anciens. Les fichiers `hip` et `amd` sont pour les cartes
AMD. Les autres cartes AMD (RX 6000) n'ont pas encore d'application prête à l'emploi : voir
[Cartes AMD](docs/AMD_HIP.md).
Ensuite :
- **Windows :** lancez le `.exe`. Il s'installe pour votre utilisateur et ne demande pas de droits d'administrateur. Si
Windows affiche « Windows a protégé votre ordinateur », choisissez **Informations complémentaires**, puis **Exécuter
quand même** (l'installateur n'est pas encore signé).
- **Linux :** rendez le fichier exécutable (`chmod +x`) et lancez-le. Il a besoin de FUSE 2 : `sudo apt install libfuse2`
(Ubuntu 22.04) ou `libfuse2t64` (Ubuntu 24.04 et plus récent).
- **Mac :** ouvrez le `.dmg` et faites glisser Strata dans Applications.
Sur le premier écran, l'application analyse votre PC et recommande un modèle adapté. Appuyez sur Installer. Le modèle est un
gros téléchargement (environ 40 à 70 Go) qui reprend là où il s'est arrêté en cas d'interruption. Quand vous fermez la
fenêtre, Strata et le moteur s'arrêtent.
### Option B : le Terminal (une ligne)
Le Terminal est la commande `strata`, pensée pour les scripts, les serveurs et les agents d'IA.
Linux et Mac :
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
```
Windows (PowerShell) :
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
Le script analyse votre PC, télécharge le Terminal, vérifie son empreinte SHA-256 et le décompresse dans votre dossier
personnel. Il ne demande pas de droits d'administrateur. Sous Windows, ouvrez ensuite une nouvelle fenêtre de terminal.
Puis lancez Strata :
```sh
strata launch
```
Il pose une question à la fois : modèle, contexte, images, qui peut se connecter. Ensuite il télécharge le moteur pour votre
carte graphique (signé, et lancé à titre d'essai avant usage) ainsi que le modèle, puis commence à servir. Pour les scripts
et les agents d'IA, utilisez des options :
```sh
strata launch --model q2_0 --yes --no-tui
```
Pour lire le script d'installation avant de l'exécuter, téléchargez-le d'abord avec `curl -fsSL <address> -o install.sh`,
lisez-le, puis lancez `sh install.sh`. Tous les écrans et toutes les options : [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md).
### Option C : laissez votre IA l'installer
Collez ceci dans Claude Code, Cursor, Codex, GitHub Copilot ou un autre assistant de programmation :
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
Il vérifie votre carte graphique, votre RAM et votre disque, et choisit le modèle qui convient. Ensuite il l'installe,
le démarre et vous explique comment connecter vos applications. Les outils d'IA peuvent aussi installer, démarrer et
arrêter Strata grâce à son [serveur MCP](docs/MCP_SERVER.md).
L'assistant vérifie votre carte graphique, votre RAM et votre disque, choisit le modèle adapté, l'installe, le démarre et
vous explique comment connecter vos applications. Les outils d'IA peuvent aussi installer, démarrer et arrêter Strata via son
[serveur MCP](docs/MCP_SERVER.md).
### Ou faites-le vous-même
Vous préférez le compiler vous-même ? Voir [Compiler depuis les sources](#compiler-depuis-les-sources).
[Téléchargez Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) et décompressez-le (ou faites un `git clone`).
**Windows :** double-cliquez sur **`START-HERE.bat`**. **Linux :** lancez **`./setup.sh`** dans le dossier Strata.
## Modèles
Les étapes sont les mêmes pour NVIDIA et AMD. L'installateur détecte votre carte et installe le bon moteur pour elle.
Il vous pose quelques questions :
L'application et `strata launch` recommandent un modèle selon votre mémoire. C'est un seul modèle en plusieurs tailles,
plus ou moins compressé. Les petites tailles sont plus rapides. Les grandes sont un peu plus intelligentes.
- quel modèle et quelle taille,
- combien de contexte (la quantité de texte que le modèle garde en tête),
- s'il doit lire les images.
### Peu de mémoire ? Essayez Gyro
Appuyez sur Entrée à chaque fois pour garder la réponse recommandée. Ensuite il télécharge le modèle (environ 70 Go)
et le démarre. Si le téléchargement s'arrête, relancez-le : il reprend là où il s'était arrêté. Votre navigateur
ouvre l'application Strata à l'adresse `http://127.0.0.1:8080`.
[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) (par agentionai) est la façon la plus légère de faire tourner ce modèle.
> **Pendant le démarrage du modèle, votre PC peut ralentir ou ne plus répondre pendant 1-3 minutes** (plus longtemps la première fois).
> Strata charge 35-55 Go dans votre RAM et en réserve une partie pour la carte graphique. C'est normal. Attendez, et
> ne fermez pas la fenêtre. La fenêtre montre ce que fait Strata.
| Taille | Mémoire nécessaire (RAM + VRAM) | Téléchargement | Ce que c'est |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27,6 Go | 58,5 Go | Pour les PC où rien d'autre ne tient. La qualité est inférieure à celle d'IQ2_XS. |
| **Gyro-M** | 35 Go | 92,0 Go | À peu près la qualité d'IQ3_XXS avec environ 20 % de mémoire GPU en moins. Il écrit les réponses 17 à 21 % plus lentement que Q2_0 sur une carte de 12 Go et sur une R9700. |
**La fois suivante**, relancez `START-HERE.bat` (ou `./setup.sh`). Il démarre tout de suite et ne télécharge rien
deux fois. Fermez sa fenêtre pour arrêter le modèle. `UPDATE.bat` (`./update.sh`) met Strata à jour sans le démarrer.
Mises à jour, Docker, plusieurs cartes, emplacement des fichiers et toutes les options : [docs/INSTALL.md](docs/INSTALL.md).
Gyro fonctionne sur les cartes NVIDIA et AMD (expérimental sur AMD), pas sur Intel Arc ni sur Mac. Détails et mesures :
[docs/GYRO.md](docs/GYRO.md). Lancez-le avec `strata launch --model gyro-s`.
## Quel modèle choisir ?
### Les autres tailles
L'installateur en recommande un selon votre RAM. Le même modèle existe en plusieurs tailles, plus ou moins
compressées. Les petites tailles sont plus rapides. Les grandes sont un peu plus intelligentes.
| Votre RAM | Prenez | Pourquoi |
| Votre mémoire (RAM + VRAM) | Prenez | Remarques |
| --- | --- | --- |
| **32 Go** | **Coder** | il tient dans 32 Go et il est fait pour le code (avec une carte de 24 Go, Q2_0 et IQ2_XS tournent aussi) |
| **48 Go** | **IQ2_XS** (ou Q2_0, le plus rapide) | les plus grandes tailles ne tiennent pas |
| **64 Go** | **IQ2_XS** (recommandé), ou IQ3_XXS / IQ3_S | toutes les tailles tiennent ; IQ3_S est le meilleur et le plus lent |
| **96 Go ou plus** | **IQ3_S**, ou l'UD-IQ4_XS d'Unsloth (~4 bits) | de la place pour les plus grandes tailles, avec tout le reste ouvert |
| **32 Go** | **Gyro-S**, ou le **[Coder](docs/MODELS.md#coder)** pour le code | Le Coder demande 32 Go de RAM et garde la moitié des experts, il est donc moins bon hors du code |
| **40 Go** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS (39,2 Go), ou **Gyro-M** (35 Go) | Swift réfléchit moins longtemps avant de répondre. Gyro-M approche la qualité d'IQ3_XXS, mais il écrit les réponses 17 à 21 % plus lentement que Q2_0 |
| **48 Go** | **IQ3_XXS** (47,0 Go) | Plus lent, meilleur |
| **64 Go** | **IQ3_S** (54,8 Go), l'original ou Swift 1.5 | Le plus lent, le meilleur. Il égale le modèle complet dans les tests publiés |
| **96 Go ou plus** | **UD-IQ4_XS** d'Unsloth (environ 72 Go de RAM) jusqu'à **Q8_0** (environ 136 Go), 4 à 8 bits | Plus proche du modèle complet. Avec moins de RAM, Strata lit une partie depuis le SSD, ce qui est beaucoup plus lent |
- **[Coder](docs/MODELS.md#coder) :** une version pour le code, avec la moitié des experts en moins. Elle atteint 91 %
du score SWE-bench Verified du modèle complet (mesuré par ses auteurs) et tient dans 32 Go de RAM. Elle est moins
bonne en dehors du code, y compris pour le chinois et les autres textes CJK (#438). Pour ces cas, prenez Q2_0,
IQ2_XS ou IQ3_S, qui gardent tous les experts.
- **[Swift 1.5](docs/MODELS.md#swift-15) :** un fine-tune qui réfléchit bien moins longtemps avant de répondre. Vous
avez la réponse plus tôt, avec à peu près la même qualité.
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs)** : la version ~4 bits d'Unsloth, entre IQ3_S et
UD-Q4_K_XL en qualité. Un téléchargement de 94 Go. Avec moins de ~80 Go de RAM, Strata en lit une partie depuis
le SSD pendant qu'il répond, il y est donc plus lent (un SSD NVMe aide).
- **[Unsloth UD-Q4_K_XL](docs/MODELS.md#unsloth-ud-q4_k_xl-experimental)** (expérimental) : le plus proche du modèle
complet. Mais Strata en lit la plus grande partie depuis le SSD pendant qu'il répond, donc il n'écrit que
7-8.5 tokens/s sur un PC avec 64 Go.
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs) :** vous l'installez à la main.
Il n'est pas dans le menu de l'installateur.
Avoir moins de RAM que ce qu'indique une ligne peut encore fonctionner : Strata passe tout seul en
[mode peu de RAM](docs/MODELS.md#a-big-graphics-card-and-little-ram). Attendez-vous à plus de lenteur, et à bien plus avec
une petite carte graphique.
Tailles, téléchargements et ce qui tient où : [docs/MODELS.md](docs/MODELS.md). Pour ajouter un autre modèle plus
tard, lancez `SETUP.bat` (Linux : `./setup.sh --setup`).
Autres choix :
- **[Swift 1.5](docs/MODELS.md#swift-15)** est une version affinée qui réfléchit moins longtemps avant de répondre, pour une qualité et un besoin en RAM à peu près identiques.
- **[NVFP4](docs/NVFP4.md)** est la version 4 bits de NVIDIA pour les cartes NVIDIA. Elle demande environ 80 Go de RAM pour garder tous les experts en RAM.
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** est un modèle plus petit et différent, qui lit les images et peut répondre à plusieurs requêtes à la fois.
- **[L'Uncensored IQ3_XXS d'OrcaRouter](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** se configure à la main.
Toutes les tailles, les téléchargements et ce qui tient où : [docs/MODELS.md](docs/MODELS.md).
## Quelle est sa vitesse ?
Mesuré avec une carte graphique de 12 Go (RTX 5070, Ryzen 5 7600, 64 Go de RAM). Voir les résultats d'autres cartes :
[vitesse de chaque taille](docs/MODELS.md#how-fast-is-each-size), [résultats de la communauté](docs/COMMUNITY_BENCHMARKS.md).
| Modèle | Les réponses s'affichent à | Lit votre prompt à |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2 650 tokens/s |
| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2 090 tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1 750 tokens/s |
| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1 620 tokens/s |
| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2 180 tokens/s |
Un token vaut environ ¾ de mot, donc 60 tokens par seconde, c'est plus rapide que votre lecture. « Lit votre prompt »
indique la vitesse à laquelle Strata absorbe un long document, du code ou un historique de discussion.
Mesuré avec une carte AMD de 16 Go (RX 9070 XT, Ryzen 9 3900X, 47 Go de RAM) :
| Modèle | Les réponses s'affichent à | Lit votre prompt à |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1 160 tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1 110 tokens/s |
| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1 420 tokens/s |
Les barres utilisent la même échelle dans les deux tableaux. Une carte avec plus de VRAM est plus rapide. Une RTX 3090
(24 Go) devrait écrire environ 100 à 140 tokens par seconde. La version du moteur derrière chaque chiffre est indiquée dans
[DETAILS.md](docs/DETAILS.md#speed-measured).
## Utilisation
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="L'onglet Monitor de l'application Strata à côté d'un agent de code"><br>
<sub>Le <b>Monitor</b> de l'application Strata (à gauche) pendant qu'un agent de code écrit le jardin de pagode de la vidéo (à droite)</sub></p>
### Dans l'application
- **Dans le navigateur :** ouvrez `http://127.0.0.1:8080`. Vous y trouvez **Chat**, un **Monitor** en direct du
modèle et de votre GPU/CPU/RAM, et **About** avec les réglages et les adresses.
- **Vos applications et agents de code :** ajoutez un fournisseur « OpenAI-compatible » avec l'URL de base
**`http://127.0.0.1:8080/v1`**. N'importe quelle clé API et n'importe quel nom de modèle fonctionnent.
- Applications qui utilisent l'API d'Anthropic : `http://127.0.0.1:8080/v1/messages` (Claude Code :
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`).
- Codex CLI et les autres applications qui utilisent l'API OpenAI Responses : `/v1/responses`
([configuration](docs/DETAILS.md#the-responses-api-and-codex-cli)).
- **Réflexion :** choisissez **off, low, medium ou high** dans le menu du chat ou dans le « reasoning effort » de
votre application. Off est le plus rapide. High est le meilleur pour les questions difficiles.
- **Images :** répondez oui à « Images? » pendant l'installation. Ensuite cliquez sur **Picture** dans le chat, ou
joignez des images dans votre application. Les cartes AMD lisent les images sous Linux via le processeur ; sous
Windows, pas encore.
- **Depuis votre téléphone ou un autre PC :** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`. Définissez
toujours une clé.
- **Une requête à la fois :** par défaut, Strata répond à une requête et les autres attendent. Pour répondre à
plusieurs en même temps, mettez `"parallel": 2` ([BATCHING.md](docs/BATCHING.md)). Sur une carte de 12 Go, chaque
réponse devient alors plus lente.
- **Longs prompts :** Strata lit en entier le premier message d'une discussion, environ 1 minute pour 30,000 tokens.
Les messages suivants démarrent en quelques secondes.
L'application comprend un **Chat**, un **Monitor** du modèle, de votre GPU, de votre CPU et de votre RAM, une page **Models** et les **Settings**. Le chat peut travailler dans un dossier de votre PC. En mode **Manual**, il demande
avant de modifier un fichier ou d'exécuter une commande. Tapez `/loop 10m <what to do>` pour répéter un message toutes les
10 minutes tant que le chat est ouvert.
Plus d'infos : [où sont stockées vos discussions](docs/INSTALL.md#where-things-are-stored), [l'API](docs/DETAILS.md#using-it).
Le chat peut chercher sur le web avec DuckDuckGo (ou Bing), sans compte. Vous pouvez ajouter une clé de recherche sous
**Settings, Tools**. La recherche est la seule fonction qui envoie quelque chose hors de votre PC (les mots recherchés).
## Un problème ?
Pour faire tourner deux modèles, activez **Run multiple models** (exécuter plusieurs modèles) et donnez à chacun sa propre carte de la même marque.
- **Mon PC s'est figé au premier démarrage de Strata.** C'est normal pendant le chargement du modèle. Attendez, et ne
fermez pas la fenêtre. Toujours figé après 10 minutes ? Redémarrez le PC, fermez les autres programmes et
réessayez, ou choisissez une taille plus petite.
- **Il s'est arrêté pendant le téléchargement ou l'installation.** Relancez `START-HERE.bat` (ou `./setup.sh`). Il
reprend là où il s'était arrêté.
- **C'est très lent et le voyant du disque clignote sans arrêt, ou il affiche « the engine stopped unexpectedly ».**
Votre PC n'a pas assez de RAM libre. Fermez les autres programmes (les navigateurs en utilisent beaucoup), ou
choisissez une taille plus petite (Q2_0 ou IQ2_XS).
- **Il dit que le port 8080 est déjà utilisé.** Strata tourne déjà. Cherchez sa fenêtre.
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>Une discussion dans l'app Strata : l'agent a écrit un petit jardin de pagode en voxels dans un seul fichier HTML, et la page s'ouvre dans l'aperçu à côté de la conversation</sub></p>
Autres problèmes et leurs solutions : [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Toujours bloqué ? Ouvrez une
[issue](https://github.com/Niko1221/Strata/issues) et joignez `strata-<model>.log` depuis le dossier Strata. Vous
avez trouvé un problème de sécurité ? Signalez-le en privé : [SECURITY.md](SECURITY.md).
### Dans le Terminal
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub><code>strata monitor</code> pendant que le modèle répond : vitesse, GPU, mémoire et les dernières requêtes</sub></p>
```sh
strata launch # guided screens: model, context, pictures, access, then serve
strata launch --model q2_0 --yes --no-tui # no questions: serve, print the address, return
strata launch --plan --model q2_0 # show what would happen; nothing changes
strata launch --status # what is being served now
strata launch --stop # stop it
strata monitor --url http://127.0.0.1:8080 # watch speed, GPU, memory and requests
strata update --check # is there a new engine? (strata update installs it)
strata doctor # check the installed engine files
```
`strata --help` liste tout. Démarrer et servir ne téléchargent rien. Seuls l'installation, `strata launch` (quand vous
choisissez un modèle) et `strata update` le font. Pour joindre Strata depuis votre téléphone ou un autre PC, utilisez
`strata launch --access lan`. La commande crée une clé d'API et refuse de démarrer sur un réseau sans elle. Aucun
navigateur ne s'ouvre, sauf si vous ajoutez `--open`. Guide : [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). Le client complet : [docs/TERMINAL.md](docs/TERMINAL.md).
### Avec des agents d'IA
Un agent installe Strata avec la commande en une ligne de l'option B et lance
`strata launch --model NAME --yes --no-tui`. Les étapes, les options et les codes de sortie sont dans [docs/AI_SETUP.md](docs/AI_SETUP.md),
et les règles du dépôt dans [AGENTS.md](AGENTS.md).
### Connecter vos autres applications
| Type d'application | URL de base |
| --- | --- |
| Compatible OpenAI (la plupart des applications et des agents) | `http://127.0.0.1:8080/v1` (n'importe quelle clé d'API, n'importe quel nom de modèle) |
| API Anthropic (Claude Code et d'autres) | `http://127.0.0.1:8080`, sans `/v1`. Ces applications ajoutent `/v1/messages` elles-mêmes. Claude Code : `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| API OpenAI Responses (Codex CLI) | `http://127.0.0.1:8080/v1/responses` ([configuration](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` affiche les réglages exacts pour OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent et OpenClaw.
`strata connect opencode --write` écrit le fichier après confirmation et garde une sauvegarde.
### Images
Les images sont activées par défaut pour les modèles qui savent les lire. Joignez ou collez une image dans le chat, ou
envoyez-la depuis votre application. Le lecteur d'images tourne sur le processeur quand votre carte graphique a moins de
24 Go (quelques secondes par image, sans prendre de mémoire à la carte) et sur la carte graphique à partir de 24 Go. La
règle est la même pour NVIDIA et AMD. Vous pouvez désactiver les images par modèle, ou choisir le lecteur avec
`--vision off`, `--vision gpu` ou `--vision cpu`.
Les chiffres derrière la règle : sur une Radeon AI PRO R9700, le lecteur de la carte a pris 57 ms pour une image de
260 tokens et celui du processeur 3,1 s. Sur des cartes de 16 Go (Tesla P100 et RTX A4000, Linux), le lecteur de la carte
a coûté 3 à 5 % de vitesse sur le texte, parce qu'il occupe environ 1,4 Go que le modèle utiliserait sinon. Sur un Mac, le lecteur tourne sur le GPU quelle que soit la mémoire (0,45 s pour une image de 260 tokens sur un M4 de 32 Go, environ 1,2 Go de la
mémoire partagée) ; le processeur met environ 50 s par image, il ne lit donc qu'avec `--vision cpu`.
### Bon à savoir
- Choisissez le niveau de réflexion (**désactivé, faible, moyen ou élevé**) dans le menu du chat ou dans le réglage « reasoning effort » de votre application. Désactivé est le plus rapide.
- Strata répond à une requête à la fois par défaut. Pour en traiter plusieurs en même temps, utilisez `--parallel N` dans `strata launch` ou
`"parallel": 2` dans la configuration ([BATCHING.md](docs/BATCHING.md)). Sur une carte de 12 Go, chaque réponse devient alors plus lente.
- Strata lit en entier le premier message d'une discussion, soit environ 1 minute pour 30 000 tokens. Les messages suivants démarrent en quelques secondes.
- Où sont stockées vos discussions et l'API complète : [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored), [docs/DETAILS.md](docs/DETAILS.md#using-it).
## Compiler depuis les sources
<details>
<summary>Windows, Linux et Mac depuis le dépôt</summary>
[Téléchargez Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) et décompressez-le, ou faites un `git clone`.
**Windows :** double-cliquez sur `START-HERE.bat`. **Linux :** lancez `./setup.sh` dans le dossier Strata.
**Mac (Apple Silicon, macOS 15 ou plus récent) :** installez une fois les Command Line Tools d'Apple avec `xcode-select --install`
(Xcode lui-même n'est pas nécessaire). Ouvrez le Terminal dans le dossier Strata et lancez `./setup.sh`. Le script compile
le moteur Mac sur votre Mac (environ 7 minutes), pose les mêmes questions, télécharge le modèle et le démarre. La fois
suivante, `./setup.sh` le démarre tout de suite. Quelle taille convient à votre Mac : [docs/MAC.md](docs/MAC.md).
L'installateur détecte votre carte et prépare le bon moteur. Il demande quel modèle et quelle taille, combien de contexte
(la quantité de texte que le modèle garde en tête) et s'il doit lire les images. Appuyez sur Entrée pour la réponse
recommandée. Puis il télécharge le modèle (environ 70 Go) et le démarre. Si le téléchargement s'arrête, relancez le script
et il reprend. Votre navigateur ouvre la page de Strata à l'adresse `http://127.0.0.1:8080`.
> Pendant le démarrage du modèle, votre PC peut être lent ou ne plus répondre pendant 1 à 3 minutes (le plus longtemps la
> première fois). Strata charge 35 à 55 Go dans votre RAM et en verrouille une partie pour la carte graphique. Patientez et
> ne fermez pas la fenêtre.
La fois suivante, relancez `START-HERE.bat` (ou `./setup.sh`). Il démarre tout de suite et ne télécharge rien deux fois.
Fermez sa fenêtre pour arrêter le modèle. `UPDATE.bat` (`./update.sh`) met Strata à jour sans le démarrer. Mise à jour,
Docker, plusieurs cartes, où vont les fichiers et toutes les options : [docs/INSTALL.md](docs/INSTALL.md).
Pour mesurer votre PC, lancez `BENCH.bat` (`./bench.sh`). Cela prend 5 à 10 minutes et produit un rapport que vous pouvez
partager ([comment](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).
**Facultatif : plus de vitesse sur une carte quand tout le modèle tient en RAM.** Deux réglages à activer soi-même, par
RubenTorney (#1904, #1905). Ils aident quand vous utilisez une seule carte graphique et que votre RAM contient tous les
experts, par exemple Q2_0 ou IQ3_XXS sur un PC de 64 Go. Ouvrez `strata-<model>.json` dans le dossier de données de Strata ([où](docs/INSTALL.md#where-things-are-stored)) et ajoutez :
- `"--adapt-async", "1"` à la liste `"args"`. Le cache des experts se met alors à jour sur un thread auxiliaire au lieu de
mettre la réponse en pause. Mesuré sur une RTX 5070 (Windows, Q2_0) : +4 à 5 % en décodage, +7 à 10 % sur le code et les
longs documents. Les réponses peuvent différer légèrement de celles du réglage par défaut, et restent identiques d'une exécution à l'autre.
- `"STRATA_PIN_BY_NEED": "1"` dans le bloc `"env"` (Windows). Windows ne peut verrouiller qu'environ la moitié de la RAM
pour la carte graphique, et ce réglage verrouille d'abord les experts dont la carte a le plus besoin. Mesuré sur la RTX 5070 : +4,4 % en lecture du prompt.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
Redémarrez Strata après la modification. Pour annuler, supprimez les deux entrées.
</details>
## Comment ça marche ?
Les modèles comme celui-ci tournent d'habitude sur des serveurs avec des centaines de gigaoctets de mémoire
graphique. Votre carte graphique a 12-24 Go. Strata fait tenir le modèle en **répartissant le travail sur tout votre
PC**. Pensez à une cuisine : ce que vous utilisez tout le temps reste sur le plan de travail, et le reste attend dans
le placard.
Les modèles de ce genre tournent d'habitude sur des serveurs qui disposent de centaines de gigaoctets de mémoire graphique.
Votre carte graphique en a 12 à 24. Strata fait tenir le modèle en répartissant le travail sur tout votre PC. Imaginez une
bibliothèque.
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="Les 24,576 experts du modèle : les plus sollicités sur la carte graphique, tous en RAM, une table de correspondance sur le SSD"></p>
- Le modèle est une équipe de 24 576 petits spécialistes (les « experts »), comme 24 576 ouvrages de référence. Chaque mot n'en demande que 10.
- **Votre carte graphique est le bureau.** Il est petit et rapide. Les quelque 4 000 livres les plus utilisés y sont ouverts.
- **Votre RAM, ce sont les étagères de la salle de lecture.** Les 24 576 livres s'y trouvent. Quand un mot demande un livre
qui n'est pas sur le bureau, votre processeur va le chercher aussitôt sur l'étagère, pendant que la carte continue de travailler.
- **Votre SSD, ce sont les archives.** Il contient une table de consultation de 29 Go. Le modèle n'en lit que quelques lignes par
mot, elle n'a donc pas besoin de tenir en RAM.
- **Le modèle est une équipe de 24,576 petits spécialistes (les « experts »).** Chaque mot n'en demande que 10.
- **Votre carte graphique** garde les quelques milliers d'experts les plus utilisés. **Votre RAM** les contient tous,
et **votre processeur** travaille sur les autres en même temps. **Votre SSD** contient une grande table de
correspondance.
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="Un petit assistant devine les mots suivants ; le grand modèle les vérifie tous d'un coup et garde les bons"></p>
Un petit assistant devine aussi les mots suivants. Le grand modèle les vérifie tous d'un coup et garde les bons. Vous
obtenez la même réponse, 1,6 à 1,8 fois plus tôt. Les textes longs sont lus par gros morceaux, jusqu'à 8 192 tokens à la
fois, à plus de 1 000 tokens par seconde.
- **Deviner, puis vérifier :** un petit assistant devine les quelques mots suivants. Le grand modèle les vérifie
tous d'un coup. Vous obtenez la même réponse, 1.6-1.8x plus tôt.
- **Les longs textes sont lus par gros morceaux** (jusqu'à 8,192 tokens à la fois), à plus de 1,000 tokens par seconde.
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
L'explication plus longue : [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Chaque partie et ses chiffres :
L'explication plus longue : [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Chaque élément avec ses chiffres :
[les détails](docs/DETAILS.md#how-it-works) et l'[article](docs/paper/Strata-Paper.pdf).
## Crédits et licence
## Un problème ?
Le modèle est [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), de l'équipe Qwen. Il a été
compressé par [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5)
- **Mon PC s'est figé au premier démarrage de Strata.** Il charge le modèle en RAM. Patientez et ne fermez pas la fenêtre.
Toujours figé après 10 minutes ? Redémarrez le PC, fermez les autres programmes et réessayez, ou choisissez une taille plus petite.
- **Ça s'est arrêté pendant le téléchargement ou l'installation.** Relancez l'application ou `strata launch`, ou relancez
`START-HERE.bat` (ou `./setup.sh`). Cela reprend là où ça s'était arrêté.
- **C'est très lent, le voyant du disque clignote sans arrêt, ou il affiche « the engine stopped unexpectedly ».** Votre PC
n'a pas assez de RAM libre. Fermez d'autres programmes (les navigateurs en consomment beaucoup) ou choisissez une taille plus petite, comme Q2_0 ou IQ2_XS.
- **Il indique qu'il n'y a pas de moteur pour ce PC.** Dans l'application, ouvrez **Settings, Engine** et appuyez sur **Update engine** (mettre à jour le moteur). Dans le Terminal, lancez `strata update`. `strata doctor` montre quels fichiers du moteur sont installés.
- **Il indique que le port 8080 est déjà utilisé.** Strata tourne déjà. Cherchez sa fenêtre.
Autres problèmes et solutions : [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Toujours bloqué ? Ouvrez une
[issue](https://github.com/Niko1221/Strata/issues). Dans l'application, joignez le fichier obtenu par **Settings, Engine, Diagnostics, View logs, Save logs**. Dans le Terminal, collez la sortie de `strata doctor`. Si
vous avez lancé depuis les sources, joignez `strata-<model>.log` du dossier Strata. Pour un problème de sécurité,
signalez-le en privé : [SECURITY.md](SECURITY.md).
## Remerciements et licence
Le modèle est [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), de l'équipe Qwen. Il a été compressé par
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5), agentionai (Gyro)
et Unsloth. Strata utilise des parties de [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). Tous les
crédits : [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata est open source sous [licence MIT](LICENSE).
Quelques parties et chaque modèle ont leur propre licence ([lesquelles](docs/HOW_IT_WORKS.md#license)).
remerciements : [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata est open source sous [licence MIT](LICENSE).
Quelques parties et tous les modèles ont leurs propres licences ([lesquelles](docs/HOW_IT_WORKS.md#license)).
## Contribuer
Vous avez trouvé un bug, ou vous voulez proposer une modification ou un benchmark ? Lisez d'abord [CONTRIBUTING.md](CONTRIBUTING.md).
Il indique ce qu'il faut inclure pour que d'autres puissent vérifier votre travail.
## Soutenir Strata
Strata est gratuit et open source. S'il vous est utile, vous pouvez soutenir son développement :
Strata est gratuit et sous licence MIT. Je le construis et je le teste sur le matériel que je peux obtenir. S'il tourne bien sur votre PC, un café m'aide
à continuer de tester de nouvelles cartes et à corriger ce que vous signalez. Aucune obligation. L'utiliser et signaler ce que vous constatez aide tout autant.
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
Vous souhaitez sponsoriser Strata ? Contactez-moi via [GitHub](https://github.com/Niko1221).
+332 -158
View File
@@ -1,211 +1,385 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
</picture>
</p>
[English](README.md) · [简体中文](README.zh-CN.md) · **日本語** · [Deutsch](README.de.md) · [Français](README.fr.md) · [Español](README.es.md) · [Português](README.pt-BR.md)
<p align="center"><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <b>日本語</b> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a></p>
<p align="center"><b>1,250 億パラメータの AI モデルを、手元のゲーミング PC で動かす</b><br>
NVIDIA または AMD のグラフィックカード(12 GB 以上) · Windows または Linux · 無料のオープンソース</p>
<h3 align="center">最先端のモデルを、手元のゲーミングPCで動かす</h3>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="Strata のモデルが書いたボクセルの五重塔の庭。ブラウザで動いている様子"></a><br>
<sub>ボクセルの五重塔の庭。1 回のプロンプトで作成し、RTX 5070 上の Strata(IQ3_S、128K コンテキスト)で実行 ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">動画全体(49 秒)</a></sub></p>
<p align="center">NVIDIA、AMD、Intel に対応 · Windows、Linux、Mac · 無料のオープンソース</p>
Strata は **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** を普通の PC で動かします。
これは大きくて賢い AI モデルで、ふつうはサーバーが必要です。チャットをしたり、コードを書いたり、画像を読んだりできます。
アプリやコーディングエージェントとも連携します。データが PC の外に出ることはありません。
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
</p>
## どのくらい速い?
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>ボクセルの塔の庭園。1回のプロンプトで生成し、RTX 5070 上の Strata で動かしています(IQ3_S、128K コンテキスト) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">動画全体(49秒)</a></sub></p>
ごく普通のゲーミング PC 2 台で測りました。1 トークンは英単語のおよそ ¾ です。
Strata は **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** を動かします。1250億パラメータのモデルで、
ふつうはサーバーが必要ですが、Strata なら一般的なPCで使えます。チャット、コード作成、画像の読み取りができ、
お使いのアプリやコーディングエージェントとも連携します。チャットとファイルはPCの外に出ません。
- **回答の書き出し:** 短いチャットで返答が表示される速さです。毎秒 60 トークンあれば、読むより速く表示されます。
- **プロンプトの読み込み:** 送った内容を取り込む速さです(ここでは 32K トークンの文書、コード、またはチャット履歴)。
**はじめての方へ。** まず[必要な環境](#必要な環境)を確認して、[アプリをインストール](#方法a-strata-アプリ)してください。
<table>
<tr><th>NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM</th><th>AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM</th></tr>
<tr><td>
- **ゲーミングPCで動きます。** RAMとグラフィックスメモリの合計が32 GBあれば始められます。
- **お使いのツールと連携します。** OpenAI と Anthropic の API に対応しているので、Claude Code、Codex CLI、OpenCode などのエージェントから接続できます。
- **画像を読めます。** 初期設定でオンです。
- **無料です。** MIT ライセンス。アカウントもクラウドも不要です。
- **プライバシーを守ります。** テレメトリはありません。Strata が何をダウンロードするか、いつインターネットを使うか、ダウンロードをどう検証するかは [docs/SAFETY.md](docs/SAFETY.md) に書いてあります。
| サイズ | 回答の書き出し | プロンプトの読み込み |
| --- | ---: | ---: |
| **Q2_0** | 94 tokens/s | 2,650 tokens/s |
| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
| **Coder** | 55 tokens/s | 2,180 tokens/s |
バージョン 0.2.1 · エンジン 0.1.46
</td><td>
## 必要な環境
| サイズ | 回答の書き出し | プロンプトの読み込み |
| --- | ---: | ---: |
| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
| **Coder** | 44 tokens/s | 1,420 tokens/s |
**RAMとグラフィックスメモリ(VRAM)の合計が、少なくとも32 GB必要です。** MacやRyzen AI Maxでは、ユニファイドメモリがその量にあたります。
メモリが多いほど、大きなモデルを使えたり、速く動いたりします。
</td></tr>
</table>
- RAM 32 GBのPCに12〜16 GBのグラフィックスカードがあれば、Coder が動きます。
- RAM 64 GBなら、どのサイズも動きます。
NVIDIA:Q2_0 はエンジン 0.1.36、ほかの行は 0.1.26 で測定(回答 4K、プロンプト 32K)。表の全体は
[DETAILS.md](docs/DETAILS.md#speed-measured) にあります。VRAM が多いカードほど速くなります。RTX 3090(24 GB)なら
毎秒およそ 100-140 トークンで書き出せるはずです。長いチャットやほかのカードについて:[モデルごとの速度](docs/MODELS.md#how-fast-is-each-size)、
[コミュニティの測定結果](docs/COMMUNITY_BENCHMARKS.md)。
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
<sub>Strata は無料です。あなたの PC でうまく動いたら、コーヒー 1 杯の支援が開発を続ける力になります。</sub></p>
## 必要なもの
| | |
| お使いのPC | 詳細 |
| --- | --- |
| **グラフィックカード** | **NVIDIA** GeForce RTX 20、30、40、50 シリーズ、または **AMD** Radeon RX 7900 XT / XTX、RX 7800 XT / 7700 XT、RX 9060 XT、RX 9070 / 9070 XT、Radeon AI PRO R9700、RX 6800 / 6900 シリーズ。**VRAM 12 GB 以上**が必要です。 |
| **RAM** | 32 GB 以上。RAM の量で[使えるモデル](#どのモデルを選べばいい)が決まります。64 GB あればすべてのサイズが動きます。 |
| **ディスク** | 空き容量が約 80 GB。できれば SSD を使ってください。初回の起動がずっと速くなります。 |
| **OS** | Windows 10 / 11 または Linux。NVIDIA または AMD の最新のグラフィックドライバー。 |
| **NVIDIA** GeForce RTX 20、30、40、50 シリーズ(12 GB以上) | アプリまたはターミナル版をインストールしてください。 |
| **AMD** Radeon RX 6800 以降、Radeon AI PRO R9700(12 GB以上) | [AMD のカード](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md)、[Intel エンジン](docs/INTEL.md) |
| **Apple Silicon の Mac**(M1 以降、macOS 15 以降) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max(Strix Halo)** などのユニファイドメモリ搭載機 | [Strix Halo](docs/STRIX_HALO.md) |
| 古いカードやプロセッサ | [古いGPU](docs/OLDER_GPUS.md)、[古いCPU](docs/INSTALL.md#older-cpus-experimental) |
ほかに必要なものはすべてインストーラーが用意します。2 枚や 3 枚のカードでモデルを分担することもできます([マルチ GPU](docs/MULTI_GPU.md))。
そのほかに必要なもの:
試験的なサポート。コミュニティのメンバーが自分のマシンで書き、テストしたものです:
- 約80 GBの空きディスク容量(SSDなら初回の起動がずっと速くなります)
- Windows 10 / 11、Linux、または macOS
- 最新のグラフィックスドライバー
- **古いグラフィックカード**(Tesla P40 / V100、GTX 10、Radeon VII / MI50、RX 6700 XT、RX 5500 XT):[Older GPUs](docs/OLDER_GPUS.md)。
- **Intel Arc**(Linux でソースからビルド):[Intel Arc](docs/INTEL_ARC.md)。
- **AMD Ryzen AI Max(Strix Halo)**(Linux でソースからビルド):[Strix Halo](docs/STRIX_HALO.md)。
- **AVX2 のない古いプロセッサー**:動きますが、遅いです。[Older CPUs](docs/INSTALL.md#older-cpus-experimental)。
全リスト:[docs/INSTALL.md](docs/INSTALL.md#what-you-need)。
同じメーカーのカードを2〜3枚使って、1つのモデルを動かせます([マルチGPU](docs/MULTI_GPU.md))。NVIDIA と AMD のカードにまたがってモデルを分けることはありません。
一覧は [docs/INSTALL.md](docs/INSTALL.md#what-you-need) にあります。
## インストール
### AI にセットアップしてもらう
3つの方法から1つ選んでください。どれも同じエンジンを起動し、同じモデルを使います。
AI コーディングアシスタント(Claude Code、Cursor、Codex、GitHub Copilot など)を使っていますか? これを貼り付けてください:
### 方法A: Strata アプリ
いちばん簡単な方法です。エンジンはアプリに入っているので、インストールしてモデルを選べばチャットを始められます。お使いのOSとグラフィックスカードに合うファイルを
[リリースページ](https://github.com/Niko1221/Strata/releases/latest)からダウンロードしてください。どれか分からない場合は、表の下の説明を見てください。
| OS | グラフィックスカード | ダウンロードするファイル |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX、ドライバー 580 以降 | [`Strata-0.2.1-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda13.exe) |
| | NVIDIA、古いカード、またはドライバーが 580 より古い場合 | [`Strata-0.2.1-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda12.exe) |
| | AMD Radeon RX 9000 シリーズまたは Radeon AI PRO R9700 | [`Strata-0.2.1-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip.exe) |
| | AMD Radeon RX 7000 シリーズまたは Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX、ドライバー 580 以降 | [`Strata-0.2.1-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda13.AppImage) |
| | NVIDIA、古いカード、またはドライバーが 580 より古い場合 | [`Strata-0.2.1-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon RX 9000 シリーズまたは Radeon AI PRO R9700 | [`Strata-0.2.1-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx9070.AppImage) |
| | AMD Radeon RX 7000 シリーズ | [`Strata-0.2.1-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-strixhalo.AppImage) |
| **Mac**(Apple Silicon、macOS 15 以降) | 内蔵 | [`Strata-0.2.1-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-mac.dmg) |
**迷ったときは:** NVIDIA の場合、ドライバーがバージョン 580 以降なら `cuda13`、それより古ければ `cuda12` を選びます。`cuda12` は古いカードと古いドライバー向けです。
`hip` と `amd` のファイルは AMD のカード用です。それ以外の AMD カード(RX 6000)には、まだ用意されたアプリがありません。
[AMD のカード](docs/AMD_HIP.md)を見てください。
そのあとの手順:
- **Windows:** `.exe` を実行します。ユーザー単位でインストールされ、管理者権限は不要です。「Windows によってPCが保護されました」と表示されたら、
**詳細情報**を選び、**実行**を選んでください(インストーラーにはまだコード署名がありません)。
- **Linux:** ファイルを実行可能にして(`chmod +x`)起動します。FUSE 2 が必要です。`sudo apt install libfuse2`(Ubuntu 22.04)
または `libfuse2t64`(Ubuntu 24.04 以降)を実行してください。
- **Mac:** `.dmg` を開き、Strata を「アプリケーション」にドラッグします。
最初の画面で、アプリがPCを調べて合うモデルをすすめます。インストールを押してください。モデルは大きく(約40〜70 GB)、中断しても続きからダウンロードできます。
ウィンドウを閉じると、Strata とエンジンが止まります。
### 方法B: ターミナル(1行)
ターミナル版は `strata` コマンドです。スクリプトやサーバー、AI エージェント向けです。
Linux と Mac:
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
```
Windows(PowerShell):
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
スクリプトはPCを調べ、ターミナル版をダウンロードし、SHA-256 を検証して、ホームフォルダに展開します。管理者権限は不要です。Windows では、終わったら
新しいターミナルを開いてください。そのあと Strata を起動します。
```sh
strata launch
```
質問は1つずつ表示されます。モデル、コンテキスト、画像、接続を許可する相手の順です。そのあと、お使いのグラフィックスカード用のエンジン
(署名付きで、使う前にテスト起動します)とモデルをダウンロードして、サーバーを開始します。スクリプトや AI エージェントからは、フラグを使います。
```sh
strata launch --model q2_0 --yes --no-tui
```
実行前にインストールスクリプトを読みたい場合は、先に `curl -fsSL <address> -o install.sh` でダウンロードし、内容を確認してから
`sh install.sh` を実行してください。すべての画面とフラグは [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md) にあります。
### 方法C: AI にセットアップを任せる
Claude Code、Cursor、Codex、GitHub Copilot などのコーディングアシスタントに、次の文を貼り付けます。
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
AI がグラフィックカード、RAM、ディスクを調べて、合うモデルを選びます。それからインストールして起動し、
アプリのつなぎ方を教えてくれます。AI ツールは Strata の [MCP サーバー](docs/MCP_SERVER.md) を通じて、
Strata のインストール、起動、停止もできます。
AI はグラフィックスカード、RAM、ディスクを調べ、合うモデルを選んで、インストールと起動をし、アプリの接続方法を教えてくれます。
AI ツールは [MCP サーバー](docs/MCP_SERVER.md)を通じて、Strata のインストール、起動、停止もできます。
### 自分でやる
自分でビルドしたい方は[ソースからビルド](#ソースからビルド)を見てください。
[Strata をダウンロード](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip)して展開します(または `git clone` します)。
**Windows:** **`START-HERE.bat`** をダブルクリック。**Linux:** Strata フォルダーで **`./setup.sh`** を実行します。
## モデル
手順は NVIDIA でも AMD でも同じです。インストーラーがカードを見つけて、それに合うエンジンを用意します。
いくつか質問があります:
アプリと `strata launch` は、メモリ量に合うモデルをすすめます。モデルは1つで、圧縮の度合いが違う複数のサイズがあります。
小さいサイズは速く、大きいサイズは少し賢くなります。
- どのモデルを、どのサイズで使うか
- コンテキストをどのくらいにするか(モデルが覚えておけるテキストの量)
- 画像を読ませるかどうか
### メモリが少ないなら Gyro
毎回 Enter を押せば、おすすめの答えになります。そのあとモデル(約 70 GB)をダウンロードして起動します。
ダウンロードが止まったら、もう一度実行してください。続きから再開します。ブラウザーで Strata アプリが
`http://127.0.0.1:8080` に開きます。
[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF)(agentionai 製)は、このモデルをいちばん小さく動かせる方法です。
> **モデルの起動中、1-3 分ほど PC が重くなったり、反応しなくなったりすることがあります**(初回がいちばん長いです)。
> Strata は 35-55 GB を RAM に読み込み、その一部をグラフィックカード用に固定します。これは正常です。
> ウィンドウを閉じずに待ってください。Strata が何をしているかはウィンドウに表示されます。
| サイズ | 必要なメモリ(RAM + VRAM) | ダウンロード | 内容 |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27.6 GB | 58.5 GB | ほかのサイズが入らないPC向け。品質は IQ2_XS より下がります。 |
| **Gyro-M** | 35 GB | 92.0 GB | 品質は IQ3_XXS 程度で、GPUメモリの使用量は約20%少なくなります。12 GBのカードと R9700 では、回答の書き出しが Q2_0 より17〜21%遅くなります。 |
**次回からは** `START-HERE.bat`(または `./setup.sh`)をもう一度実行します。すぐに起動し、同じものを 2 回ダウンロードすることはありません。
モデルを止めるときはウィンドウを閉じます。`UPDATE.bat`(`./update.sh`)を使うと、Strata を起動せずに更新できます。
更新、Docker、複数のカード、ファイルの保存場所、すべてのオプションについて:[docs/INSTALL.md](docs/INSTALL.md)。
Gyro は NVIDIA と AMD のカードで動きます(AMD は実験的)。Intel Arc と Mac では動きません。詳細と測定結果は
[docs/GYRO.md](docs/GYRO.md) にあります。`strata launch --model gyro-s` で起動します。
## どのモデルを選べばいい
### ほかのサイズ
インストーラーが RAM に合わせてひとつ勧めます。同じモデルにも、圧縮の強さが違う複数のサイズがあります。
小さいサイズほど速く、大きいサイズほど少し賢くなります。
| RAM | 選ぶもの | 理由 |
| メモリ(RAM + VRAM) | 選ぶモデル | 備考 |
| --- | --- | --- |
| **32 GB** | **Coder** | 32 GB に収まり、コード向けに作られている(24 GB のカードなら Q2_0 と IQ2_XS も動く) |
| **48 GB** | **IQ2_XS**(または最速の Q2_0) | 大きいサイズは収まらない |
| **64 GB** | **IQ2_XS**(おすすめ)、または IQ3_XXS / IQ3_S | すべてのサイズが収まる。IQ3_S がいちばん賢く、いちばん遅い |
| **96 GB 以上** | **IQ3_S**、または Unsloth の UD-IQ4_XS(約 4-bit) | ほかのものを全部開いたままでも、最大のサイズが入る余裕がある |
| **32 GB** | **Gyro-S**、またはコード向けの **[Coder](docs/MODELS.md#coder)** | Coder はRAM 32 GBが必要で、エキスパートを半分だけ保持するため、コード以外は弱くなります |
| **40 GB** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS(39.2 GB)、または **Gyro-M**(35 GB) | Swift は答える前に考える時間が短くなります。Gyro-M は品質が IQ3_XXS に近いものの、回答の書き出しは Q2_0 より17〜21%遅くなります |
| **48 GB** | **IQ3_XXS**(47.0 GB) | 遅くなりますが、品質は上がります |
| **64 GB** | **IQ3_S**(54.8 GB)、オリジナルまたは Swift 1.5 | いちばん遅く、いちばん高品質です。公開されているテストでは完全版と同等です |
| **96 GB 以上** | Unsloth の **UD-IQ4_XS**(RAM 約72 GB)から **Q8_0**(約136 GB)まで、4〜8ビット | 完全版に近づきます。RAM が足りないと、Strata は一部を SSD から読むため、ずっと遅くなります |
- **[Coder](docs/MODELS.md#coder):** エキスパートを半分取り除いたコーディング向け版です。フルモデルの SWE-bench Verified スコアの
91% に届き(作者による測定)、32 GB の RAM に収まります。コード以外では弱く、中国語などの CJK のテキストも苦手です(#438)。
そうした用途には、すべてのエキスパートを残している Q2_0、IQ2_XS、IQ3_S を選んでください。
- **[Swift 1.5](docs/MODELS.md#swift-15):** 答える前に考える時間がずっと短いファインチューン版です。
ほぼ同じ品質のまま、答えが早く返ってきます。
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** Unsloth の約 4-bit 版で、品質は IQ3_S と UD-Q4_K_XL の間です。
ダウンロードは 94 GB。RAM が約 80 GB 未満だと、答えている間に Strata がその一部を SSD から読むので遅くなります(NVMe SSD が有効です)。
- **[Unsloth UD-Q4_K_XL](docs/MODELS.md#unsloth-ud-q4_k_xl-experimental)**(試験的):フルモデルにいちばん近いものです。
ただし答えている間、Strata はその大部分を SSD から読むので、64 GB の PC では毎秒 7-8.5 トークンしか書き出せません。
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** 手動でセットアップします。
インストーラーのメニューにはありません。
表の行より RAM が少なくても動くことがあります。Strata は自動で[低RAMモード](docs/MODELS.md#a-big-graphics-card-and-little-ram)に切り替わります。
速度は落ち、グラフィックスカードが小さいとさらに大きく落ちます。
サイズ、ダウンロード、どれがどこに収まるかについて:[docs/MODELS.md](docs/MODELS.md)。あとで別のモデルを追加するには
`SETUP.bat`(Linux:`./setup.sh --setup`)を実行します。
そのほかの選択肢:
- **[Swift 1.5](docs/MODELS.md#swift-15)** はファインチューン版で、答える前に考える時間が短くなります。品質と RAM の必要量はほぼ同じです。
- **[NVFP4](docs/NVFP4.md)** は NVIDIA カード向けの NVIDIA 製4ビット版です。すべてのエキスパートを RAM に置くには、約80 GBの RAM が必要です。
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** は、より小さい別のモデルです。画像を読め、複数のリクエストに同時に答えられます。
- **[OrcaRouter の Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** は手動で設定します。
すべてのサイズ、ダウンロード、どの構成で何が動くかは [docs/MODELS.md](docs/MODELS.md) にあります。
## どのくらい速い?
12 GBのグラフィックスカード(RTX 5070、Ryzen 5 7600、RAM 64 GB)での数値です。ほかのカードの結果は
[各モデルの速度](docs/MODELS.md#how-fast-is-each-size)、[コミュニティの結果](docs/COMMUNITY_BENCHMARKS.md)を見てください。
| モデル | 回答が出る速さ | プロンプトを読む速さ |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2,650 tokens/s |
| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2,090 tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1,750 tokens/s |
| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1,620 tokens/s |
| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2,180 tokens/s |
1トークンは英単語の約¾にあたるので、毎秒60トークンは読む速さより速くなります。「プロンプトを読む速さ」は、長い文書やコード、チャット履歴を Strata が取り込む速さです。
16 GBの AMD カード(RX 9070 XT、Ryzen 9 3900X、RAM 47 GB)での測定値:
| モデル | 回答が出る速さ | プロンプトを読む速さ |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1,160 tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1,110 tokens/s |
| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1,420 tokens/s |
バーは2つの表で同じ目盛りです。VRAM が多いカードほど速くなり、RTX 3090(24 GB)なら毎秒100〜140トークンほどになるはずです。各数値のエンジンのバージョンは
[DETAILS.md](docs/DETAILS.md#speed-measured) にあります。
## 使い方
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="Strata アプリの Monitor タブと、その横で動くコーディングエージェント"><br>
<sub>Strata アプリの <b>Monitor</b>(左)。コーディングエージェントが動画の五重塔の庭を書いているところ(右)</sub></p>
### アプリで
- **ブラウザーで:** `http://127.0.0.1:8080` を開きます。**Chat**、モデルと GPU/CPU/RAM の様子をリアルタイムで見る **Monitor**、
設定とアドレスが載った **About** があります。
- **アプリやコーディングエージェントから:** 「OpenAI-compatible」プロバイダーを追加し、ベース URL に
**`http://127.0.0.1:8080/v1`** を指定します。API キーとモデル名は何でもかまいません。
- Anthropic の API を使うアプリ:`http://127.0.0.1:8080/v1/messages`(Claude Code:
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`)。
- Codex CLI など OpenAI Responses API を使うアプリ:`/v1/responses`
([設定方法](docs/DETAILS.md#the-responses-api-and-codex-cli))。
- **思考(Thinking):** チャットのメニュー、またはアプリの「reasoning effort」で **off、low、medium、high** を選びます。
off がいちばん速く、high は難しい質問に向いています。
- **画像:** セットアップで「Images?」に yes と答えます。そのあとチャットで **Picture** をクリックするか、アプリで画像を添付します。
AMD のカードは、Linux ではプロセッサーを使って画像を読みます。Windows ではまだ読めません。
- **スマートフォンや別の PC から:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`。キーは必ず設定してください。
- **リクエストは 1 つずつ:** 初期設定では Strata は 1 つのリクエストに答え、ほかは待ちます。複数に同時に答えるには
`"parallel": 2` を設定します([BATCHING.md](docs/BATCHING.md))。12 GB のカードでは、それぞれの回答が遅くなります。
- **長いプロンプト:** Strata はチャットの最初のメッセージをすべて読みます。30,000 トークンあたり約 1 分かかります。
2 通目以降のメッセージは数秒で始まります。
アプリには、**チャット**、モデルとGPU・CPU・RAMの状態を見る**モニター**、**モデル**ページ、**設定**があります。チャットはPC上のフォルダで作業できます。
**手動**モードでは、ファイルを変更したりコマンドを実行したりする前に確認します。`/loop 10m <やること>` と入力すると、チャットを開いている間、メッセージを10分ごとに繰り返し送ります。
詳しくは:[チャットの保存場所](docs/INSTALL.md#where-things-are-stored)、[API](docs/DETAILS.md#using-it)。
チャットは DuckDuckGo(または Bing)でウェブ検索ができ、アカウントは不要です。**設定、ツール**で検索キーを追加することもできます。
検索は、PCの外に何かを送る唯一の機能で、送られるのは検索語です。
2つのモデルを動かすには、**複数のモデルを実行**をオンにして、それぞれのモデルに同じメーカーのカードを1枚ずつ割り当てます。
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>Strata アプリでのチャット。エージェントが小さなボクセルの塔の庭を 1 つの HTML ファイルとして書き、そのページが会話の横のプレビューで開きます</sub></p>
### ターミナルで
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub>モデルが返信を書いている間の <code>strata monitor</code>。速度、GPU、メモリ、最近のリクエストが表示されます</sub></p>
```sh
strata launch # ガイド付きの画面: モデル、コンテキスト、画像、アクセス、そして開始
strata launch --model q2_0 --yes --no-tui # 質問なし: 開始し、アドレスを表示して戻る
strata launch --plan --model q2_0 # 実行内容を表示するだけで、何も変更しない
strata launch --status # 今、何が動いているか
strata launch --stop # 停止する
strata monitor --url http://127.0.0.1:8080 # 速度、GPU、メモリ、リクエストを見る
strata update --check # 新しいエンジンがあるか確認(strata update でインストール)
strata doctor # インストール済みのエンジンファイルを調べる
```
`strata --help` ですべてのコマンドを確認できます。起動とサーバーの稼働では、何もダウンロードしません。ダウンロードするのは、インストール、
`strata launch`(モデルを選んだとき)、`strata update` だけです。スマートフォンや別のPCから Strata に接続するには、
`strata launch --access lan` を使います。API キーが作られ、キーがない状態ではネットワーク上で起動しません。`--open` を付けない限り、ブラウザは開きません。
ガイド: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md)。クライアント全体: [docs/TERMINAL.md](docs/TERMINAL.md)。
### AI エージェントと使う
エージェントは、方法Bの1行コマンドで Strata をインストールし、
`strata launch --model NAME --yes --no-tui` を実行します。手順、フラグ、終了コードは [docs/AI_SETUP.md](docs/AI_SETUP.md) に、
リポジトリのルールは [AGENTS.md](AGENTS.md) にあります。
### ほかのアプリをつなぐ
| アプリの種類 | ベース URL |
| --- | --- |
| OpenAI 互換(ほとんどのアプリとエージェント) | `http://127.0.0.1:8080/v1`(API キーもモデル名も何でも可) |
| Anthropic API(Claude Code など) | `http://127.0.0.1:8080`、`/v1` は付けません。これらのアプリは `/v1/messages` を自分で付けます。Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| OpenAI Responses API(Codex CLI) | `http://127.0.0.1:8080/v1/responses`([設定方法](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` は、OpenCode、Codex CLI、Claude Code、oh-my-pi、pi、Hermes Agent、OpenClaw の正確な設定を表示します。
`strata connect opencode --write` は、確認したうえでファイルを書き込み、バックアップを残します。
### 画像
画像を読めるモデルでは、画像は初期設定でオンです。チャットに画像を添付または貼り付けるか、アプリから送ってください。
グラフィックスカードが24 GB未満のときは、画像リーダーがプロセッサで動きます(画像1枚につき数秒、カードのメモリは使いません)。24 GB以上のときはグラフィックスカードで動きます。
このルールは NVIDIA と AMD で同じです。画像はモデルごとにオフにでき、`--vision off`、`--vision gpu`、`--vision cpu` でリーダーを選べます。
このルールの根拠となる数値です。Radeon AI PRO R9700 では、カードのリーダーは260トークンの画像に57 ms、プロセッサのリーダーは3.1秒かかりました。
16 GBのカード(Tesla P100 と RTX A4000、Linux)では、カードのリーダーを使うとテキストの速度が3〜5%下がりました。
本来モデルが使う約1.4 GBをリーダーが占めるためです。Mac ではメモリ量にかかわらず GPU が読み取ります(32 GB の M4 で260トークンの画像に0.45秒、共有メモリを約1.2 GB使います)。プロセッサでは1枚あたり約50秒かかるため、`--vision cpu` を指定したときだけ使います。
### 知っておくと便利なこと
- 思考のレベル(**off、low、medium、high**)は、チャットのメニューか、お使いのアプリの「reasoning effort」で選びます。off がいちばん速くなります。
- Strata は初期設定では、一度に1つのリクエストに答えます。同時に複数処理するには、`strata launch` で `--parallel N` を使うか、設定に
`"parallel": 2` を書きます([BATCHING.md](docs/BATCHING.md))。12 GBのカードでは、1つずつの回答が遅くなります。
- Strata はチャットの最初のメッセージを全文読みます。30,000トークンあたり約1分かかります。続きのメッセージは数秒で始まります。
- チャットの保存場所と API 全体: [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored)、[docs/DETAILS.md](docs/DETAILS.md#using-it)。
## ソースからビルド
<details>
<summary>リポジトリから Windows、Linux、Mac 用にビルドする</summary>
[Strata をダウンロード](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip)して展開するか、`git clone` します。
**Windows:** `START-HERE.bat` をダブルクリックします。**Linux:** Strata フォルダで `./setup.sh` を実行します。
**Mac(Apple Silicon、macOS 15 以降):** 最初に一度、`xcode-select --install` で Apple のコマンドラインツールをインストールします
(Xcode 本体は不要です)。Strata フォルダでターミナルを開き、`./setup.sh` を実行します。Mac 用エンジンをお使いの Mac 上でビルドし
(約7分)、同じ質問をして、モデルをダウンロードし、起動します。2回目以降は `./setup.sh` ですぐに起動します。
お使いの Mac に合うサイズは [docs/MAC.md](docs/MAC.md) を見てください。
インストーラーはカードを見つけて、合うエンジンを設定します。どのモデルとサイズにするか、コンテキストをどれだけにするか(モデルが一度に覚えておけるテキストの量)、
画像を読ませるかを聞かれます。おすすめの答えでよければ Enter を押します。そのあとモデル(約70 GB)をダウンロードして起動します。
ダウンロードが止まったら、もう一度実行すると続きから再開します。ブラウザで `http://127.0.0.1:8080` の Strata ページが開きます。
> モデルの起動中は、PCが1〜3分ほど(初回がいちばん長い)遅くなったり、応答しなくなったりすることがあります。Strata は35〜55 GBをRAMに読み込み、
> その一部をグラフィックスカード用に固定するためです。ウィンドウを閉じずに待ってください。
次回からは `START-HERE.bat`(または `./setup.sh`)をもう一度実行するだけです。すぐに起動し、同じものを2回ダウンロードすることはありません。ウィンドウを閉じるとモデルが止まります。
`UPDATE.bat`(`./update.sh`)は、起動せずに Strata を更新します。更新、Docker、複数のカード、ファイルの保存先、すべてのオプションは
[docs/INSTALL.md](docs/INSTALL.md) にあります。
PCの性能を測るには、`BENCH.bat`(`./bench.sh`)を実行します。5〜10分かかり、共有できるレポートが作られます
([方法](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark))。
**オプション: モデル全体がRAMに入るとき、1枚のカードをさらに速くする。** RubenTorney による、自分でオンにする2つの設定です(#1904、#1905)。
グラフィックスカードが1枚で、RAM がすべてのエキスパートを保持できるときに効果があります。たとえば64 GBのPCでの Q2_0 や IQ3_XXS です。
Strata のデータフォルダにある `strata-<model>.json`([場所](docs/INSTALL.md#where-things-are-stored))を開いて、次を追加します。
- `"args"` のリストに `"--adapt-async", "1"` を追加します。エキスパートキャッシュが、回答を止めずに補助スレッドで更新されます。
RTX 5070(Windows、Q2_0)での測定: デコードが4〜5%、コードや長い文書では7〜10%速くなりました。回答は初期設定と少し違うことがありますが、実行ごとの結果は変わりません。
- `"env"` ブロックに `"STRATA_PIN_BY_NEED": "1"` を追加します(Windows)。Windows ではグラフィックスカード用に固定できる RAM が半分ほどしかありません。
この設定は、カードが最初に必要とするエキスパートを優先して固定します。RTX 5070 での測定: プロンプトの読み取りが4.4%速くなりました。
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
変更したら Strata を再起動してください。元に戻すには、2つの項目を削除します。
</details>
## 仕組み
このようなモデルは、ふつう数百GBのグラフィックスメモリを持つサーバーで動きます。お使いのグラフィックスカードは12〜24 GBです。Strata は、PC全体で作業を分担して、モデルを収めます。
図書館にたとえてみます。
- モデルは、24,576人の小さな専門家(「エキスパート」)からなるチームです。24,576冊の参考書のようなものです。1つの単語に必要なのは、そのうち10冊だけです。
- **グラフィックスカードは机です。** 小さくて速く、最もよく使う約4,000冊が開いて置いてあります。
- **RAM は閲覧室の書棚です。** 24,576冊すべてがここにあります。単語に必要な本が机になければ、プロセッサがすぐに書棚から読み、その間もカードは作業を続けます。
- **SSD は書庫です。** 29 GBの参照テーブルがしまってあります。モデルは単語ごとにその数行だけを読むので、RAM に置いておく必要はありません。
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
小さな助手が、次の数単語も予想します。大きなモデルがそれをまとめて確認し、正しいものだけを残します。答えは同じで、1.6〜1.8倍早く返ります。
長い文章は、一度に最大8,192トークンの大きな塊で、毎秒1,000トークンを超える速さで読みます。
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
詳しい説明は [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md) にあります。各部分と数値は
[詳細](docs/DETAILS.md#how-it-works)と[論文](docs/paper/Strata-Paper.pdf)を見てください。
## うまくいかないとき
- **Strata の初回起動で PC が固まった。** モデルの読み込み中はよくあることです。ウィンドウを閉じずに待ってください。
10 分たっても固まったまま? PC を再起動し、ほかのプログラムを閉じてもう一度試すか、小さいサイズを選んでください。
- **ダウンロードやインストールの途中で止まった。** `START-HERE.bat`(または `./setup.sh`)をもう一度実行してください。
止まったところから続きます。
- **とても遅く、ディスクのランプが点滅し続ける。または「the engine stopped unexpectedly」と表示される。** PC の空き RAM が
足りません。ほかのプログラムを閉じるか(ブラウザーは多く使います)、小さいサイズ(Q2_0 または IQ2_XS)を選んでください。
- **ポート 8080 がすでに使われていると表示される。** Strata はもう動いています。そのウィンドウを探してください。
- **Strata を初めて起動したときに、PCが固まりました。** モデルを RAM に読み込んでいます。ウィンドウを閉じずに待ってください。10分たっても固まったままなら、
PCを再起動し、ほかのプログラムを閉じてからもう一度試すか、小さいサイズを選んでください。
- **ダウンロードやインストールの途中で止まりました。** アプリか `strata launch` をもう一度起動するか、`START-HERE.bat`(または
`./setup.sh`)をもう一度実行してください。止まったところから再開します。
- **とても遅い、ディスクのランプが点滅し続ける、または「エンジンが予期せず停止しました」と表示される。** PCの空き RAM が足りません。
ほかのプログラムを閉じるか(ブラウザはメモリを多く使います)、Q2_0 や IQ2_XS のような小さいサイズを選んでください。
- **このPCに合うエンジンがないと表示される。** アプリでは、**設定、エンジン**を開いて**エンジンを更新**を押します。ターミナルでは
`strata update` を実行します。`strata doctor` で、どのエンジンファイルがインストールされているか分かります。
- **ポート 8080 はすでに使われていると表示される。** Strata がすでに動いています。ウィンドウを探してください。
ほかの問題と解決方法:[docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)。それでも解決しない場合は
[issue](https://github.com/Niko1221/Strata/issues) を作成し、Strata フォルダーにある `strata-<model>.log` を添付してください。
セキュリティの問題を見つけたら、非公開で報告してください:[SECURITY.md](SECURITY.md)。
## しくみ
このようなモデルは、ふつう数百ギガバイトのグラフィックメモリを持つサーバーで動きます。あなたのグラフィックカードは
12-24 GB です。Strata は **PC 全体で作業を分担する** ことで、モデルを収めています。台所を思い浮かべてください。
いつも使う道具は調理台に置き、残りは食料庫にしまっておきます。
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="モデルの 24,576 個のエキスパート:よく使うものはグラフィックカードに、すべては RAM に、ルックアップテーブルは SSD に"></p>
- **モデルは 24,576 個の小さな専門家(「エキスパート」)のチームです。** 1 語ごとに必要なのはそのうち 10 個だけです。
- **グラフィックカード** は、よく使われる数千個のエキスパートを持ちます。**RAM** はすべてのエキスパートを持ち、
**プロセッサー** が残りを同時に処理します。**SSD** は大きなルックアップテーブルを持ちます。
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="小さなヘルパーが次の単語を予想し、大きなモデルがまとめて確認して正しいものを残す"></p>
- **予想してから確認:** 小さなヘルパーが次の数語を予想します。大きなモデルがそれをまとめて確認します。
答えは同じまま、1.6-1.8x 早く返ってきます。
- **長いテキストは大きなまとまりで読みます**(一度に最大 8,192 トークン)。速さは毎秒 1,000 トークン以上です。
もっと詳しい説明:[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md)。各部分とその数値:
[詳細](docs/DETAILS.md#how-it-works)と[論文](docs/paper/Strata-Paper.pdf)。
そのほかの問題と対処法は [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) にあります。それでも解決しない場合は、
[issue](https://github.com/Niko1221/Strata/issues) を作成してください。アプリでは、**設定、エンジン、診断、ログを見る、ログを保存**で作られるファイルを添付します。
ターミナルでは、`strata doctor` の出力を貼り付けます。ソースから動かしている場合は、Strata フォルダの `strata-<model>.log` を添付してください。
セキュリティ上の問題は、非公開で報告してください: [SECURITY.md](SECURITY.md)。
## クレジットとライセンス
モデルは Qwen チームの [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) です。
圧縮は [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF)、UkisAI(Swift 1.5)、
Unsloth によるものです。Strata は [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp) の一部を使っています。すべてのクレジット:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits)。Strata は [MIT License](LICENSE) のオープンソースです。一部のパーツと
すべてのモデルには、それぞれ独自のライセンスがあります([どれか](docs/HOW_IT_WORKS.md#license))。
モデルは Qwen チームの [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) です。圧縮は
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF)、UkisAI(Swift 1.5)、agentionai(Gyro)、
Unsloth が行いました。Strata は [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp) の一部を使っています。すべてのクレジットは
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits) にあります。Strata は [MIT ライセンス](LICENSE)のオープンソースです。一部のコンポーネントと
すべてのモデルには、それぞれのライセンスがあります([該当するもの](docs/HOW_IT_WORKS.md#license))。
## Strata を支援する
## コントリビューション
Strata は無料のオープンソースです。役に立ったら、開発を支援していただけます:
バグを見つけた、変更やベンチマークを送りたい、という方は、先に [CONTRIBUTING.md](CONTRIBUTING.md) を読んでください。
ほかの人があなたの成果を確認できるように、何を書くべきかが説明してあります。
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>
## Strata を応援する
Strata は無料で、MIT ライセンスです。私が入手できるハードウェアで開発とテストをしています。お使いのPCで快適に動いたなら、コーヒー1杯分の支援は、
新しいカードのテストと、報告いただいた不具合の修正を続ける助けになります。無理のない範囲で大丈夫です。使って、気づいたことを報告してもらえるだけでも同じように助かります。
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
Strata のスポンサーに関心がある方は、[GitHub](https://github.com/Niko1221) から連絡してください。
+330 -146
View File
@@ -1,212 +1,396 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
</picture>
</p>
**English** · [简体中文](README.zh-CN.md) · [日本語](README.ja.md) · [Deutsch](README.de.md) · [Français](README.fr.md) · [Español](README.es.md) · [Português](README.pt-BR.md)
<p align="center"><b>English</b> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a></p>
<p align="center"><b>Run a 125-billion-parameter AI model on your own gaming PC</b><br>
NVIDIA or AMD graphics card (12 GB or more) · Windows or Linux · free and open source</p>
<h3 align="center">Run a state-of-the-art model on your own gaming PC</h3>
<p align="center">Support for NVIDIA, AMD, Intel · Windows, Linux, Mac · free and open source</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>A voxel pagoda garden, 1 shot prompt running on an RTX 5070 with Strata (IQ3_S, 128K context) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">full video (49 s)</a></sub></p>
Strata runs **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** on a normal PC. This is a
large, smart AI model that usually needs a server. It chats, writes code, reads pictures and works with your apps
and coding agents. Nothing leaves your PC.
Strata runs **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**, a 125-billion-parameter model that
normally needs a server, on a normal PC. It chats, writes code, reads pictures and works with your apps and coding
agents. Your chats and files stay on your PC.
## How fast is it?
**New here?** Check [what you need](#what-you-need), then [install the app](#option-a-the-strata-app).
We measured it on two ordinary gaming PCs. A token is about ¾ of a word.
- **Runs on a gaming PC.** 32 GB of RAM and graphics memory together is enough to start.
- **Works with your tools.** It speaks the OpenAI and Anthropic APIs, so Claude Code, Codex CLI, OpenCode and other agents connect to it.
- **Reads pictures.** They are on by default.
- **Free.** MIT license, no account, no cloud.
- **Private.** No telemetry. What Strata downloads, when it uses the internet and how it checks downloads is in [docs/SAFETY.md](docs/SAFETY.md).
- **Writes answers:** how fast the reply appears in a short chat. 60 tokens per second is faster than you can read.
- **Reads your prompt:** how fast it takes in what you send (here a 32K-token document, code or chat history).
<table>
<tr><th>NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM</th><th>AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM</th></tr>
<tr><td>
| Size | Writes answers | Reads your prompt |
| --- | ---: | ---: |
| **Q2_0** | 94 tokens/s | 2,650 tokens/s |
| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
| **Coder** | 55 tokens/s | 2,180 tokens/s |
</td><td>
| Size | Writes answers | Reads your prompt |
| --- | ---: | ---: |
| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
| **Coder** | 44 tokens/s | 1,420 tokens/s |
</td></tr>
</table>
NVIDIA: Q2_0 with engine 0.1.36, the other rows with 0.1.26 (4K answers, 32K prompts). The full tables are in
[DETAILS.md](docs/DETAILS.md#speed-measured). A card with more VRAM is faster: an RTX 3090 (24 GB) should write
about 100-140 tokens per second. Long chats and other cards: [speed of each model](docs/MODELS.md#how-fast-is-each-size),
[community results](docs/COMMUNITY_BENCHMARKS.md).
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
<sub>Strata is free. If it runs well on your PC, a coffee keeps the work on it going.</sub></p>
Version 0.2.1 · engine 0.1.46
## What you need
| | |
**You need at least 32 GB of RAM and graphics memory (VRAM) together.** On a Mac or a Ryzen AI Max, that is the unified
memory. More memory gives a bigger model or a faster one.
- A 32 GB PC with a 12-16 GB card runs the Coder.
- 64 GB of RAM runs every size.
| Your computer | Details |
| --- | --- |
| **Graphics card** | **NVIDIA** GeForce RTX 20, 30, 40 or 50 series, or **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 or RX 6800 / 6900 series. It needs **12 GB of VRAM or more**. |
| **RAM** | 32 GB or more. Your RAM decides [which model](#which-model-should-i-pick) fits. 64 GB runs every size. |
| **Disk** | About 80 GB free. Use an SSD if you can: the first start is much faster. |
| **System** | Windows 10 / 11 or Linux, and a current graphics driver from NVIDIA or AMD. |
| **NVIDIA** GeForce RTX 20, 30, 40, 50 series (12 GB or more) | Install the app or the Terminal. |
| **AMD** Radeon RX 6800 and newer, Radeon AI PRO R9700 (12 GB or more) | [AMD cards](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md), [Intel engine](docs/INTEL.md) |
| **Apple Silicon Mac** (M1 or newer, macOS 15 or newer) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max (Strix Halo)** and other unified-memory systems | [Strix Halo](docs/STRIX_HALO.md) |
| Older cards and processors | [Older GPUs](docs/OLDER_GPUS.md), [Older CPUs](docs/INSTALL.md#older-cpus-experimental) |
The installer sets up everything else. Two or three cards can share the model ([multi-GPU](docs/MULTI_GPU.md)).
Also needed:
Experimental, written and tested by community members on their own machines:
- about 80 GB of free disk (an SSD makes the first start much faster)
- Windows 10 / 11, Linux or macOS
- a current graphics driver
- **Older graphics cards** (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT): [Older GPUs](docs/OLDER_GPUS.md).
- **Intel Arc**, built from source on Linux: [Intel Arc](docs/INTEL_ARC.md).
- **AMD Ryzen AI Max (Strix Halo)**, built from source on Linux: [Strix Halo](docs/STRIX_HALO.md).
- **Older processors without AVX2**: they work, but slowly. [Older CPUs](docs/INSTALL.md#older-cpus-experimental).
The full list: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
Two or three cards of the same make can share a model ([multi-GPU](docs/MULTI_GPU.md)). Strata never splits a model across
NVIDIA and AMD cards. Full list: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
## Install
### Let your AI set it up
Pick one option. They all start the same engine and use the same models.
Do you use an AI coding assistant (Claude Code, Cursor, Codex, GitHub Copilot, ...)? Paste this into it:
### Option A: the Strata app
The easiest way. The app has the engine inside: install it, pick a model, chat. Download the file for your system and
graphics card from the [releases page](https://github.com/Niko1221/Strata/releases/latest). Not sure which one? See the
line under the table.
| System | Your graphics card | File to download |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX, driver 580 or newer | [`Strata-0.2.1-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda13.exe) |
| | NVIDIA, older card or driver older than 580 | [`Strata-0.2.1-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda12.exe) |
| | AMD Radeon RX 9000 series or Radeon AI PRO R9700 | [`Strata-0.2.1-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip.exe) |
| | AMD Radeon RX 7000 series or Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX, driver 580 or newer | [`Strata-0.2.1-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda13.AppImage) |
| | NVIDIA, older card or driver older than 580 | [`Strata-0.2.1-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon RX 9000 series or Radeon AI PRO R9700 | [`Strata-0.2.1-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx9070.AppImage) |
| | AMD Radeon RX 7000 series | [`Strata-0.2.1-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-strixhalo.AppImage) |
| **Mac** (Apple Silicon, macOS 15 or newer) | built in | [`Strata-0.2.1-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-mac.dmg) |
**If you are unsure:** for NVIDIA, take `cuda13` if your driver is version 580 or newer, otherwise `cuda12`. `cuda12` is
for older cards and older drivers. The `hip` and `amd` files are for AMD cards. Other AMD cards (RX 6000) have no
ready-made app yet: see [AMD cards](docs/AMD_HIP.md).
Then:
- **Windows:** run the `.exe`. It installs for your user and needs no administrator rights. If Windows says "Windows
protected your PC", choose **More info**, then **Run anyway** (the installer is not code-signed yet).
- **Linux:** make the file runnable (`chmod +x`) and start it. It needs FUSE 2: `sudo apt install libfuse2` (Ubuntu 22.04)
or `libfuse2t64` (Ubuntu 24.04 and newer).
- **Mac:** open the `.dmg` and drag Strata onto Applications.
On the first screen the app reads your PC and recommends a model that fits. Press install. The model is a big download
(about 40-70 GB) and continues where it stopped if it is interrupted. Closing the window stops Strata and the engine.
### Option B: the Terminal (one line)
The Terminal is the `strata` command, for scripts, servers and AI agents.
Linux and Mac:
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
```
Windows (PowerShell):
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
The script checks your PC, downloads the Terminal, verifies its SHA-256 and unpacks it into your home folder. It needs no
administrator rights. On Windows, open a new terminal window afterwards. Then start Strata:
```sh
strata launch
```
It asks one question at a time: model, context, pictures, who may connect. Then it downloads the engine for your graphics
card (signed, and test-started before use) and the model, and starts serving. For scripts and AI agents, use flags:
```sh
strata launch --model q2_0 --yes --no-tui
```
To read the install script before it runs, download it first with `curl -fsSL <address> -o install.sh`, read it, then run
`sh install.sh`. All screens and flags: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md).
### Option C: let your AI set it up
Paste this into Claude Code, Cursor, Codex, GitHub Copilot or another coding assistant:
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
It checks your graphics card, RAM and disk and picks the model that fits. Then it installs and starts it and tells
you how to connect your apps. AI tools can also install, start and stop Strata through its
[MCP server](docs/MCP_SERVER.md).
It checks your graphics card, RAM and disk, picks the model that fits, installs and starts it, and tells you how to
connect your apps. AI tools can also install, start and stop Strata through its [MCP server](docs/MCP_SERVER.md).
### Or do it yourself
Prefer to build it yourself? See [Build from source](#build-from-source).
[Download Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) and unzip it (or `git clone` it).
**Windows:** double-click **`START-HERE.bat`**. **Linux:** run **`./setup.sh`** in the Strata folder.
## Models
The steps are the same for NVIDIA and AMD. The installer finds your card and sets up the right engine for it. It
asks you a few questions:
The app and `strata launch` recommend a model for your memory. It is one model in several sizes, compressed more or less.
Smaller sizes are faster. Larger sizes are a bit smarter.
- which model and which size,
- how much context (how much text the model keeps in mind),
- whether it should read pictures.
### Low on memory? Try Gyro
Press Enter each time for the recommended answer. Then it downloads the model (about 70 GB) and starts it. If the
download stops, run it again: it continues where it left off. Your browser opens the Strata app at
`http://127.0.0.1:8080`.
[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) (by agentionai) is the smallest way to run this model.
> **While the model starts, your PC can be slow or stop responding for 1-3 minutes** (longest the first time).
> Strata loads 35-55 GB into your RAM and locks part of it for the graphics card. This is normal. Wait, and don't
> close the window. The window shows what Strata is doing.
| Size | Memory it needs (RAM + VRAM) | Download | What it is |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27.6 GB | 58.5 GB | For PCs that fit nothing else. Quality is lower than IQ2_XS. |
| **Gyro-M** | 35 GB | 92.0 GB | About IQ3_XXS quality with about 20% less GPU memory. It writes answers 17-21% slower than Q2_0 on a 12 GB card and on an R9700. |
**Next time**, run `START-HERE.bat` (or `./setup.sh`) again. It starts right away and downloads nothing twice. Close
its window to stop the model. `UPDATE.bat` (`./update.sh`) updates Strata without starting it. Updating, Docker,
several cards, where the files go and every option: [docs/INSTALL.md](docs/INSTALL.md).
Gyro runs on NVIDIA and AMD cards (AMD is experimental), not on Intel Arc or the Mac. Details and measurements:
[docs/GYRO.md](docs/GYRO.md). Start it with `strata launch --model gyro-s`.
## Which model should I pick?
### The other sizes
The installer recommends one for your RAM. The same model comes in several sizes, compressed more or less. Smaller
sizes are faster. Larger sizes are a bit smarter.
| Your RAM | Take | Why |
| Your memory (RAM + VRAM) | Take | Notes |
| --- | --- | --- |
| **32 GB** | **Coder** | it fits 32 GB, and it is made for code (with a 24 GB card, Q2_0 and IQ2_XS run too) |
| **48 GB** | **IQ2_XS** (or Q2_0, the fastest) | the larger sizes do not fit |
| **64 GB** | **IQ2_XS** (recommended), or IQ3_XXS / IQ3_S | every size fits; IQ3_S is the best and the slowest |
| **96 GB or more** | **IQ3_S**, or Unsloth's UD-IQ4_XS (~4-bit) | room for the largest sizes with everything else open |
| **32 GB** | **Gyro-S**, or the **[Coder](docs/MODELS.md#coder)** for code | The Coder needs 32 GB of RAM and keeps half of the experts, so it is weaker outside code |
| **40 GB** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS (39.2 GB), or **Gyro-M** (35 GB) | Swift thinks for a shorter time before it answers. Gyro-M is close to IQ3_XXS in quality, but it writes answers 17-21% slower than Q2_0 |
| **48 GB** | **IQ3_XXS** (47.0 GB) | Slower, better |
| **64 GB** | **IQ3_S** (54.8 GB), the original or Swift 1.5 | The slowest, the best: it matches the full model on the published tests |
| **96 GB or more** | Unsloth's **UD-IQ4_XS** (about 72 GB of RAM) to **Q8_0** (about 136 GB), 4 to 8 bit | Closer to the full model. With less RAM, Strata reads part of them from the SSD, which is much slower |
- **[Coder](docs/MODELS.md#coder):** a coding version with half of the experts removed. It reaches 91% of the full
model's SWE-bench Verified score (measured by its authors) and fits 32 GB of RAM. It is weaker outside code,
including Chinese and other CJK text (#438). For those, take Q2_0, IQ2_XS or IQ3_S, which keep every expert.
- **[Swift 1.5](docs/MODELS.md#swift-15):** a fine-tune that thinks for a much shorter time before it answers. You
get the answer sooner, at about the same quality.
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** Unsloth's ~4-bit version, between IQ3_S and
UD-Q4_K_XL in quality. A 94 GB download. With less than ~80 GB of RAM, Strata reads part of it from the SSD
while it answers, so it is slower there (an NVMe SSD helps).
- **[Unsloth UD-Q4_K_XL](docs/MODELS.md#unsloth-ud-q4_k_xl-experimental)** (experimental): the closest to the full
model. But Strata reads most of it from the SSD while it answers, so it writes only 7-8.5 tokens/s on a 64 GB PC.
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** you set it up by hand. It is
not in the installer's menu.
Less RAM than a row says can still work: Strata switches to its
[low-RAM mode](docs/MODELS.md#a-big-graphics-card-and-little-ram) by itself. Expect it to be slower, and much slower
with a small graphics card.
Sizes, downloads and what fits where: [docs/MODELS.md](docs/MODELS.md). To add another model later, run
`SETUP.bat` (Linux: `./setup.sh --setup`).
More choices:
- **[Swift 1.5](docs/MODELS.md#swift-15)** is a fine-tune that thinks for a shorter time before it answers, at about the same quality and RAM.
- **[NVFP4](docs/NVFP4.md)** is NVIDIA's 4-bit version for NVIDIA cards. It needs about 80 GB of RAM to keep every expert in RAM.
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** is a smaller, different model that reads pictures and can answer several requests at once.
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** is set up by hand.
Every size, download and what fits where: [docs/MODELS.md](docs/MODELS.md).
## How fast is it?
Based on a 12 GB graphics card (RTX 5070, Ryzen 5 7600, 64 GB RAM). See results from other cards:
[speed of each model](docs/MODELS.md#how-fast-is-each-size), [community results](docs/COMMUNITY_BENCHMARKS.md).
| Model | Answers appear at | Reads your prompt at |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2,650 tokens/s |
| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2,090 tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1,750 tokens/s |
| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1,620 tokens/s |
| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2,180 tokens/s |
A token is about ¾ of a word, so 60 tokens per second is faster than you can read. "Reads your prompt" is how fast Strata
takes in a long document, code or chat history.
Measured on a 16 GB AMD card (RX 9070 XT, Ryzen 9 3900X, 47 GB RAM):
| Model | Answers appear at | Reads your prompt at |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1,160 tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1,110 tokens/s |
| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1,420 tokens/s |
The bars use one scale for both tables. A card with more VRAM is faster: an RTX 3090 (24 GB) should write about 100-140
tokens per second. The engine version behind each number is in [DETAILS.md](docs/DETAILS.md#speed-measured).
## Using it
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="The Strata app's Monitor tab next to a coding agent"><br>
<sub>The Strata app's <b>Monitor</b> (left) while a coding agent writes the pagoda garden from the video (right)</sub></p>
### In the app
- **In the browser:** open `http://127.0.0.1:8080`. It has **Chat**, a live **Monitor** of the model and your
GPU/CPU/RAM, and **About** with the settings and addresses.
- **Your apps and coding agents:** add an "OpenAI-compatible" provider with the base URL
**`http://127.0.0.1:8080/v1`**. Any API key and any model name work.
- Apps that use Anthropic's API: `http://127.0.0.1:8080/v1/messages` (Claude Code:
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`).
- Codex CLI and other apps that use the OpenAI Responses API: `/v1/responses`
([setup](docs/DETAILS.md#the-responses-api-and-codex-cli)).
- **Thinking:** choose **off, low, medium or high** in the chat menu or in your app's "reasoning effort". Off is the
fastest. High is best for hard questions.
- **Pictures:** say yes to "Images?" in setup. Then click **Picture** in the chat, or attach pictures in your app.
AMD cards read pictures on Linux through the processor; on Windows they can't yet.
- **From your phone or another PC:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`. Always set a key.
- **One request at a time:** by default Strata answers one request, and the others wait. To answer several at once,
set `"parallel": 2` ([BATCHING.md](docs/BATCHING.md)). On a 12 GB card this makes each answer slower.
- **Long prompts:** Strata reads the first message of a chat in full, about 1 minute per 30,000 tokens. Follow-up
messages start in seconds.
The app has a **Chat**, a **Monitor** of the model and your GPU, CPU and RAM, a **Models** page and **Settings**. The chat
can work in a folder on your PC. In **Manual** mode it asks before it changes a file or runs a command. Type
`/loop 10m <what to do>` to repeat a message every 10 minutes while the chat is open.
More: [where your chats are stored](docs/INSTALL.md#where-things-are-stored), [the API](docs/DETAILS.md#using-it).
The chat can search the web with DuckDuckGo (or Bing), with no account. You can add a search key under **Settings, Tools**.
Search is the one feature that sends something off your PC, the search words.
## Something went wrong?
To run two models, turn on **Run multiple models** and give each model its own card of the same make.
- **My PC froze the first time Strata started.** This is normal while it loads the model. Wait, and don't close the
window. Still frozen after 10 minutes? Restart the PC, close other programs and try again, or pick a smaller size.
- **It stopped while downloading or installing.** Run `START-HERE.bat` (or `./setup.sh`) again. It continues where
it stopped.
- **It's very slow and the disk light keeps blinking, or it says "the engine stopped unexpectedly".** Your PC does
not have enough free RAM. Close other programs (browsers use a lot), or pick a smaller size (Q2_0 or IQ2_XS).
- **It says port 8080 is already in use.** Strata is already running. Look for its window.
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>A chat in the Strata app: the agent wrote a small voxel pagoda garden as one HTML file, and the page opens in the preview next to the conversation</sub></p>
More problems and their fixes: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Still stuck? Open an
[issue](https://github.com/Niko1221/Strata/issues) and attach `strata-<model>.log` from the Strata folder. Found a
security problem? Report it privately: [SECURITY.md](SECURITY.md).
### In the Terminal
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub><code>strata monitor</code> while the model writes a reply: speed, GPU, memory and the recent requests</sub></p>
```sh
strata launch # guided screens: model, context, pictures, access, then serve
strata launch --model q2_0 --yes --no-tui # no questions: serve, print the address, return
strata launch --plan --model q2_0 # show what would happen; nothing changes
strata launch --status # what is being served now
strata launch --stop # stop it
strata monitor --url http://127.0.0.1:8080 # watch speed, GPU, memory and requests
strata update --check # is there a new engine? (strata update installs it)
strata doctor # check the installed engine files
```
`strata --help` lists everything. Starting and serving never download anything. Only the install, `strata launch` (when
you choose a model) and `strata update` do. To reach Strata from your phone or another PC, use
`strata launch --access lan`. It makes an API key and will not start on a network without one. No browser opens unless
you add `--open`. Guide: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). Whole client: [docs/TERMINAL.md](docs/TERMINAL.md).
### With AI agents
An agent installs Strata with the one-line command in Option B and runs
`strata launch --model NAME --yes --no-tui`. The steps, flags and exit codes are in [docs/AI_SETUP.md](docs/AI_SETUP.md),
and the repository rules are in [AGENTS.md](AGENTS.md).
### Connect your other apps
| App type | Base URL |
| --- | --- |
| OpenAI-compatible (most apps and agents) | `http://127.0.0.1:8080/v1` (any API key, any model name) |
| Anthropic API (Claude Code and others) | `http://127.0.0.1:8080`, without `/v1`. These apps add `/v1/messages` themselves. Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| OpenAI Responses API (Codex CLI) | `http://127.0.0.1:8080/v1/responses` ([setup](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` prints the exact settings for OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent and OpenClaw.
`strata connect opencode --write` writes the file after asking and keeps a backup.
### Pictures
Pictures are on by default for models that can read them. Attach or paste a picture in the chat, or send it from your app.
The picture reader runs on the processor when your graphics card has less than 24 GB (a few seconds per picture, nothing
taken from the card's memory) and on the graphics card from 24 GB. The rule is the same for NVIDIA and AMD. Turn pictures
off per model, or choose the reader with `--vision off`, `--vision gpu` or `--vision cpu`.
The numbers behind the rule: on a Radeon AI PRO R9700 the card's reader took 57 ms for a 260-token picture and the
processor's took 3.1 s. On 16 GB cards (Tesla P100 and RTX A4000, Linux) the card's reader cost 3-5% of text speed,
because it holds about 1.4 GB that the model would use otherwise. On a Mac the reader runs on the Mac's GPU whatever its memory (0.45 s for a 260-token picture on an M4 with 32 GB, about 1.2 GB of the shared
memory); the processor reader needs about 50 s per picture there, so it runs only when you ask for it with `--vision cpu`.
### Good to know
- Choose the thinking level (**off, low, medium or high**) in the chat menu or in your app's "reasoning effort". Off is fastest.
- Strata answers one request at a time by default. For several at once, use `--parallel N` in `strata launch` or
`"parallel": 2` in the config ([BATCHING.md](docs/BATCHING.md)). On a 12 GB card this makes each answer slower.
- Strata reads the first message of a chat in full, about 1 minute per 30,000 tokens. Follow-up messages start in seconds.
- Where your chats are stored and the whole API: [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored), [docs/DETAILS.md](docs/DETAILS.md#using-it).
## Build from source
<details>
<summary>Windows, Linux and Mac from the repository</summary>
[Download Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) and unzip it, or `git clone` it.
**Windows:** double-click `START-HERE.bat`. **Linux:** run `./setup.sh` in the Strata folder.
**Mac (Apple Silicon, macOS 15 or newer):** install Apple's Command Line Tools once with `xcode-select --install`
(Xcode itself is not needed). Open Terminal in the Strata folder and run `./setup.sh`. It builds the Mac engine on your
Mac (about 7 minutes), asks the same questions, downloads the model and starts it. Next time, `./setup.sh` starts it
right away. Which size fits your Mac: [docs/MAC.md](docs/MAC.md).
The installer finds your card and sets up the right engine. It asks which model and size, how much context (how much text
the model keeps in mind) and whether it should read pictures. Press Enter for the recommended answer. Then it downloads the
model (about 70 GB) and starts it. If the download stops, run it again and it continues. Your browser opens the Strata
page at `http://127.0.0.1:8080`.
> While the model starts, your PC can be slow or stop responding for 1-3 minutes (longest the first time). Strata loads
> 35-55 GB into your RAM and locks part of it for the graphics card. Wait, and don't close the window.
Next time, run `START-HERE.bat` (or `./setup.sh`) again. It starts right away and downloads nothing twice. Close its
window to stop the model. `UPDATE.bat` (`./update.sh`) updates Strata without starting it. Updating, Docker, several cards,
where the files go and every option: [docs/INSTALL.md](docs/INSTALL.md).
To measure your PC, run `BENCH.bat` (`./bench.sh`). It takes 5-10 minutes and writes a report you can share
([how](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).
**Optional: more speed on one card when the whole model fits in RAM.** Two opt-in settings by RubenTorney (#1904, #1905).
They help when you run one graphics card and your RAM holds every expert, for example Q2_0 or IQ3_XXS on a 64 GB PC. Open
`strata-<model>.json` in the Strata data folder ([where](docs/INSTALL.md#where-things-are-stored)) and add:
- `"--adapt-async", "1"` to the `"args"` list. The expert cache then updates on a helper thread instead of pausing the
answer. Measured on an RTX 5070 (Windows, Q2_0): +4-5% decode, +7-10% on code and long documents. Answers can differ
slightly from the default and stay the same from run to run.
- `"STRATA_PIN_BY_NEED": "1"` in the `"env"` block (Windows). Windows can lock only about half the RAM for the graphics
card, and this locks the experts the card needs first. Measured on the RTX 5070: +4.4% prompt reading.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
Restart Strata after the change. To undo, remove the two entries.
</details>
## How does it work?
Models like this one usually run on servers with hundreds of gigabytes of graphics memory. Your graphics card has
12-24 GB. Strata makes the model fit by **sharing the work across your whole PC**. Think of a kitchen: the things
you use all the time stay on the counter, and the rest waits in the pantry.
12-24 GB. Strata makes the model fit by sharing the work across your whole PC. Picture a library.
- The model is a team of 24,576 small specialists ("experts"), like 24,576 reference books. Each word needs only 10 of them.
- **Your graphics card is the desk.** It is small and fast. The roughly 4,000 books used most often lie open on it.
- **Your RAM is the shelves in the reading room.** All 24,576 books are there. When a word needs a book that is not on the
desk, your processor reads it from the shelf at once, while the card keeps working.
- **Your SSD is the archive.** It holds a 29 GB lookup table. The model reads a few rows of it per word, so it does not
need to sit in RAM.
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
- **The model is a team of 24,576 small specialists ("experts").** Each word needs only 10 of them.
- **Your graphics card** keeps the few thousand experts that are used most often. **Your RAM** holds all of them,
and **your processor** works on the rest at the same time. **Your SSD** holds a big lookup table.
A small helper also guesses the next few words. The big model checks all of them at once and keeps the right ones. You get
the same answer, 1.6-1.8x sooner. Long texts are read in big pieces, up to 8,192 tokens at a time, at over 1,000 tokens
per second.
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
- **Guess, then check:** a small helper guesses the next few words. The big model checks them all at once. You get
the same answer, 1.6-1.8x sooner.
- **Long texts are read in big pieces** (up to 8,192 tokens at a time), at over 1,000 tokens per second.
The longer explanation: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Every part and its numbers:
[the details](docs/DETAILS.md#how-it-works) and the [paper](docs/paper/Strata-Paper.pdf).
## Something went wrong?
- **My PC froze the first time Strata started.** It loads the model into RAM. Wait, and don't close the window. Still
frozen after 10 minutes? Restart the PC, close other programs and try again, or pick a smaller size.
- **It stopped while downloading or installing.** Start the app or `strata launch` again, or run `START-HERE.bat` (or
`./setup.sh`) again. It continues where it stopped.
- **It is very slow, the disk light keeps blinking, or it says "the engine stopped unexpectedly".** Your PC does not have
enough free RAM. Close other programs (browsers use a lot), or pick a smaller size such as Q2_0 or IQ2_XS.
- **It says there is no engine for this PC.** In the app, open **Settings, Engine** and press **Update engine**. In the
Terminal, run `strata update`. `strata doctor` shows which engine files are installed.
- **It says port 8080 is already in use.** Strata is already running. Look for its window.
More problems and fixes: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Still stuck? Open an
[issue](https://github.com/Niko1221/Strata/issues). In the app, attach the file from **Settings, Engine, Diagnostics, View
logs, Save logs**. In the Terminal, paste the output of `strata doctor`. From source, attach `strata-<model>.log` from the
Strata folder. For a security problem, report it privately: [SECURITY.md](SECURITY.md).
## Credits and license
The model is [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) by the Qwen team. It was
compressed by [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5)
The model is [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) by the Qwen team. It was compressed by
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5), agentionai (Gyro)
and Unsloth. Strata uses parts of [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). All credits:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata is open source under the [MIT License](LICENSE). A few
parts and every model have their own licenses ([which ones](docs/HOW_IT_WORKS.md#license)).
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata is open source under the [MIT License](LICENSE). A few parts and
every model have their own licenses ([which ones](docs/HOW_IT_WORKS.md#license)).
## Contributing
Found a bug, or want to send a change or a benchmark? Read [CONTRIBUTING.md](CONTRIBUTING.md) first. It says what to
include so that others can check your work.
## Support Strata
Strata is free and open source. If it is useful to you, you can support its development:
Strata is free and MIT-licensed. I build it and test it on whatever hardware I can get. If it runs well on your PC, a coffee helps
me keep testing new cards and fixing what you report. No pressure. Using it and reporting what you find helps just as much.
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
Interested in sponsoring Strata? Contact me via [GitHub](https://github.com/Niko1221).
+335 -158
View File
@@ -1,217 +1,394 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
</picture>
</p>
[English](README.md) · [简体中文](README.zh-CN.md) · [日本語](README.ja.md) · [Deutsch](README.de.md) · [Français](README.fr.md) · [Español](README.es.md) · **Português**
<p align="center"><a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <b>Português</b></p>
<p align="center"><b>Rode um modelo de IA de 125 bilhões de parâmetros no seu próprio PC gamer</b><br>
Placa de vídeo NVIDIA ou AMD (12 GB ou mais) · Windows ou Linux · gratuito e de código aberto</p>
<h3 align="center">Rode um modelo de ponta no seu próprio PC gamer</h3>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="Um jardim com pagode em voxels que o modelo do Strata escreveu, rodando no navegador"></a><br>
<sub>Um jardim com pagode em voxels, feito com um único prompt numa RTX 5070 com o Strata (IQ3_S, contexto de 128K) ·
<p align="center">Suporte a NVIDIA, AMD, Intel · Windows, Linux, Mac · gratuito e de código aberto</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>Um jardim de pagodes em voxels, gerado com um único prompt, rodando numa RTX 5070 com o Strata (IQ3_S, contexto de 128K) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">vídeo completo (49 s)</a></sub></p>
O Strata roda o **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** num PC comum. É um modelo
de IA grande e inteligente que normalmente precisa de um servidor. Ele conversa, escreve código, entende imagens e
funciona com seus apps e agentes de programação. Nada sai do seu PC.
O Strata roda o **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**, um modelo de 125 bilhões de parâmetros que
normalmente precisa de um servidor, num PC comum. Ele conversa, escreve código, lê imagens e funciona com os seus aplicativos e
agentes de programação. Suas conversas e arquivos ficam no seu PC.
## Qual é a velocidade?
**Primeira vez aqui?** Veja [o que você precisa](#o-que-você-precisa) e depois [instale o app](#opção-a-o-app-strata).
Medimos em dois PCs gamer comuns. Um token equivale a cerca de ¾ de uma palavra.
- **Roda num PC gamer.** 32 GB somando memória RAM e memória da placa de vídeo já bastam para começar.
- **Funciona com as suas ferramentas.** Ele fala as APIs da OpenAI e da Anthropic, então Claude Code, Codex CLI, OpenCode e outros agentes se conectam a ele.
- **Lê imagens.** Já vêm ativadas.
- **Gratuito.** Licença MIT, sem conta, sem nuvem.
- **Privado.** Sem telemetria. O que o Strata baixa, quando usa a internet e como confere os downloads está em [docs/SAFETY.md](docs/SAFETY.md).
- **Escreve respostas:** a velocidade com que a resposta aparece numa conversa curta. 60 tokens por segundo é mais rápido do que você consegue ler.
- **Lê seu prompt:** a velocidade com que ele absorve o que você envia (aqui, um documento, código ou histórico de conversa de 32K tokens).
Versão 0.2.1 · mecanismo 0.1.46
<table>
<tr><th>NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB de RAM</th><th>AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB de RAM</th></tr>
<tr><td>
## O que você precisa
| Tamanho | Escreve respostas | Lê seu prompt |
| --- | ---: | ---: |
| **Q2_0** | 94 tokens/s | 2,650 tokens/s |
| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
| **Coder** | 55 tokens/s | 2,180 tokens/s |
**Você precisa de pelo menos 32 GB somando a memória RAM e a memória da placa de vídeo (VRAM).** Num Mac ou num Ryzen AI Max, isso é a
memória unificada. Mais memória permite um modelo maior ou mais rápido.
</td><td>
- Um PC de 32 GB com uma placa de 12 a 16 GB roda o Coder.
- 64 GB de RAM rodam todos os tamanhos.
| Tamanho | Escreve respostas | Lê seu prompt |
| --- | ---: | ---: |
| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
| **Coder** | 44 tokens/s | 1,420 tokens/s |
</td></tr>
</table>
NVIDIA: Q2_0 com o engine 0.1.36, as outras linhas com 0.1.26 (respostas de 4K, prompts de 32K). As tabelas
completas estão em [DETAILS.md](docs/DETAILS.md#speed-measured). Uma placa com mais VRAM é mais rápida: uma
RTX 3090 (24 GB) deve escrever cerca de 100-140 tokens por segundo. Conversas longas e outras placas:
[velocidade de cada modelo](docs/MODELS.md#how-fast-is-each-size), [resultados da comunidade](docs/COMMUNITY_BENCHMARKS.md).
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
<sub>O Strata é gratuito. Se ele roda bem no seu PC, um café ajuda a manter o trabalho nele.</sub></p>
## Do que você precisa
| | |
| Seu computador | Detalhes |
| --- | --- |
| **Placa de vídeo** | **NVIDIA** GeForce RTX séries 20, 30, 40 ou 50, ou **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 ou séries RX 6800 / 6900. Ela precisa de **12 GB de VRAM ou mais**. |
| **RAM** | 32 GB ou mais. Sua RAM decide [qual modelo](#qual-modelo-devo-escolher) cabe. Com 64 GB, roda qualquer tamanho. |
| **Disco** | Cerca de 80 GB livres. Use um SSD se puder: a primeira inicialização fica bem mais rápida. |
| **Sistema** | Windows 10 / 11 ou Linux, e um driver de vídeo atualizado da NVIDIA ou da AMD. |
| **NVIDIA** GeForce RTX série 20, 30, 40, 50 (12 GB ou mais) | Instale o app ou o Terminal. |
| **AMD** Radeon RX 6800 e mais novas, Radeon AI PRO R9700 (12 GB ou mais) | [Placas AMD](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md), [mecanismo Intel](docs/INTEL.md) |
| **Mac com Apple Silicon** (M1 ou mais novo, macOS 15 ou mais novo) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max (Strix Halo)** e outros sistemas de memória unificada | [Strix Halo](docs/STRIX_HALO.md) |
| Placas e processadores mais antigos | [GPUs antigas](docs/OLDER_GPUS.md), [CPUs antigas](docs/INSTALL.md#older-cpus-experimental) |
O instalador cuida de todo o resto. Duas ou três placas podem dividir o modelo ([multi-GPU](docs/MULTI_GPU.md)).
Você também precisa de:
Experimental, escrito e testado por membros da comunidade nas próprias máquinas:
- cerca de 80 GB de espaço livre no disco (um SSD deixa a primeira inicialização bem mais rápida)
- Windows 10 / 11, Linux ou macOS
- um driver de vídeo atualizado
- **Placas de vídeo mais antigas** (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT): [Older GPUs](docs/OLDER_GPUS.md).
- **Intel Arc**, compilado a partir do código-fonte no Linux: [Intel Arc](docs/INTEL_ARC.md).
- **AMD Ryzen AI Max (Strix Halo)**, compilado a partir do código-fonte no Linux: [Strix Halo](docs/STRIX_HALO.md).
- **Processadores mais antigos sem AVX2**: funcionam, mas devagar. [Older CPUs](docs/INSTALL.md#older-cpus-experimental).
A lista completa: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
Duas ou três placas da mesma marca podem dividir um modelo ([multi-GPU](docs/MULTI_GPU.md)). O Strata nunca divide um modelo entre
placas NVIDIA e AMD. Lista completa: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
## Instalação
### Deixe sua IA instalar
Escolha uma opção. Todas iniciam o mesmo mecanismo e usam os mesmos modelos.
Você usa um assistente de programação com IA (Claude Code, Cursor, Codex, GitHub Copilot, ...)? Cole isto nele:
### Opção A: o app Strata
O jeito mais fácil. O app já traz o mecanismo: instale, escolha um modelo e converse. Baixe o arquivo para o seu sistema e a sua
placa de vídeo na [página de releases](https://github.com/Niko1221/Strata/releases/latest). Não sabe qual? Veja a explicação
logo abaixo da tabela.
| Sistema | Sua placa de vídeo | Arquivo para baixar |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX, driver 580 ou mais novo | [`Strata-0.2.1-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda13.exe) |
| | NVIDIA, placa mais antiga ou driver anterior ao 580 | [`Strata-0.2.1-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda12.exe) |
| | AMD Radeon série RX 9000 ou Radeon AI PRO R9700 | [`Strata-0.2.1-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip.exe) |
| | AMD Radeon série RX 7000 ou Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX, driver 580 ou mais novo | [`Strata-0.2.1-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda13.AppImage) |
| | NVIDIA, placa mais antiga ou driver anterior ao 580 | [`Strata-0.2.1-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon série RX 9000 ou Radeon AI PRO R9700 | [`Strata-0.2.1-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx9070.AppImage) |
| | AMD Radeon série RX 7000 | [`Strata-0.2.1-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-strixhalo.AppImage) |
| **Mac** (Apple Silicon, macOS 15 ou mais novo) | integrada | [`Strata-0.2.1-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-mac.dmg) |
**Se você estiver em dúvida:** para NVIDIA, use `cuda13` se o seu driver for a versão 580 ou mais nova, e `cuda12` nos outros casos. O `cuda12`
serve para placas e drivers mais antigos. Os arquivos `hip` e `amd` são para placas AMD. Outras placas AMD (RX 6000) ainda não têm app pronto: veja [Placas AMD](docs/AMD_HIP.md).
Depois:
- **Windows:** execute o `.exe`. Ele instala só para o seu usuário e não pede direitos de administrador. Se o Windows disser "O Windows
protegeu o computador", escolha **Mais informações** e depois **Executar assim mesmo** (o instalador ainda não tem assinatura de código).
- **Linux:** torne o arquivo executável (`chmod +x`) e inicie. É preciso ter o FUSE 2: `sudo apt install libfuse2` (Ubuntu 22.04)
ou `libfuse2t64` (Ubuntu 24.04 e mais novos).
- **Mac:** abra o `.dmg` e arraste o Strata para Aplicativos.
Na primeira tela, o app lê o seu PC e recomenda um modelo que caiba nele. Clique em instalar. O modelo é um download grande
(cerca de 40 a 70 GB) e continua de onde parou se for interrompido. Fechar a janela encerra o Strata e o mecanismo.
### Opção B: o Terminal (uma linha)
O Terminal é o comando `strata`, para scripts, servidores e agentes de IA.
Linux e Mac:
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
```
Windows (PowerShell):
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
O script confere o seu PC, baixa o Terminal, verifica o SHA-256 e descompacta na sua pasta pessoal. Não precisa de direitos de
administrador. No Windows, abra uma nova janela do terminal depois. Então inicie o Strata:
```sh
strata launch
```
Ele faz uma pergunta por vez: modelo, contexto, imagens, quem pode se conectar. Depois baixa o mecanismo para a sua placa de vídeo
(assinada e testada antes do uso) e o modelo, e começa a servir. Em scripts e agentes de IA, use flags:
```sh
strata launch --model q2_0 --yes --no-tui
```
Para ler o script de instalação antes de executá-lo, baixe-o primeiro com `curl -fsSL <address> -o install.sh`, leia e então rode
`sh install.sh`. Todas as telas e flags: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md).
### Opção C: deixe a sua IA configurar
Cole isto no Claude Code, Cursor, Codex, GitHub Copilot ou outro assistente de programação:
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
Ele verifica sua placa de vídeo, RAM e disco e escolhe o modelo que cabe. Depois instala, inicia e explica como
conectar seus apps. Ferramentas de IA também podem instalar, iniciar e parar o Strata pelo
[servidor MCP](docs/MCP_SERVER.md) dele.
Ele confere a sua placa de vídeo, a RAM e o disco, escolhe o modelo que cabe, instala e inicia, e diz como conectar os seus
aplicativos. Ferramentas de IA também podem instalar, iniciar e parar o Strata pelo [servidor MCP](docs/MCP_SERVER.md) dele.
### Ou faça você mesmo
Prefere compilar por conta própria? Veja [Compilar a partir do código-fonte](#compilar-a-partir-do-código-fonte).
[Baixe o Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) e descompacte (ou use `git clone`).
**Windows:** clique duas vezes em **`START-HERE.bat`**. **Linux:** rode **`./setup.sh`** na pasta do Strata.
## Modelos
Os passos são os mesmos para NVIDIA e AMD. O instalador encontra sua placa e configura o engine certo para ela.
Ele faz algumas perguntas:
- qual modelo e qual tamanho,
- quanto contexto (quanto texto o modelo guarda na memória),
- se ele deve entender imagens.
Aperte Enter em cada uma para usar a resposta recomendada. Depois ele baixa o modelo (cerca de 70 GB) e o inicia.
Se o download parar, rode de novo: ele continua de onde parou. Seu navegador abre o app do Strata em
`http://127.0.0.1:8080`.
> **Enquanto o modelo inicia, seu PC pode ficar lento ou parar de responder por 1-3 minutos** (mais na primeira vez).
> O Strata carrega 35-55 GB na sua RAM e reserva parte dela para a placa de vídeo. Isso é normal. Espere e não
> feche a janela. A janela mostra o que o Strata está fazendo.
**Da próxima vez**, rode `START-HERE.bat` (ou `./setup.sh`) de novo. Ele inicia na hora e não baixa nada duas
vezes. Feche a janela para parar o modelo. `UPDATE.bat` (`./update.sh`) atualiza o Strata sem iniciá-lo.
Atualização, Docker, várias placas, onde ficam os arquivos e todas as opções: [docs/INSTALL.md](docs/INSTALL.md).
## Qual modelo devo escolher?
O instalador recomenda um de acordo com sua RAM. O mesmo modelo vem em vários tamanhos, mais ou menos comprimidos.
O app e o `strata launch` recomendam um modelo para a sua memória. É um único modelo em vários tamanhos, mais ou menos comprimidos.
Os tamanhos menores são mais rápidos. Os maiores são um pouco mais inteligentes.
| Sua RAM | Escolha | Por quê |
### Pouca memória? Experimente o Gyro
O [Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) (da agentionai) é a menor forma de rodar este modelo.
| Tamanho | Memória necessária (RAM + VRAM) | Download | O que é |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27,6 GB | 58,5 GB | Para PCs onde nenhum outro tamanho cabe. A qualidade é menor que a do IQ2_XS. |
| **Gyro-M** | 35 GB | 92,0 GB | Qualidade próxima à do IQ3_XXS, com cerca de 20% menos memória da GPU. Ele escreve as respostas 17 a 21% mais devagar que o Q2_0 numa placa de 12 GB e numa R9700. |
O Gyro roda em placas NVIDIA e AMD (AMD é experimental), e não roda em Intel Arc nem no Mac. Detalhes e medições:
[docs/GYRO.md](docs/GYRO.md). Inicie com `strata launch --model gyro-s`.
### Os outros tamanhos
| Sua memória (RAM + VRAM) | Escolha | Observações |
| --- | --- | --- |
| **32 GB** | **Coder** | cabe em 32 GB e é feito para código (com uma placa de 24 GB, Q2_0 e IQ2_XS também rodam) |
| **48 GB** | **IQ2_XS** (ou Q2_0, o mais rápido) | os tamanhos maiores não cabem |
| **64 GB** | **IQ2_XS** (recomendado), ou IQ3_XXS / IQ3_S | todos os tamanhos cabem; IQ3_S é o melhor e o mais lento |
| **96 GB ou mais** | **IQ3_S**, ou o UD-IQ4_XS da Unsloth (~4-bit) | espaço para os maiores tamanhos com todo o resto aberto |
| **32 GB** | **Gyro-S**, ou o **[Coder](docs/MODELS.md#coder)** para código | O Coder precisa de 32 GB de RAM e mantém metade dos experts, então é mais fraco fora de código |
| **40 GB** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS (39,2 GB), ou **Gyro-M** (35 GB) | O Swift pensa por menos tempo antes de responder. O Gyro-M chega perto do IQ3_XXS em qualidade, mas escreve as respostas 17 a 21% mais devagar que o Q2_0 |
| **48 GB** | **IQ3_XXS** (47,0 GB) | Mais lento, melhor |
| **64 GB** | **IQ3_S** (54,8 GB), o original ou o Swift 1.5 | O mais lento, o melhor: empata com o modelo completo nos testes publicados |
| **96 GB ou mais** | **UD-IQ4_XS** da Unsloth (cerca de 72 GB de RAM) até **Q8_0** (cerca de 136 GB), de 4 a 8 bits | Mais perto do modelo completo. Com menos RAM, o Strata lê parte deles do SSD, o que é bem mais lento |
- **[Coder](docs/MODELS.md#coder):** uma versão para programação com metade dos experts removida. Ela alcança 91%
da pontuação do modelo completo no SWE-bench Verified (medido pelos autores) e cabe em 32 GB de RAM. É mais fraca
fora do código, inclusive em chinês e outros textos CJK (#438). Para isso, escolha Q2_0, IQ2_XS ou IQ3_S, que
mantêm todos os experts.
- **[Swift 1.5](docs/MODELS.md#swift-15):** um fine-tune que pensa por bem menos tempo antes de responder. Você
recebe a resposta mais cedo, com quase a mesma qualidade.
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** a versão de ~4 bits da Unsloth, entre IQ3_S e
UD-Q4_K_XL em qualidade. Um download de 94 GB. Com menos de ~80 GB de RAM, o Strata lê uma parte dele do SSD
enquanto responde, então ali ele fica mais lento (um SSD NVMe ajuda).
- **[Unsloth UD-Q4_K_XL](docs/MODELS.md#unsloth-ud-q4_k_xl-experimental)** (experimental): o mais próximo do modelo
completo. Mas o Strata lê a maior parte dele do SSD enquanto responde, então ele escreve só 7-8.5 tokens/s num PC
com 64 GB.
- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** você configura à mão. Ele
não está no menu do instalador.
Com menos RAM do que a linha indica, ainda pode funcionar: o Strata muda sozinho para o
[modo de pouca RAM](docs/MODELS.md#a-big-graphics-card-and-little-ram). Espere mais lentidão, e muito mais com uma placa de vídeo pequena.
Tamanhos, downloads e o que cabe onde: [docs/MODELS.md](docs/MODELS.md). Para adicionar outro modelo depois, rode
`SETUP.bat` (Linux: `./setup.sh --setup`).
Mais opções:
- **[Swift 1.5](docs/MODELS.md#swift-15)** é um ajuste fino que pensa por menos tempo antes de responder, com qualidade e RAM parecidas.
- **[NVFP4](docs/NVFP4.md)** é a versão de 4 bits da NVIDIA para placas NVIDIA. Precisa de cerca de 80 GB de RAM para manter todos os experts na RAM.
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** é outro modelo, menor, que lê imagens e pode atender vários pedidos ao mesmo tempo.
- **[Uncensored IQ3_XXS da OrcaRouter](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** é configurado manualmente.
Todos os tamanhos, downloads e o que cabe em cada configuração: [docs/MODELS.md](docs/MODELS.md).
## Qual é a velocidade?
Com base numa placa de vídeo de 12 GB (RTX 5070, Ryzen 5 7600, 64 GB de RAM). Veja resultados de outras placas:
[velocidade de cada modelo](docs/MODELS.md#how-fast-is-each-size), [resultados da comunidade](docs/COMMUNITY_BENCHMARKS.md).
| Modelo | As respostas aparecem a | Lê o seu prompt a |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2.650 tokens/s |
| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2.090 tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1.750 tokens/s |
| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1.620 tokens/s |
| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2.180 tokens/s |
Um token equivale a cerca de ¾ de palavra, então 60 tokens por segundo é mais rápido do que você lê. "Lê o seu prompt" é a velocidade com que o Strata
absorve um documento longo, um código ou um histórico de conversa.
Medido numa placa AMD de 16 GB (RX 9070 XT, Ryzen 9 3900X, 47 GB de RAM):
| Modelo | As respostas aparecem a | Lê o seu prompt a |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1.160 tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1.110 tokens/s |
| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1.420 tokens/s |
As barras usam a mesma escala nas duas tabelas. Uma placa com mais VRAM é mais rápida: uma RTX 3090 (24 GB) deve escrever cerca de 100 a 140
tokens por segundo. A versão do mecanismo por trás de cada número está em [DETAILS.md](docs/DETAILS.md#speed-measured).
## Como usar
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="A aba Monitor do app Strata ao lado de um agente de programação"><br>
<sub>O <b>Monitor</b> do app Strata (à esquerda) enquanto um agente de programação escreve o jardim com pagode do vídeo (à direita)</sub></p>
### No app
- **No navegador:** abra `http://127.0.0.1:8080`. Lá tem o **Chat**, um **Monitor** ao vivo do modelo e da sua
GPU/CPU/RAM, e o **About** com as configurações e os endereços.
- **Seus apps e agentes de programação:** adicione um provedor "OpenAI-compatible" com a URL base
**`http://127.0.0.1:8080/v1`**. Qualquer chave de API e qualquer nome de modelo funcionam.
- Apps que usam a API da Anthropic: `http://127.0.0.1:8080/v1/messages` (Claude Code:
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`).
- Codex CLI e outros apps que usam a OpenAI Responses API: `/v1/responses`
([configuração](docs/DETAILS.md#the-responses-api-and-codex-cli)).
- **Raciocínio:** escolha **off, low, medium ou high** no menu do chat ou no "reasoning effort" do seu app. Off é o
mais rápido. High é o melhor para perguntas difíceis.
- **Imagens:** responda sim para "Images?" na instalação. Depois clique em **Picture** no chat, ou anexe imagens no
seu app. Placas AMD entendem imagens no Linux usando o processador; no Windows, ainda não.
- **Do seu celular ou de outro PC:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`. Sempre defina uma chave.
- **Uma requisição por vez:** por padrão, o Strata responde uma requisição e as outras esperam. Para responder
várias ao mesmo tempo, defina `"parallel": 2` ([BATCHING.md](docs/BATCHING.md)). Numa placa de 12 GB, isso deixa
cada resposta mais lenta.
- **Prompts longos:** o Strata lê a primeira mensagem de uma conversa inteira, cerca de 1 minuto a cada 30,000
tokens. As mensagens seguintes começam em segundos.
O app tem um **Chat**, um **Monitor** do modelo e da sua GPU, CPU e RAM, uma página de **Modelos** e as **Configurações**. O chat
pode trabalhar numa pasta do seu PC. No modo **Manual**, ele pergunta antes de alterar um arquivo ou executar um comando. Digite
`/loop 10m <o que fazer>` para repetir uma mensagem a cada 10 minutos enquanto o chat estiver aberto.
Mais: [onde suas conversas ficam salvas](docs/INSTALL.md#where-things-are-stored), [a API](docs/DETAILS.md#using-it).
O chat pode pesquisar na web com o DuckDuckGo (ou o Bing), sem conta. Você pode adicionar uma chave de busca em **Configurações, Ferramentas**.
A busca é o único recurso que envia algo para fora do seu PC: os termos da pesquisa.
## Algo deu errado?
Para rodar dois modelos, ative **Executar vários modelos** e dê a cada modelo a sua própria placa da mesma marca.
- **Meu PC travou na primeira vez que o Strata iniciou.** Isso é normal enquanto ele carrega o modelo. Espere e não
feche a janela. Ainda travado depois de 10 minutos? Reinicie o PC, feche outros programas e tente de novo, ou
escolha um tamanho menor.
- **Parou durante o download ou a instalação.** Rode `START-HERE.bat` (ou `./setup.sh`) de novo. Ele continua de
onde parou.
- **Está muito lento e a luz do disco não para de piscar, ou aparece "the engine stopped unexpectedly".** Seu PC
não tem RAM livre suficiente. Feche outros programas (navegadores usam muita), ou escolha um tamanho menor (Q2_0
ou IQ2_XS).
- **Diz que a porta 8080 já está em uso.** O Strata já está rodando. Procure a janela dele.
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>Um chat no app Strata: o agente escreveu um pequeno jardim de pagode em voxels em um único arquivo HTML, e a página abre na pré-visualização ao lado da conversa</sub></p>
Mais problemas e como resolver: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Ainda sem solução? Abra uma
[issue](https://github.com/Niko1221/Strata/issues) e anexe o `strata-<model>.log` da pasta do Strata. Encontrou um
problema de segurança? Informe em particular: [SECURITY.md](SECURITY.md).
### No Terminal
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub><code>strata monitor</code> enquanto o modelo responde: velocidade, GPU, memória e as solicitações recentes</sub></p>
```sh
strata launch # telas guiadas: modelo, contexto, imagens, acesso e depois servir
strata launch --model q2_0 --yes --no-tui # sem perguntas: serve, mostra o endereço e volta
strata launch --plan --model q2_0 # mostra o que aconteceria; nada muda
strata launch --status # o que está sendo servido agora
strata launch --stop # para o serviço
strata monitor --url http://127.0.0.1:8080 # acompanha velocidade, GPU, memória e requisições
strata update --check # há um mecanismo novo? (strata update instala)
strata doctor # confere os arquivos do mecanismo instalada
```
`strata --help` lista tudo. Iniciar e servir nunca baixam nada. Só a instalação, o `strata launch` (quando você
escolhe um modelo) e o `strata update` baixam. Para acessar o Strata do celular ou de outro PC, use
`strata launch --access lan`. Ele cria uma chave de API e não inicia numa rede sem uma. Nenhum navegador abre, a menos que
você adicione `--open`. Guia: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). Cliente completo: [docs/TERMINAL.md](docs/TERMINAL.md).
### Com agentes de IA
Um agente instala o Strata com o comando de uma linha da Opção B e executa
`strata launch --model NAME --yes --no-tui`. Os passos, as flags e os códigos de saída estão em [docs/AI_SETUP.md](docs/AI_SETUP.md),
e as regras do repositório estão em [AGENTS.md](AGENTS.md).
### Conecte os seus outros aplicativos
| Tipo de aplicativo | URL base |
| --- | --- |
| Compatível com OpenAI (a maioria dos apps e agentes) | `http://127.0.0.1:8080/v1` (qualquer chave de API, qualquer nome de modelo) |
| API da Anthropic (Claude Code e outros) | `http://127.0.0.1:8080`, sem `/v1`. Esses aplicativos adicionam `/v1/messages` sozinhos. Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| OpenAI Responses API (Codex CLI) | `http://127.0.0.1:8080/v1/responses` ([configuração](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` mostra as configurações exatas para OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent e OpenClaw.
`strata connect opencode --write` grava o arquivo depois de perguntar e mantém um backup.
### Imagens
As imagens vêm ativadas nos modelos que sabem lê-las. Anexe ou cole uma imagem no chat, ou envie pelo seu aplicativo.
O leitor de imagens roda no processador quando a sua placa de vídeo tem menos de 24 GB (alguns segundos por imagem, sem tirar nada
da memória da placa) e roda na placa de vídeo a partir de 24 GB. A regra é a mesma para NVIDIA e AMD. Desative as imagens
por modelo, ou escolha o leitor com `--vision off`, `--vision gpu` ou `--vision cpu`.
Os números por trás da regra: numa Radeon AI PRO R9700, o leitor da placa levou 57 ms para uma imagem de 260 tokens e o do
processador levou 3,1 s. Em placas de 16 GB (Tesla P100 e RTX A4000, Linux), o leitor da placa custou de 3 a 5% da velocidade do texto,
porque ocupa cerca de 1,4 GB que o modelo usaria de outra forma. No Mac, a GPU faz a leitura, seja qual for a memória (0,45 s para uma imagem de 260 tokens em um M4 de 32 GB, cerca de 1,2 GB da memória compartilhada);
o processador leva cerca de 50 s por imagem, então só lê com `--vision cpu`.
### Bom saber
- Escolha o nível de raciocínio (**off, low, medium ou high**) no menu do chat ou no "reasoning effort" do seu aplicativo. Off é o mais rápido.
- Por padrão, o Strata responde a um pedido por vez. Para vários ao mesmo tempo, use `--parallel N` no `strata launch` ou
`"parallel": 2` na configuração ([BATCHING.md](docs/BATCHING.md)). Numa placa de 12 GB, isso deixa cada resposta mais lenta.
- O Strata lê a primeira mensagem de uma conversa por inteiro, cerca de 1 minuto a cada 30.000 tokens. As mensagens seguintes começam em segundos.
- Onde ficam as suas conversas e a API completa: [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored), [docs/DETAILS.md](docs/DETAILS.md#using-it).
## Compilar a partir do código-fonte
<details>
<summary>Windows, Linux e Mac a partir do repositório</summary>
[Baixe o Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) e descompacte, ou use `git clone`.
**Windows:** dê dois cliques em `START-HERE.bat`. **Linux:** execute `./setup.sh` na pasta do Strata.
**Mac (Apple Silicon, macOS 15 ou mais novo):** instale uma vez as Command Line Tools da Apple com `xcode-select --install`
(o Xcode em si não é necessário). Abra o Terminal na pasta do Strata e execute `./setup.sh`. Ele compila o mecanismo do Mac no seu
Mac (cerca de 7 minutos), faz as mesmas perguntas, baixa o modelo e inicia. Da próxima vez, `./setup.sh` inicia
na hora. Qual tamanho cabe no seu Mac: [docs/MAC.md](docs/MAC.md).
O instalador encontra a sua placa e configura o mecanismo certo. Ele pergunta qual modelo e tamanho, quanto contexto (quanto texto
o modelo mantém em mente) e se deve ler imagens. Pressione Enter para aceitar a resposta recomendada. Depois ele baixa o
modelo (cerca de 70 GB) e inicia. Se o download parar, execute de novo e ele continua. O navegador abre a página do Strata em
`http://127.0.0.1:8080`.
> Enquanto o modelo inicia, o seu PC pode ficar lento ou parar de responder por 1 a 3 minutos (mais tempo na primeira vez). O Strata carrega
> de 35 a 55 GB na sua RAM e trava parte dela para a placa de vídeo. Espere e não feche a janela.
Da próxima vez, execute `START-HERE.bat` (ou `./setup.sh`) de novo. Ele inicia na hora e não baixa nada duas vezes. Feche a janela
dele para parar o modelo. `UPDATE.bat` (`./update.sh`) atualiza o Strata sem iniciá-lo. Atualização, Docker, várias placas,
onde ficam os arquivos e todas as opções: [docs/INSTALL.md](docs/INSTALL.md).
Para medir o seu PC, execute `BENCH.bat` (`./bench.sh`). Leva de 5 a 10 minutos e gera um relatório que você pode compartilhar
([como](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).
**Opcional: mais velocidade numa placa quando o modelo inteiro cabe na RAM.** Duas configurações opcionais do RubenTorney (#1904, #1905).
Elas ajudam quando você usa uma placa de vídeo e a sua RAM comporta todos os experts, por exemplo Q2_0 ou IQ3_XXS num PC de 64 GB. Abra
`strata-<model>.json` na pasta de dados do Strata ([onde fica](docs/INSTALL.md#where-things-are-stored)) e adicione:
- `"--adapt-async", "1"` à lista `"args"`. O cache de experts passa a ser atualizado numa thread auxiliar, em vez de pausar a
resposta. Medido numa RTX 5070 (Windows, Q2_0): +4 a 5% na geração, +7 a 10% em código e documentos longos. As respostas podem diferir
um pouco do padrão e continuam iguais de uma execução para outra.
- `"STRATA_PIN_BY_NEED": "1"` no bloco `"env"` (Windows). O Windows só consegue travar cerca de metade da RAM para a placa
de vídeo, e isso trava primeiro os experts de que a placa precisa. Medido na RTX 5070: +4,4% na leitura do prompt.
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
Reinicie o Strata depois da alteração. Para desfazer, remova as duas entradas.
</details>
## Como funciona?
Modelos como este normalmente rodam em servidores com centenas de gigabytes de memória de vídeo. Sua placa de vídeo
tem 12-24 GB. O Strata faz o modelo caber **dividindo o trabalho pelo PC inteiro**. Pense numa cozinha: o que você
usa o tempo todo fica na bancada, e o resto espera na despensa.
Modelos como este costumam rodar em servidores com centenas de gigabytes de memória de vídeo. A sua placa de vídeo tem
12 a 24 GB. O Strata faz o modelo caber dividindo o trabalho entre o PC inteiro. Imagine uma biblioteca.
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="Os 24,576 experts do modelo: os mais usados na placa de vídeo, todos na RAM, uma tabela de consulta no SSD"></p>
- O modelo é uma equipe de 24.576 pequenos especialistas ("experts"), como 24.576 livros de consulta. Cada palavra precisa de apenas 10 deles.
- **A sua placa de vídeo é a mesa.** Ela é pequena e rápida. Os cerca de 4.000 livros mais usados ficam abertos nela.
- **A sua RAM são as estantes da sala de leitura.** Os 24.576 livros estão lá. Quando uma palavra precisa de um livro que não está na
mesa, o seu processador lê da estante na hora, enquanto a placa continua trabalhando.
- **O seu SSD é o arquivo.** Ele guarda uma tabela de consulta de 29 GB. O modelo lê algumas linhas dela por palavra, então ela não
precisa ficar na RAM.
- **O modelo é uma equipe de 24,576 pequenos especialistas ("experts").** Cada palavra precisa de só 10 deles.
- **Sua placa de vídeo** guarda os poucos milhares de experts usados com mais frequência. **Sua RAM** guarda todos
eles, e **seu processador** trabalha no resto ao mesmo tempo. **Seu SSD** guarda uma grande tabela de consulta.
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="Um pequeno ajudante adivinha as próximas palavras; o modelo grande confere todas de uma vez e fica com as certas"></p>
Um pequeno ajudante também adivinha as próximas palavras. O modelo grande confere todas de uma vez e fica com as certas. Você recebe
a mesma resposta, de 1,6 a 1,8 vez mais rápido. Textos longos são lidos em blocos grandes, de até 8.192 tokens por vez, a mais de 1.000 tokens
por segundo.
- **Adivinhar e conferir:** um pequeno ajudante adivinha as próximas palavras. O modelo grande confere todas de
uma vez. Você recebe a mesma resposta, 1.6-1.8x mais cedo.
- **Textos longos são lidos em grandes pedaços** (até 8,192 tokens por vez), a mais de 1,000 tokens por segundo.
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
A explicação mais longa: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Cada parte e seus números:
[os detalhes](docs/DETAILS.md#how-it-works) e o [artigo](docs/paper/Strata-Paper.pdf).
## Algo deu errado?
- **Meu PC travou na primeira vez que o Strata iniciou.** Ele carrega o modelo na RAM. Espere e não feche a janela. Ainda
travado depois de 10 minutos? Reinicie o PC, feche outros programas e tente de novo, ou escolha um tamanho menor.
- **Parou durante o download ou a instalação.** Abra o app ou o `strata launch` de novo, ou execute `START-HERE.bat` (ou
`./setup.sh`) outra vez. Ele continua de onde parou.
- **Está muito lento, a luz do disco não para de piscar, ou aparece "o mecanismo parou inesperadamente".** O seu PC não tem
RAM livre suficiente. Feche outros programas (os navegadores usam muita), ou escolha um tamanho menor, como Q2_0 ou IQ2_XS.
- **Diz que não há mecanismo para este PC.** No app, abra **Configurações, Mecanismo** e clique em **Atualizar mecanismo**. No
Terminal, execute `strata update`. O `strata doctor` mostra quais arquivos do mecanismo estão instalados.
- **Diz que a porta 8080 já está em uso.** O Strata já está rodando. Procure a janela dele.
Mais problemas e soluções: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Ainda sem resolver? Abra uma
[issue](https://github.com/Niko1221/Strata/issues). No app, anexe o arquivo de **Configurações, Mecanismo, Diagnóstico, Ver
registros, Salvar logs**. No Terminal, cole a saída de `strata doctor`. Se estiver rodando pelo código-fonte, anexe o `strata-<model>.log` da pasta do
Strata. Para um problema de segurança, informe em particular: [SECURITY.md](SECURITY.md).
## Créditos e licença
O modelo é o [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), da equipe Qwen. Ele foi
comprimido por [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5)
O modelo é o [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), da equipe Qwen. Ele foi comprimido por
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5), agentionai (Gyro)
e Unsloth. O Strata usa partes do [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). Todos os créditos:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). O Strata é de código aberto sob a [MIT License](LICENSE).
Algumas partes e todos os modelos têm licenças próprias ([quais](docs/HOW_IT_WORKS.md#license)).
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). O Strata é de código aberto sob a [Licença MIT](LICENSE). Algumas partes e
todos os modelos têm licenças próprias ([quais](docs/HOW_IT_WORKS.md#license)).
## Contribuindo
Encontrou um bug, ou quer enviar uma alteração ou um benchmark? Leia primeiro o [CONTRIBUTING.md](CONTRIBUTING.md). Ele diz o que
incluir para que outras pessoas possam conferir o seu trabalho.
## Apoie o Strata
O Strata é gratuito e de código aberto. Se ele é útil para você, você pode apoiar o desenvolvimento:
O Strata é gratuito e tem licença MIT. Eu o desenvolvo e testo em todo hardware que consigo arranjar. Se ele roda bem no seu PC, um café me ajuda
a continuar testando placas novas e corrigindo o que você relatar. Sem pressão. Usar e relatar o que encontrar ajuda tanto quanto.
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
Quer patrocinar o Strata? Fale comigo pelo [GitHub](https://github.com/Niko1221).
+327 -146
View File
@@ -1,200 +1,381 @@
<h1 align="center">Strata</h1>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">
<img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">
</picture>
</p>
[English](README.md) · **简体中文** · [日本語](README.ja.md) · [Deutsch](README.de.md) · [Français](README.fr.md) · [Español](README.es.md) · [Português](README.pt-BR.md)
<p align="center"><a href="README.md">English</a> · <b>简体中文</b> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a></p>
<p align="center"><b>在你自己的游戏电脑上运行 1250 亿参数的 AI 模型</b><br>
NVIDIA 或 AMD 显卡(12 GB 及以上)· Windows 或 Linux · 免费开源</p>
<h3 align="center">在你自己的游戏电脑上运行最先进的模型</h3>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="Strata 的模型写出的体素宝塔花园,在浏览器中运行"></a><br>
<sub>体素宝塔花园,一次提示生成,在 RTX 5070 上用 Strata 运行(IQ3_S,128K 上下文)·
<p align="center">支持 NVIDIA、AMD、Intel · Windows、Linux、Mac · 免费开源</p>
<p align="center">
<a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>
<a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>
</p>
<p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
<sub>一座体素宝塔花园,由一条提示词一次生成,在装有 RTX 5070 的电脑上用 Strata 运行(IQ3_S,128K 上下文) ·
<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">完整视频(49 秒)</a></sub></p>
Strata 能在普通电脑上运行 **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**。这是一个又大又聪明的
AI 模型,通常要用服务器才能跑。它能聊天、写代码、看图片,还能配合你的应用和编程智能体一起工作。所有数据都留在你的电脑上。
Strata 运行 **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**。这是一个 1250 亿参数的模型,
通常需要服务器才能跑,现在用普通电脑就行。它能聊天、写代码、看图片,也能配合你的应用和编程智能体使用。
你的聊天记录和文件都留在你自己的电脑上。
## 速度有多快?
**第一次用?** 先看[硬件要求](#硬件要求),再[安装应用](#方式-astrata-应用)。
我们在两台普通的游戏电脑上做了测试。一个 token 大约相当于 ¾ 个英文单词。
- **游戏电脑就能跑。** 内存和显存加起来有 32 GB 就可以开始。
- **能接入你的工具。** 支持 OpenAI 和 Anthropic 的 API,Claude Code、Codex CLI、OpenCode 等智能体都能连上。
- **能看图片。** 默认开启。
- **免费。** MIT 许可证,不用注册账号,不走云端。
- **隐私。** 没有遥测。Strata 下载什么、什么时候联网、如何校验下载内容,都写在 [docs/SAFETY.md](docs/SAFETY.md)。
- **写回答:** 短对话中回复出现的速度。每秒 60 个 token 已经比你的阅读速度还快。
- **读提示:** 读入你发送内容的速度(这里是一份 32K token 的文档、代码或聊天记录)。
版本 0.2.1 · 引擎 0.1.46
<table>
<tr><th>NVIDIA:RTX 5070(12 GB)、Ryzen 5 7600、64 GB 内存</th><th>AMD:RX 9070 XT(16 GB)、Ryzen 9 3900X、47 GB 内存</th></tr>
<tr><td>
## 硬件要求
| 规格 | 写回答 | 读提示 |
| --- | ---: | ---: |
| **Q2_0** | 94 tokens/s | 2,650 tokens/s |
| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
| **Coder** | 55 tokens/s | 2,180 tokens/s |
**内存和显存(VRAM)加起来至少要有 32 GB。** Mac 和 Ryzen AI Max 上指的是统一内存。内存越大,能跑的模型越大,
或者速度越快。
</td><td>
- 32 GB 内存加一张 12-16 GB 显存的显卡,可以跑 Coder。
- 64 GB 内存可以跑所有尺寸。
| 规格 | 写回答 | 读提示 |
| --- | ---: | ---: |
| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
| **Coder** | 44 tokens/s | 1,420 tokens/s |
</td></tr>
</table>
NVIDIA:Q2_0 用的是引擎 0.1.36,其他各行用的是 0.1.26(4K 回答,32K 提示)。完整表格见
[DETAILS.md](docs/DETAILS.md#speed-measured)。显存越大的显卡越快:RTX 3090(24 GB)写回答应该能达到每秒约
100-140 个 token。长对话和其他显卡的数据:[各模型的速度](docs/MODELS.md#how-fast-is-each-size)、
[社区测试结果](docs/COMMUNITY_BENCHMARKS.md)。
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="请我喝杯咖啡" height="50"></a><br>
<sub>Strata 是免费的。如果它在你的电脑上跑得不错,请我喝杯咖啡,让这个项目继续下去。</sub></p>
## 你需要什么
| | |
| 你的电脑 | 说明 |
| --- | --- |
| **显卡** | **NVIDIA** GeForce RTX 20、30、40 或 50 系列,或 **AMD** Radeon RX 7900 XT / XTX、RX 7800 XT / 7700 XT、RX 9060 XT、RX 9070 / 9070 XT、Radeon AI PRO R9700 或 RX 6800 / 6900 系列。需要 **12 GB 或以上显存**。 |
| **内存** | 32 GB 或以上。内存大小决定能装下[哪个模型](#该选哪个模型)。64 GB 可以运行所有规格。 |
| **硬盘** | 约 80 GB 可用空间。尽量用 SSD:第一次启动会快很多。 |
| **系统** | Windows 10 / 11 或 Linux,以及 NVIDIA 或 AMD 的最新显卡驱动。 |
| **NVIDIA** GeForce RTX 20、30、40、50 系列(12 GB 及以上) | 安装应用或终端版即可。 |
| **AMD** Radeon RX 6800 及更新型号、Radeon AI PRO R9700(12 GB 及以上) | [AMD 显卡](docs/AMD_HIP.md) |
| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md)、[Intel 引擎](docs/INTEL.md) |
| **Apple 芯片 Mac**(M1 及更新,macOS 15 及以上) | [Mac](docs/MAC.md) |
| **AMD Ryzen AI Max(Strix Halo)** 及其他统一内存系统 | [Strix Halo](docs/STRIX_HALO.md) |
| 较旧的显卡和处理器 | [旧显卡](docs/OLDER_GPUS.md)、[旧处理器](docs/INSTALL.md#older-cpus-experimental) |
其他的都由安装程序搞定。两到三张显卡可以分担同一个模型([多 GPU](docs/MULTI_GPU.md))。
另外还需要:
实验性支持,由社区成员在自己的机器上编写和测试:
- 约 80 GB 的可用磁盘空间(用固态硬盘,首次启动会快很多)
- Windows 10 / 11、Linux 或 macOS
- 较新的显卡驱动
- **较老的显卡**(Tesla P40 / V100、GTX 10、Radeon VII / MI50、RX 6700 XT、RX 5500 XT):[较老的 GPU](docs/OLDER_GPUS.md)。
- **Intel Arc**,在 Linux 上从源码构建:[Intel Arc](docs/INTEL_ARC.md)。
- **AMD Ryzen AI Max(Strix Halo)**,在 Linux 上从源码构建:[Strix Halo](docs/STRIX_HALO.md)。
- **不支持 AVX2 的老处理器**:能用,但很慢。[较老的 CPU](docs/INSTALL.md#older-cpus-experimental)。
完整列表:[docs/INSTALL.md](docs/INSTALL.md#what-you-need)。
同一品牌的两三张显卡可以共同运行一个模型([多显卡](docs/MULTI_GPU.md))。Strata 不会把一个模型拆到 NVIDIA 和 AMD
显卡上。完整列表见 [docs/INSTALL.md](docs/INSTALL.md#what-you-need)。
## 安装
### 让你的 AI 帮你装
三种方式任选一种。它们启动的是同一个引擎,用的是同样的模型。
你在用 AI 编程助手吗(Claude Code、Cursor、Codex、GitHub Copilot 等)?把下面这段粘贴给它:
### 方式 A:Strata 应用
最简单的方式。应用内置引擎:安装、选模型、开始聊天。请根据你的系统和显卡,从
[发布页](https://github.com/Niko1221/Strata/releases/latest)下载对应的文件。不确定选哪个?看表格下面那一段。
| 系统 | 你的显卡 | 要下载的文件 |
| --- | --- | --- |
| **Windows 10 / 11** | NVIDIA RTX,驱动 580 或更新 | [`Strata-0.2.1-windows-cuda13.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda13.exe) |
| | NVIDIA,较旧的显卡,或驱动低于 580 | [`Strata-0.2.1-windows-cuda12.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-cuda12.exe) |
| | AMD Radeon RX 9000 系列或 Radeon AI PRO R9700 | [`Strata-0.2.1-windows-hip.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip.exe) |
| | AMD Radeon RX 7000 系列或 Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-windows-hip-rdna3.exe`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-windows-hip-rdna3.exe) |
| **Linux** | NVIDIA RTX,驱动 580 或更新 | [`Strata-0.2.1-linux-nvidia-cuda13.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda13.AppImage) |
| | NVIDIA,较旧的显卡,或驱动低于 580 | [`Strata-0.2.1-linux-nvidia-cuda12.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-nvidia-cuda12.AppImage) |
| | AMD Radeon RX 9000 系列或 Radeon AI PRO R9700 | [`Strata-0.2.1-linux-amd-rx9070.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx9070.AppImage) |
| | AMD Radeon RX 7000 系列 | [`Strata-0.2.1-linux-amd-rx7900.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-rx7900.AppImage) |
| | AMD Ryzen AI Max (Strix Halo) | [`Strata-0.2.1-linux-amd-strixhalo.AppImage`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-linux-amd-strixhalo.AppImage) |
| **Mac**(Apple 芯片,macOS 15 及以上) | 内置 | [`Strata-0.2.1-mac.dmg`](https://github.com/Niko1221/Strata/releases/latest/download/Strata-0.2.1-mac.dmg) |
**不确定时:** NVIDIA 显卡的驱动版本在 580 及以上就选 `cuda13`,否则选 `cuda12`。`cuda12` 适用于较旧的显卡和驱动。
`hip` 和 `amd` 的文件是给 AMD 显卡用的。其他 AMD 显卡(RX 6000)目前还没有现成的应用,
请看 [AMD 显卡](docs/AMD_HIP.md)。
然后:
- **Windows:** 运行 `.exe`。它只为当前用户安装,不需要管理员权限。如果 Windows 提示“Windows 已保护你的电脑”,
点击**更多信息**,再点**仍要运行**(安装程序还没有代码签名)。
- **Linux:** 给文件加上可执行权限(`chmod +x`),然后启动。需要 FUSE 2:`sudo apt install libfuse2`(Ubuntu 22.04),
或 `libfuse2t64`(Ubuntu 24.04 及更新)。
- **Mac:** 打开 `.dmg`,把 Strata 拖到“应用程序”。
第一个界面里,应用会检测你的电脑,并推荐合适的模型。点击安装即可。模型很大,要下载约 40-70 GB,中断后会从断点继续。
关闭窗口会同时停止 Strata 和引擎。
### 方式 B:终端(一行命令)
终端版就是 `strata` 命令,适合脚本、服务器和 AI 智能体。
Linux 和 Mac:
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
```
Windows(PowerShell):
```powershell
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
脚本会检查你的电脑,下载终端版,校验 SHA-256,然后解压到你的用户目录。不需要管理员权限。在 Windows 上,装完后请
打开一个新的终端窗口。然后启动 Strata:
```sh
strata launch
```
它会一次问一个问题:模型、上下文、图片、允许谁连接。接着为你的显卡下载引擎(已签名,使用前会先试启动),再下载
模型,并开始提供服务。在脚本和 AI 智能体里,请使用参数:
```sh
strata launch --model q2_0 --yes --no-tui
```
如果想在运行前先看看安装脚本,先用 `curl -fsSL <address> -o install.sh` 下载,读完后再用 `sh install.sh` 运行。
所有界面和参数见 [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md)。
### 方式 C:让你的 AI 来安装
把下面这段话粘贴到 Claude Code、Cursor、Codex、GitHub Copilot 或其他编程助手里:
```text
Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
```
它会检查你的显卡、内存和硬盘,选出合适的模型。然后安装并启动它,再告诉你怎么连接你的应用。AI 工具也可以通过
Strata 的 [MCP 服务器](docs/MCP_SERVER.md)来安装、启动和停止 Strata。
它会检查你的显卡、内存和磁盘,选出合适的模型,完成安装和启动,并告诉你怎么连接你的应用。AI 工具也可以通过 Strata 的
[MCP 服务器](docs/MCP_SERVER.md)安装、启动和停止 Strata。
### 或者自己动手
想自己动手编译?请看[从源码构建](#从源码构建)。
[下载 Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) 并解压(或者用 `git clone`)。
**Windows:** 双击 **`START-HERE.bat`**。**Linux:** 在 Strata 文件夹里运行 **`./setup.sh`**。
## 模型
NVIDIA 和 AMD 的步骤完全一样。安装程序会识别你的显卡,并装好对应的引擎。它会问你几个问题:
应用和 `strata launch` 会按你的内存推荐模型。它是同一个模型的几种尺寸,压缩程度不同。尺寸小的更快,尺寸大的稍微更聪明一点。
- 用哪个模型、哪个规格,
- 上下文多大(模型能记住多少文字),
- 是否要识别图片。
### 内存不够?试试 Gyro
每次直接按回车就是推荐选项。之后它会下载模型(约 70 GB)并启动。如果下载中断了,再运行一次即可:它会从中断的地方继续。
浏览器会打开 Strata 应用,地址是 `http://127.0.0.1:8080`。
[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF)(作者 agentionai)是运行这个模型占用最小的方式。
> **模型启动时,你的电脑可能会变慢或卡住 1-3 分钟**(第一次最久)。
> Strata 会把 35-55 GB 加载到内存,并为显卡锁定其中一部分。这是正常的。请耐心等待,不要关闭窗口。
> 窗口里会显示 Strata 正在做什么。
| 尺寸 | 所需内存(RAM + VRAM) | 下载大小 | 说明 |
| --- | ---: | ---: | --- |
| **Gyro-S** | 27.6 GB | 58.5 GB | 给其他尺寸都放不下的电脑。质量低于 IQ2_XS。 |
| **Gyro-M** | 35 GB | 92.0 GB | 质量约相当于 IQ3_XXS,显存占用少约 20%。在 12 GB 显卡和 R9700 上,输出回答比 Q2_0 慢 17-21%。 |
**下次使用时**,再运行一次 `START-HERE.bat`(或 `./setup.sh`)。它会马上启动,已下载的东西不会重复下载。关闭它的窗口就能停止模型。
`UPDATE.bat`(`./update.sh`)只更新 Strata,不启动。更新、Docker、多张显卡、文件存放位置以及所有选项:
[docs/INSTALL.md](docs/INSTALL.md)。
Gyro 可在 NVIDIA 和 AMD 显卡上运行(AMD 为实验性支持),不支持 Intel Arc 和 Mac。详情和测试数据:
[docs/GYRO.md](docs/GYRO.md)。用 `strata launch --model gyro-s` 启动。
## 该选哪个模型
### 其他尺寸
安装程序会根据你的内存推荐一个。同一个模型有几种规格,压缩程度不同。规格越小越快,越大越聪明一些。
| 你的内存 | 选择 | 原因 |
| 你的内存(RAM + VRAM) | 选择 | 备注 |
| --- | --- | --- |
| **32 GB** | **Coder** | 32 GB 装得下,而且专为代码打造(如果显卡是 24 GB,Q2_0 和 IQ2_XS 也能跑) |
| **48 GB** | **IQ2_XS**(或 Q2_0,最快) | 更大的规格装不下 |
| **64 GB** | **IQ2_XS**(推荐),或 IQ3_XXS / IQ3_S | 所有规格都装得下;IQ3_S 最好,也最慢 |
| **96 GB 或以上** | **IQ3_S**,或 Unsloth 的 UD-IQ4_XS(约 4-bit) | 开着其他程序也能放下最大的规格 |
| **32 GB** | **Gyro-S**,写代码可选 **[Coder](docs/MODELS.md#coder)** | Coder 需要 32 GB 内存,只保留一半的专家,所以代码以外的能力较弱 |
| **40 GB** | **[Swift 1.5](docs/MODELS.md#swift-15)** IQ2_XS(39.2 GB),或 **Gyro-M**(35 GB) | Swift 回答前思考的时间更短。Gyro-M 的质量接近 IQ3_XXS,但输出回答比 Q2_0 慢 17-21% |
| **48 GB** | **IQ3_XXS**(47.0 GB) | 更慢,更好 |
| **64 GB** | **IQ3_S**(54.8 GB),原版或 Swift 1.5 | 最慢,最好:在公开测试中与完整模型持平 |
| **96 GB 及以上** | Unsloth 的 **UD-IQ4_XS**(约 72 GB 内存)到 **Q8_0**(约 136 GB),4 到 8 位 | 更接近完整模型。内存不足时,Strata 会从固态硬盘读取一部分,速度会慢很多 |
- **[Coder](docs/MODELS.md#coder):** 编程版本,去掉了一半专家。它达到完整模型 SWE-bench Verified 分数的 91%(由其作者测得),
32 GB 内存就能装下。代码以外的能力较弱,包括中文和其他中日韩文本(#438)。这类用途请选 Q2_0、IQ2_XS 或 IQ3_S,
它们保留了所有专家。
- **[Swift 1.5](docs/MODELS.md#swift-15):** 一个微调版本,回答前思考的时间短得多。你能更早拿到答案,质量基本不变。
- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloth-ud-iq4_xs):** Unsloth 的约 4-bit 版本,质量介于 IQ3_S 和 UD-Q4_K_XL 之间。
下载 94 GB。内存少于约 80 GB 时,Strata 回答时要从 SSD 读取其中一部分,所以会更慢(NVMe SSD 有帮助)。
- **[Unsloth UD-Q4_K_XL](docs/MODELS.md#unsloth-ud-q4_k_xl-experimental)**(实验性):最接近完整模型。但 Strata
回答时要从 SSD 读取其中大部分内容,所以在 64 GB 的电脑上每秒只能写 7-8.5 个 token。
- **[OrcaRouter 的 Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** 需要手动设置,不在安装程序的菜单里。
内存比这一行写的少,也可能能用:Strata 会自动切换到[低内存模式](docs/MODELS.md#a-big-graphics-card-and-little-ram)。
速度会变慢,显卡较小时会慢得多。
规格、下载以及各配置能装下什么:[docs/MODELS.md](docs/MODELS.md)。以后想再添加模型,运行
`SETUP.bat`(Linux:`./setup.sh --setup`)。
更多选择:
- **[Swift 1.5](docs/MODELS.md#swift-15)** 是微调版本,回答前思考的时间更短,质量和内存占用大致相同。
- **[NVFP4](docs/NVFP4.md)** 是 NVIDIA 为 NVIDIA 显卡提供的 4 位版本。要把所有专家放进内存,需要约 80 GB 内存。
- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** 是另一个更小的模型,能看图片,还能同时回答多个请求。
- **[OrcaRouter 的 Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** 需要手动设置。
所有尺寸、下载大小,以及各种配置能跑什么:[docs/MODELS.md](docs/MODELS.md)。
## 速度如何?
以下数据基于 12 GB 显卡(RTX 5070、Ryzen 5 7600、64 GB 内存)。其他显卡的结果见:
[各模型的速度](docs/MODELS.md#how-fast-is-each-size)、[社区测试结果](docs/COMMUNITY_BENCHMARKS.md)。
| 模型 | 回答输出速度 | 读取提示词速度 |
| --- | --- | --- |
| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2,650 tokens/s |
| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2,090 tokens/s |
| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1,750 tokens/s |
| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1,620 tokens/s |
| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2,180 tokens/s |
一个 token 约等于 ¾ 个英文单词,所以每秒 60 个 token 比你阅读的速度还快。“读取提示词”指 Strata 吸收长文档、代码或聊天历史的速度。
在 16 GB 的 AMD 显卡上测得的结果(RX 9070 XT、Ryzen 9 3900X、47 GB 内存):
| 模型 | 回答输出速度 | 读取提示词速度 |
| --- | --- | --- |
| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1,160 tokens/s |
| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1,110 tokens/s |
| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1,420 tokens/s |
两张表的进度条使用同一个比例。显存更大的显卡更快:RTX 3090(24 GB)每秒大约能输出 100-140 个 token。每个数字对应的引擎版本见
[DETAILS.md](docs/DETAILS.md#speed-measured)。
## 使用方法
<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="Strata 应用的 Monitor 标签页,旁边是一个编程智能体"><br>
<sub>Strata 应用的 <b>Monitor</b>(左),编程智能体正在写视频里的宝塔花园(右)</sub></p>
### 在应用里
- **在浏览器里:** 打开 `http://127.0.0.1:8080`。里面有 **Chat**(聊天)、实时 **Monitor**(监控模型和你的
GPU/CPU/内存),以及 **About**(设置和地址)。
- **你的应用和编程智能体:** 添加一个“OpenAI 兼容”的提供商,base URL 填 **`http://127.0.0.1:8080/v1`**。
API key 和模型名随便填都行。
- 使用 Anthropic API 的应用:`http://127.0.0.1:8080/v1/messages`(Claude Code:
`ANTHROPIC_BASE_URL=http://127.0.0.1:8080`)。
- Codex CLI 和其他使用 OpenAI Responses API 的应用:`/v1/responses`
([设置方法](docs/DETAILS.md#the-responses-api-and-codex-cli))。
- **思考:** 在聊天菜单或应用的“reasoning effort”里选择 **off、low、medium 或 high**。off 最快,
high 最适合难题。
- **图片:** 在安装时对“Images?”选是。然后在聊天里点 **Picture**,或在你的应用里附上图片。
AMD 显卡在 Linux 上通过处理器识别图片;在 Windows 上暂时还不行。
- **从手机或另一台电脑访问:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`。一定要设置 key。
- **一次处理一个请求:** 默认情况下 Strata 一次只回答一个请求,其他请求排队等待。想同时回答多个,
设置 `"parallel": 2`([BATCHING.md](docs/BATCHING.md))。在 12 GB 显卡上,这会让每个回答变慢。
- **长提示:** Strata 会完整读入对话的第一条消息,大约每 30,000 个 token 需要 1 分钟。之后的消息几秒内就开始回答。
应用包含**聊天**、显示模型以及 GPU、CPU 和内存状态的**监控**、**模型**页面和**设置**。聊天可以在你电脑上的某个文件夹里工作。
在**手动**模式下,修改文件或运行命令之前它会先询问你。输入 `/loop 10m <要做的事>`,在聊天保持打开期间每 10 分钟重复发送一次消息。
更多:[聊天记录存在哪里](docs/INSTALL.md#where-things-are-stored)、[API](docs/DETAILS.md#using-it)。
聊天可以用 DuckDuckGo(或 Bing)搜索网页,不需要账号。你也可以在**设置,工具**里添加搜索密钥。搜索是唯一会把内容发出电脑的功能,
发出的就是搜索词。
要同时运行两个模型,请打开**运行多个模型**,并为每个模型分配同一品牌的单独一张显卡。
<p align="center"><img src="docs/media/app-chat.png" width="900" alt="The Strata app: a chat where the agent built a small voxel pagoda garden, with the page open in the preview on the right"><br>
<sub>Strata 应用中的一次对话:智能体用一个 HTML 文件写出了一个小型体素宝塔花园,页面在对话旁的预览中打开</sub></p>
### 在终端里
<p align="center"><img src="docs/media/terminal-monitor.png" width="900" alt="The Strata Terminal monitor overview while the model writes a reply"><br>
<sub>模型回复时的 <code>strata monitor</code>:速度、GPU、内存和最近的请求</sub></p>
```sh
strata launch # 引导界面:模型、上下文、图片、访问方式,然后开始服务
strata launch --model q2_0 --yes --no-tui # 不提问:开始服务,打印地址,然后返回
strata launch --plan --model q2_0 # 显示将要执行的操作;不做任何更改
strata launch --status # 当前正在提供什么服务
strata launch --stop # 停止服务
strata monitor --url http://127.0.0.1:8080 # 查看速度、GPU、内存和请求
strata update --check # 有新引擎吗?(strata update 会安装它)
strata doctor # 检查已安装的引擎文件
```
`strata --help` 会列出所有命令。启动和提供服务时不会下载任何东西。只有安装、`strata launch`(在你选择模型时)和
`strata update` 会下载。要从手机或另一台电脑访问 Strata,请使用 `strata launch --access lan`。它会生成一个 API 密钥,
没有密钥就不会在网络上启动。除非你加上 `--open`,否则不会打开浏览器。指南:[docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md)。
完整的客户端说明:[docs/TERMINAL.md](docs/TERMINAL.md)。
### 配合 AI 智能体
智能体用方式 B 中的一行命令安装 Strata,然后运行 `strata launch --model NAME --yes --no-tui`。步骤、参数和退出码见
[docs/AI_SETUP.md](docs/AI_SETUP.md),仓库规则见 [AGENTS.md](AGENTS.md)。
### 连接你的其他应用
| 应用类型 | 基础 URL |
| --- | --- |
| 兼容 OpenAI(大多数应用和智能体) | `http://127.0.0.1:8080/v1`(API 密钥和模型名称随便填) |
| Anthropic API(Claude Code 等) | `http://127.0.0.1:8080`,不带 `/v1`。这些应用会自己加上 `/v1/messages`。Claude Code:`ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |
| OpenAI Responses API(Codex CLI) | `http://127.0.0.1:8080/v1/responses`([设置方法](docs/DETAILS.md#the-responses-api-and-codex-cli)) |
`strata connect` 会打印 OpenCode、Codex CLI、Claude Code、oh-my-pi、pi、Hermes Agent 和 OpenClaw 的具体设置。
`strata connect opencode --write` 会在询问后写入配置文件,并保留备份。
### 图片
能看图片的模型默认开启图片功能。在聊天里附加或粘贴图片,或者从你的应用发送图片。显卡的显存小于 24 GB 时,读图由处理器完成
(每张图片几秒钟,不占用显卡的显存);显存达到 24 GB 时,由显卡完成。NVIDIA 和 AMD 的规则相同。你可以按模型关闭图片功能,
也可以用 `--vision off`、`--vision gpu` 或 `--vision cpu` 选择读图方式。
这条规则背后的数据:在 Radeon AI PRO R9700 上,显卡读一张 260 token 的图片用了 57 ms,处理器用了 3.1 秒。在 16 GB 的显卡
(Tesla P100 和 RTX A4000,Linux)上,用显卡读图会让文字速度下降 3-5%,因为它占用约 1.4 GB 显存,否则这部分会被模型使用。
在 Mac 上,无论内存多大,都由 GPU 读图(32 GB 的 M4 上读一张 260 token 的图片用 0.45 秒,占用约 1.2 GB 共享内存);处理器读一张约需 50 秒,所以只有指定 `--vision cpu` 时才用。
### 使用须知
- 在聊天菜单或你的应用的“reasoning effort”里选择思考级别(**off、low、medium 或 high**)。off 最快。
- Strata 默认一次回答一个请求。要同时处理多个,请在 `strata launch` 中使用 `--parallel N`,或在配置里写
`"parallel": 2`([BATCHING.md](docs/BATCHING.md))。在 12 GB 显卡上,这会让每个回答变慢。
- Strata 会完整读取聊天的第一条消息,每 30,000 个 token 约需 1 分钟。之后的消息几秒钟就开始。
- 聊天记录存放的位置和完整的 API:[docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored)、[docs/DETAILS.md](docs/DETAILS.md#using-it)。
## 从源码构建
<details>
<summary>从仓库构建 Windows、Linux 和 Mac 版本</summary>
[下载 Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip)并解压,或者用 `git clone` 获取。
**Windows:** 双击 `START-HERE.bat`。**Linux:** 在 Strata 文件夹里运行 `./setup.sh`。
**Mac(Apple 芯片,macOS 15 及以上):** 先用 `xcode-select --install` 安装一次 Apple 的命令行工具(不需要完整的 Xcode)。
在 Strata 文件夹里打开终端,运行 `./setup.sh`。它会在你的 Mac 上构建 Mac 引擎(约 7 分钟),问同样的问题,下载模型并启动。
下次运行 `./setup.sh` 会直接启动。你的 Mac 适合哪个尺寸:[docs/MAC.md](docs/MAC.md)。
安装程序会找到你的显卡,并设置合适的引擎。它会问你用哪个模型和尺寸、上下文多长(模型能记住多少文字),以及是否要读取图片。
直接按回车就是推荐的选择。然后它会下载模型(约 70 GB)并启动。如果下载中断,再运行一次就会继续。浏览器会打开
`http://127.0.0.1:8080` 上的 Strata 页面。
> 模型启动期间,你的电脑可能会变慢,或者在 1-3 分钟内没有响应(第一次最久)。Strata 会把 35-55 GB 加载到内存里,并锁定其中一部分
> 给显卡使用。请等待,不要关闭窗口。
下次再运行 `START-HERE.bat`(或 `./setup.sh`)会立即启动,不会重复下载。关闭它的窗口就能停止模型。`UPDATE.bat`
(`./update.sh`)只更新 Strata,不启动。更新、Docker、多显卡、文件存放位置和所有选项:[docs/INSTALL.md](docs/INSTALL.md)。
要测试你的电脑,运行 `BENCH.bat`(`./bench.sh`)。需要 5-10 分钟,并生成一份可以分享的报告
([方法](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark))。
**可选:整个模型都放得进内存时,让单卡更快。** 这是 RubenTorney 提供的两个需要手动开启的设置(#1904、#1905)。
当你只用一张显卡,并且内存装得下所有专家时,它们会有帮助,例如在 64 GB 的电脑上跑 Q2_0 或 IQ3_XXS。打开 Strata 数据文件夹里的
`strata-<model>.json`([位置](docs/INSTALL.md#where-things-are-stored)),然后添加:
- 在 `"args"` 列表中加入 `"--adapt-async", "1"`。这样专家缓存会在辅助线程上更新,不会让回答暂停。在 RTX 5070
(Windows,Q2_0)上测得:解码快 4-5%,代码和长文档快 7-10%。回答可能与默认设置略有不同,但每次运行的结果保持一致。
- 在 `"env"` 块中加入 `"STRATA_PIN_BY_NEED": "1"`(Windows)。Windows 只能为显卡锁定大约一半的内存,
这个设置会优先锁定显卡最先需要的专家。在 RTX 5070 上测得:读取提示词快 4.4%。
```json
"args": [ ..., "--adapt-async", "1" ],
"env": { "STRATA_PIN_BY_NEED": "1" }
```
修改后请重启 Strata。要撤销,删除这两项即可。
</details>
## 工作原理
这类模型通常运行在有几百 GB 显存的服务器上。你的显卡只有 12-24 GB。Strata 把工作分给整台电脑,让模型装得下。
可以把它想成一座图书馆。
- 模型是一个由 24,576 位小专家(“experts”)组成的团队,好比 24,576 本参考书。每个词只需要其中 10 本。
- **你的显卡是书桌。** 它小而快。最常用的大约 4,000 本书摊开放在上面。
- **你的内存是阅览室的书架。** 24,576 本书都在这里。某个词需要一本不在书桌上的书时,处理器会立刻从书架上取来读,显卡则继续工作。
- **你的固态硬盘是档案室。** 里面存着一张 29 GB 的查找表。模型每个词只读其中几行,所以不用常驻内存。
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
另外有一个小助手会猜接下来的几个词。大模型一次检查所有猜测,保留正确的部分。得到的答案不变,但快了 1.6-1.8 倍。
长文本按大块读取,一次最多 8,192 个 token,速度超过每秒 1,000 个 token。
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
更详细的解释:[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md)。每个部分和它的数据:
[详细说明](docs/DETAILS.md#how-it-works)和[论文](docs/paper/Strata-Paper.pdf)。
## 出问题了?
- **Strata 第一次启动时电脑卡死了。** 加载模型时这是正常的。请耐心等待,不要关闭窗口。
10 分钟后还卡着?重启电脑,关掉其他程序再试一次,或者换一个更小的规格。
- **下载或安装时中断了。** 再运行一次 `START-HERE.bat`(或 `./setup.sh`)。它会从中断的地方继续。
- **非常慢,硬盘灯一直闪,或者提示“the engine stopped unexpectedly”。** 你的电脑可用内存不够。
关掉其他程序(浏览器很占内存),或者换一个更小的规格(Q2_0 或 IQ2_XS)。
- **第一次启动 Strata 时我的电脑卡住了。** 它在把模型加载到内存。请等待,不要关闭窗口。10 分钟后还是卡住?重启电脑,
关闭其他程序后再试,或者选一个更小的尺寸。
- **下载或安装时中断了。** 重新启动应用或 `strata launch`,或者再运行一次 `START-HERE.bat`(或 `./setup.sh`)。
它会从中断的地方继续。
- **非常慢、硬盘灯一直闪,或者提示“引擎意外停止”。** 你的电脑可用内存不够。关闭其他程序(浏览器很占内存),
或者选一个更小的尺寸,比如 Q2_0 或 IQ2_XS。
- **提示这台电脑没有可用的引擎。** 在应用里打开**设置,引擎**,点击**更新引擎**。在终端里运行 `strata update`。
`strata doctor` 会显示已安装哪些引擎文件。
- **提示 8080 端口已被占用。** Strata 已经在运行了。找一下它的窗口。
更多问题和解决办法:[docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)。还是解决不了?提交一个
[issue](https://github.com/Niko1221/Strata/issues),并附上 Strata 文件夹里的 `strata-<model>.log`。发现了安全问题?
请私下报告:[SECURITY.md](SECURITY.md)。
## 它是怎么工作的?
这类模型通常运行在拥有数百 GB 显存的服务器上。而你的显卡只有 12-24 GB。Strata 让模型装得下的办法是
**把工作分摊到整台电脑上**。可以想象一个厨房:常用的东西放在台面上,其余的放在储藏室里。
<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="模型的 24,576 个专家:最常用的在显卡上,全部在内存里,一张查找表在 SSD 上"></p>
- **这个模型是由 24,576 个小专家(“experts”)组成的团队。** 每个词只需要其中 10 个。
- **你的显卡** 存放最常用的几千个专家。**你的内存** 存放全部专家,
**你的处理器** 同时处理其余的部分。**你的 SSD** 存放一张大查找表。
<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="一个小助手猜测接下来的几个词;大模型一次性全部检查,保留正确的"></p>
- **先猜,再检查:** 一个小助手先猜接下来的几个词,大模型再一次性检查它们。答案完全一样,但快 1.6-1.8 倍。
- **长文本分大块读入**(每次最多 8,192 个 token),速度超过每秒 1,000 个 token。
更详细的解释:[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md)。每个部分及其数据:
[详细说明](docs/DETAILS.md#how-it-works)和[论文](docs/paper/Strata-Paper.pdf)。
更多问题和解决办法:[docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)。还是解决不了?请提交
[issue](https://github.com/Niko1221/Strata/issues)。在应用里,附上**设置,引擎,诊断,查看日志,保存日志**得到的文件。
在终端里,粘贴 `strata doctor` 的输出。从源码运行时,附上 Strata 文件夹里的 `strata-<model>.log`。
如果是安全问题,请私下报告:[SECURITY.md](SECURITY.md)。
## 致谢与许可证
模型是 Qwen 团队的 [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)。它由
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF)、UkisAI(Swift 1.5)
和 Unsloth 压缩。Strata 使用了 [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp) 的部分代码。完整致谢:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits)。Strata 以 [MIT 许可证](LICENSE)开源。少数部分和每个模型
有各自的许可证([具体是哪些](docs/HOW_IT_WORKS.md#license))。
模型是 Qwen 团队的 [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)。压缩工作由
[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF)、UkisAI(Swift 1.5)、agentionai(Gyro)
和 Unsloth 完成。Strata 使用了 [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp) 的部分代码。全部致谢:
[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits)。Strata 以 [MIT 许可证](LICENSE)开源。少数部分和所有模型有各自的许可证
([具体是哪些](docs/HOW_IT_WORKS.md#license))。
## 参与贡献
发现了 bug,或者想提交修改或基准测试结果?请先阅读 [CONTRIBUTING.md](CONTRIBUTING.md)。其中说明了需要附上哪些内容,
方便其他人核实你的工作。
## 支持 Strata
Strata 免费且开源。如果它对你有用,欢迎支持它的开发:
Strata 免费,采用 MIT 许可证。我在能拿到的各种硬件上开发和测试它。如果它在你的电脑上跑得不错,请我喝杯咖啡,能帮我继续测试新显卡、
修复你们反馈的问题。不用有压力。使用它并反馈你的发现,同样很有帮助。
<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="请我喝杯咖啡" height="50"></a></p>
<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
有兴趣赞助 Strata?请通过 [GitHub](https://github.com/Niko1221) 联系我。
+52 -1
View File
@@ -39,6 +39,57 @@ this PC only. The details and every setting are in [docs/DETAILS.md](docs/DETAIL
- **MCP tools** are opt-in: only the servers you put in the run config, only for requests from Strata's own page
that ask for them. They run with your user's rights, and the model decides when to call them.
- **The request monitor** (`/api-monitor`, which keeps the last prompts and answers in memory) is off unless
`"api_monitor": true` is set.
`"api_monitor": true` is set. Apart from it, the server keeps in its process memory the text and tokens of its
last few prompts, whatever `"api_monitor"` says, to encode the next turn faster; no endpoint shows them, and they
are gone when the server stops.
## The desktop app and the Terminal
Strata comes in three parts that share one engine: the **desktop app**, the **Terminal** (`strata`) and the **engine** that runs
the model. The section above is about the engine's HTTP server. This is what the other two add.
- **The app's host listens on this PC only.** The app serves its page from a local host process on `127.0.0.1` (port 8791 unless
changed). A request to its `/api/` endpoints that carries a browser `Origin` from any other site gets 403. The app shell holds a
random secret (32 characters or more, compared in constant time) that is needed to quit the host or tell it which window is open,
and the host gives its own helper processes a second random secret. By default the models the app starts are served on this PC only.
- **Sharing is a separate, key-protected listener.** "Share with other devices" (and the Cloudflare tunnel) will not start until an
API key is set. It opens a different listener that answers only `/v1/models`, `/v1/chat/completions` and `/v1/messages`, never the
app's page or its host endpoints.
- **App and Terminal talk through an owner channel.** One owner process per installation starts and ends the engine, and a second
client (the Terminal) attaches to it. On macOS and Linux that is a private Unix socket in a folder only you can open. On Windows it
is a TCP listener on `127.0.0.1` that answers only requests carrying a 256-bit token kept in an owner-only folder. That is a weaker
guarantee than a socket other users cannot open: on Windows the token is what protects it, as with any loopback service that has a
token (`strata_control/owner_transport.py`). The bridge between the app shell and the host (used for the app update) is also loopback and protected by a token.
- **Starting a model and serving never download anything.** Downloads and updates are one explicit step: `strata update`, or "Update
engine" and "Update Strata" in the app's Settings. Models and saved tuning are never replaced by an update.
- **Updates come from a signed feed.** The release publishes `strata-feed.json` and `strata-feed.json.sig` (Ed25519). An installed
Strata checks the signature against the public key it carries (`strata_control/update_keys.json`), refuses a feed older than the last
one it accepted (a replayed old feed) or expired, and checks every engine package against its SHA-256, its signed manifest and the
card in your PC before it test-starts it and switches to it. Any failure keeps the engine you had. The app installer itself is signed
with a separate key (Tauri's minisign key) that the app and the Terminal both check. The private keys are kept offline by the owner.
See [packaging/UPDATES.md](packaging/UPDATES.md).
### How far to trust `curl | sh` and `irm | iex`
The Terminal installs with one line:
```sh
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
Piping a script into a shell runs whatever the server sends, so it deserves the same care as running any downloaded program.
What the script does is short and written at its top: it fetches the Terminal archive named in the release's `install-manifest.txt`,
checks its SHA-256, unpacks it into your home folder (no administrator rights, nothing outside your home), checks the release's signed feed
with the Terminal it just unpacked and removes it again if that fails, and puts `strata` on your PATH.
Those checks have a limit. The checksum comes from the same release as the archive, and the public key that checks the feed arrives inside
that archive, so for the first install all trust rests on HTTPS and on the `Niko1221/Strata` GitHub release itself. The checks catch a
damaged or half-finished download and a feed that was not signed by the release key; they do not protect you if the release has been replaced
by someone who controls it. Later updates are checked against the key already on your PC, which is a stronger position.
If that is not enough: save the script (`curl -fsSLO https://github.com/Niko1221/Strata/releases/latest/download/install.sh`), read it, and
run it from the file; or install the desktop app or build from source instead and check the code you build.
Strata has not had an outside security audit yet.
What Strata downloads, when it uses the internet, what stays on your PC and how downloads are checked is written up in [docs/SAFETY.md](docs/SAFETY.md).
+13
View File
@@ -2,8 +2,13 @@
rem Strata for Windows: the first run installs everything and starts the model; later runs just start it.
rem Needs only an NVIDIA or AMD graphics driver. Python is installed for your user account if it is missing (no admin needed).
setlocal
rem Python reads and writes text as UTF-8, not in the Windows code page (cp1251, cp866, ...): a folder or user name with Cyrillic or other letters stays intact (#3202).
set "PYTHONUTF8=1"
title Strata
cd /d "%~dp0"
rem A bundled Python runtime (the python\ folder of the terminal archive) is used when it is there: nothing to find,
rem install or download. SETUP.bat and UPDATE.bat run this file, so they use it too.
if exist "python\python.exe" goto bundled
if exist ".venv\Scripts\python.exe" goto run
call :findpy
@@ -36,6 +41,14 @@ echo Could not create the Python environment in .venv
pause
exit /b 1
:bundled
set "PYTHONNOUSERSITE=1"
set "PYTHONHOME="
set "PYTHONPATH="
"python\python.exe" setup.py %*
if errorlevel 1 pause
exit /b
:run
".venv\Scripts\python.exe" setup.py %*
if errorlevel 1 pause
+53 -7
View File
@@ -2,7 +2,8 @@
rem Update Strata without starting the model (#475): the newest code (git pull, when this folder is a git clone), then
rem what START-HERE.bat does before a start - the engine (a new one when this version needs it), the Python packages,
rem each installed model's settings and draft subset. The model files are not touched. Start the model later with
rem START-HERE.bat. Options are passed on to setup.py.
rem START-HERE.bat. Options are passed on to setup.py. The bundled Python (python\ beside this file) is used when
rem there is one: START-HERE.bat, which this file runs, picks it.
setlocal
title Strata - update
cd /d "%~dp0"
@@ -16,12 +17,57 @@ rem All of it in one block: cmd reads a .bat file while it runs it, and the git
exit /b 1
)
echo Getting the newest Strata ^(git pull^) ...
git pull --ff-only || (
echo.
echo git pull did not succeed ^(the reason is above^): nothing was updated. Files you changed here can stop it:
echo "git status" lists them.
pause
exit /b 1
git pull --ff-only
if errorlevel 1 (
rem The history was rewritten on 2026-10-06, see #1276: a clone from before has no commit in common with origin/main.
set "REWR="
git merge-base HEAD origin/main >nul 2>nul
if errorlevel 1 set "REWR=1"
git rev-parse --is-shallow-repository 2>nul | findstr /x true >nul && set "REWR="
if defined REWR (
echo.
echo This clone is on the repository's old history ^(cleaned up on 2026-10-06^): it has no commit in common
echo with origin/main, so it cannot be updated by git pull.
set "DIRTY="
for /f "delims=" %%i in ('git status --porcelain --untracked-files^=no') do set "DIRTY=1"
if defined DIRTY (
echo You have local changes to tracked files here, so nothing was touched. To move to the new history
echo yourself ^(your untracked files, models and settings stay where they are^):
echo git branch pre-cleanup-backup
echo git stash push
echo git checkout -B main origin/main
echo then run UPDATE.bat again.
pause
exit /b 1
)
set "BK=pre-cleanup-backup"
git rev-parse --verify -q refs/heads/pre-cleanup-backup >nul 2>nul && set "BK=pre-cleanup-backup-%RANDOM%"
set "BR=main"
for /f "delims=" %%i in ('git symbolic-ref --short -q HEAD 2^>nul') do set "BR=%%i"
call git branch %%BK%% HEAD
if errorlevel 1 (
echo Could not make the backup branch: nothing was updated.
pause
exit /b 1
)
call git checkout -q -B %%BR%% origin/main
if errorlevel 1 (
call git branch -D %%BK%% >nul 2>nul
echo.
echo Could not move to the new history ^(the reason is above^): nothing was updated. By hand:
echo git checkout -B main origin/main
pause
exit /b 1
)
call echo Moved to the new history. Your old commits are kept in the branch %%BK%%.
echo Untracked files ^(models, settings, the engine^) were not touched.
) else (
echo.
echo git pull did not succeed ^(the reason is above^): nothing was updated. Files you changed here can stop it:
echo "git status" lists them.
pause
exit /b 1
)
)
) else (
echo This copy of Strata was not made with git, so it cannot fetch new files itself. Download the newest one:
+232
View File
@@ -0,0 +1,232 @@
{
"name": "strata-desktop",
"version": "0.1.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "strata-desktop",
"version": "0.1.0",
"devDependencies": {
"@tauri-apps/cli": "2.12.3"
}
},
"node_modules/@tauri-apps/cli": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli/-/cli-2.12.3.tgz",
"integrity": "sha512-f7/LsY74SVb1ChmOPw1ajOpxii0GSzDAgmWHRci8JUYuuai04rXsSbFobpmB0sP2DdlvMD/B5ZotEn8GOGMRUg==",
"dev": true,
"license": "Apache-2.0 OR MIT",
"bin": {
"tauri": "tauri.js"
},
"engines": {
"node": ">= 10"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/tauri"
},
"optionalDependencies": {
"@tauri-apps/cli-darwin-arm64": "2.12.3",
"@tauri-apps/cli-darwin-x64": "2.12.3",
"@tauri-apps/cli-linux-arm-gnueabihf": "2.12.3",
"@tauri-apps/cli-linux-arm64-gnu": "2.12.3",
"@tauri-apps/cli-linux-arm64-musl": "2.12.3",
"@tauri-apps/cli-linux-riscv64-gnu": "2.12.3",
"@tauri-apps/cli-linux-x64-gnu": "2.12.3",
"@tauri-apps/cli-linux-x64-musl": "2.12.3",
"@tauri-apps/cli-win32-arm64-msvc": "2.12.3",
"@tauri-apps/cli-win32-ia32-msvc": "2.12.3",
"@tauri-apps/cli-win32-x64-msvc": "2.12.3"
}
},
"node_modules/@tauri-apps/cli-darwin-arm64": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-darwin-arm64/-/cli-darwin-arm64-2.12.3.tgz",
"integrity": "sha512-xmVNoJOOKzKLj7GLHXmXLimRby9M42aBIJa0sB80N7o24eKV5JiVmToKTRLVH9yhL05GZ/B6rSSOmbzNFJwBAA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-darwin-x64": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-darwin-x64/-/cli-darwin-x64-2.12.3.tgz",
"integrity": "sha512-URd+0usfmUQQyb5kNPiKt2OT8gp7oGU7QR+qOppB6e35mpVBFFVNhWc9oWb3p0u71fDJ09DpF4T6m9xYxoIB8g==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-linux-arm-gnueabihf": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm-gnueabihf/-/cli-linux-arm-gnueabihf-2.12.3.tgz",
"integrity": "sha512-0kwUVx2LbmOoTRlwF1oDww7th4/NSluv4ZBQ6KRZsFKg4g0TlTcIiJ0VehqkQYUeC9xIT1Mk6XtF52mI155VSQ==",
"cpu": [
"arm"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-linux-arm64-gnu": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm64-gnu/-/cli-linux-arm64-gnu-2.12.3.tgz",
"integrity": "sha512-gKSGDx6fx+HBLQmG6guChFdisbrCSdx9Yli+j5teOa64gm1CeM+C6NLru5Pj3ZkkEep05BmCFjxvX+cjqAycZQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-linux-arm64-musl": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm64-musl/-/cli-linux-arm64-musl-2.12.3.tgz",
"integrity": "sha512-Yr59qMy/gcvAfBuFdCGlAmGiLsYWMFDbsP4bAI6iNBzcJTGrYWZGeiQhGDo29MQwfF/NUBqyM2qljXu5tZ6wjQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-linux-riscv64-gnu": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-riscv64-gnu/-/cli-linux-riscv64-gnu-2.12.3.tgz",
"integrity": "sha512-jKr0/REBMnLKC0OwPQZkyge0mKDaNZ0b/AbfGwVyCgFfwS730EVLfVDo3nt7MUu6jJaytRq08+xSj5qfy+U1+Q==",
"cpu": [
"riscv64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-linux-x64-gnu": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-x64-gnu/-/cli-linux-x64-gnu-2.12.3.tgz",
"integrity": "sha512-gTkEqJAGCMROGanxHnzaJzQ+/rXl39HWVg0/hDODS1wDWBGqoceqGC6Qm1+mQbyT11xnsB8mKu/gyVID/YpFIw==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-linux-x64-musl": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-x64-musl/-/cli-linux-x64-musl-2.12.3.tgz",
"integrity": "sha512-T2dTxL1svRym71fZtaW4pZYXbthbfFsEJr1NeOUg/p7laWXZbUt/8DMSZf8recu9f7+1F/Lv00HtzsEu6vDO9w==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-win32-arm64-msvc": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-arm64-msvc/-/cli-win32-arm64-msvc-2.12.3.tgz",
"integrity": "sha512-eoiQwpEZWbpsVHcM9BAzqqe4TCHszvr3rsM7pUQ4ofLK0lF1i1d/jX0Qo5DYMqcrUtwQuDdDuOCbcT0Izf1VwQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-win32-ia32-msvc": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-ia32-msvc/-/cli-win32-ia32-msvc-2.12.3.tgz",
"integrity": "sha512-kE+KUg6Dbbkgvcw0m99wJVIToFQzJqpMBAXah74WqxaLzssNwSTUccrHp6+QVzMhjQP330tvekFmnQF+0NbE6Q==",
"cpu": [
"ia32"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 10"
}
},
"node_modules/@tauri-apps/cli-win32-x64-msvc": {
"version": "2.12.3",
"resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-x64-msvc/-/cli-win32-x64-msvc-2.12.3.tgz",
"integrity": "sha512-L7wndWAgX2mLW7alswpL3d1R0P9FN2WYBgHaQkAYEBxHHGkY4vFX9CjcRQ1LAxDAMIJxrY7/aDziFJeChS84NQ==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0 OR MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 10"
}
}
}
}
+20
View File
@@ -0,0 +1,20 @@
{
"name": "strata-desktop",
"productName": "Strata",
"version": "0.2.1",
"private": true,
"description": "Strata Desktop - local AI, tuned to one model.",
"scripts": {
"build:ui": "npm --prefix ../host run build",
"typecheck:ui": "npm --prefix ../host run typecheck",
"test:ui": "npm --prefix ../host test",
"tauri": "tauri",
"dev": "npm run build:ui && tauri dev",
"build": "npm run build:ui && tauri build",
"build:app": "npm run build:ui && tauri build --bundles app",
"dmg": "npm run build:ui && tauri build --bundles dmg"
},
"devDependencies": {
"@tauri-apps/cli": "2.12.3"
}
}
+361
View File
@@ -0,0 +1,361 @@
#!/usr/bin/env python3
"""The installed Windows app, closed every way, with a stub engine: nothing may be left running.
python apps/desktop/scripts/app-close-matrix.py --install <folder the installer wrote> [--scenarios all] [--report out.json]
For each case it starts the REAL installed `strata-desktop.exe` (its window opens), loads a stub engine through the host's API
(profile + `runtime start` under Control's owner runtime, exactly the app's path), records every process of the tree (the app,
the host, Control, the owner host, the server, the engine), ends the app one way, and asserts:
* no process of the tree is left, the engine's port is free, the runtime lease can be taken again;
* the engine got the clean stop where the way of closing allows one (normal close, window X);
* the next load of the same slot works (a lost owner's state is cleaned, not a blocker).
It also proves a second launch focuses the open window and starts nothing, and that one app runs two engines and closes both.
Everything per-user lives under a scratch folder; the stub engine is a script. No real engine ever runs.
"""
from __future__ import annotations
import argparse
import ctypes
import json
import os
import subprocess
import sys
import tempfile
import time
import urllib.error
import urllib.request
from ctypes import wintypes
from pathlib import Path
ROOT = Path(__file__).resolve().parents[3]
sys.path.insert(0, str(ROOT))
import psutil # noqa: E402
from strata_control import procs, runtime # noqa: E402
from tests.terminal.lifecycle_matrix import install_fake_engine, port_open # noqa: E402
if os.name != "nt":
raise SystemExit("this matrix drives the Windows installer's app")
user32 = ctypes.WinDLL("user32", use_last_error=True)
WM_CLOSE = 0x0010
def main_window(pid: int) -> int | None:
found = []
@ctypes.WINFUNCTYPE(wintypes.BOOL, wintypes.HWND, wintypes.LPARAM)
def visit(hwnd, _):
owner = wintypes.DWORD()
user32.GetWindowThreadProcessId(hwnd, ctypes.byref(owner))
if owner.value == pid and user32.IsWindowVisible(hwnd):
buf = ctypes.create_unicode_buffer(256)
user32.GetWindowTextW(hwnd, buf, 256)
if buf.value.strip() == "Strata":
found.append(hwnd)
return True
user32.EnumWindows(visit, 0)
return found[0] if found else None
class Scratch:
def __init__(self, root: Path, install: Path):
self.root, self.install = root, install
self.components = root / "components"
self.models = root / "models"
self.mark = root / "mark"
for d in (self.components, self.models, self.mark, root / "local", root / "home", root / "appdata", root / "webview"):
d.mkdir(parents=True, exist_ok=True)
install_fake_engine(self.components)
from tools.strata_tokenizer import BYTE_TO_UNICODE
tok = self.models / "tokenizer"
tok.mkdir()
tokens = [BYTE_TO_UNICODE[i] for i in range(256)] + ["ab", "<|im_start|>", "<|im_end|>", "<|endoftext|>"]
(tok / "vocab.json").write_text(json.dumps({t: i for i, t in enumerate(tokens)}))
(tok / "merges.txt").write_text("a b")
(tok / "token_type.json").write_text(json.dumps([0] * 257 + [3, 3, 3]))
for name in ("stub", "stub2"):
(self.models / f"strata-{name}.json").write_text(json.dumps({
"exe": "component:strata-engine", "backend": "cuda", "tokenizer": str(tok), "args": [], "cwd": str(root),
"model_name": name, "host": "127.0.0.1", "port": 0}))
self.env = {**os.environ, "STRATA_COMPONENTS_DIR": str(self.components), "STRATA_PROFILE_DIR": str(root / "profiles"),
"STRATA_RUNTIME_DIR": str(root / "runtime"), "STRATA_INSTALL_DIR": str(self.models),
"STRATA_MODELS_DIR": str(root / "models-unused"), "STRATA_ENGINE_BACKEND": "cuda",
"STRATA_HOME": str(root / "home"), "APPDATA": str(root / "appdata"), "LOCALAPPDATA": str(root / "local"),
"WEBVIEW2_USER_DATA_FOLDER": str(root / "webview"), "FAKE_ENGINE_MARK": str(self.mark),
"FAKE_ENGINE_SLOW_STEPS": "0", "PYTHONDONTWRITEBYTECODE": "1"}
self.env.pop("STRATA_API_KEY", None)
for key in ("STRATA_COMPONENTS_DIR", "STRATA_PROFILE_DIR", "STRATA_RUNTIME_DIR", "LOCALAPPDATA", "STRATA_ENGINE_BACKEND",
"STRATA_NO_ACQUIRE", "FAKE_ENGINE_MARK"):
os.environ[key] = self.env.get(key, "1") # this process reads the same runtime and records folders
class App:
def __init__(self, scratch: Scratch):
self.s = scratch
self.proc = subprocess.Popen([str(scratch.install / "strata-desktop.exe")], cwd=scratch.install, env=scratch.env,
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
self.base = None
def host(self):
for child in psutil.Process(self.proc.pid).children(recursive=True):
if child.name().lower() == "node.exe":
return child
return None
def wait_up(self, seconds=90):
deadline = time.monotonic() + seconds
while time.monotonic() < deadline:
if self.proc.poll() is not None:
raise RuntimeError(f"the app exited with {self.proc.returncode}")
try:
host = self.host()
if host:
for c in host.net_connections(kind="tcp"):
if c.status == "LISTEN" and c.laddr.ip == "127.0.0.1":
base = f"http://127.0.0.1:{c.laddr.port}"
try:
if urllib.request.urlopen(base + "/api/health", timeout=2).status == 200 and main_window(self.proc.pid):
self.base = base
return
except OSError:
pass
except psutil.Error:
pass
time.sleep(0.5)
raise RuntimeError("the app did not come up")
def call(self, path, method="GET", body=None):
req = urllib.request.Request(self.base + path, method=method, data=None if body is None else json.dumps(body).encode(),
headers={"content-type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=120) as r:
return json.loads(r.read() or b"null")
except urllib.error.HTTPError as exc:
raise RuntimeError(f"{method} {path} -> {exc.code} {exc.read()[:300]!r}") from None
def load(self, model, multiple=False):
self.call("/api/native?refresh=1")
started = self.call("/api/native/operations", "POST", {"kind": "load", "args": {"model": model, **({"multiple": True} if multiple else {})}})
deadline = time.monotonic() + 180
while time.monotonic() < deadline:
state = self.call("/api/native")
job = next((j for j in state["jobs"] if j["id"] == started["id"]), None)
if job and job["state"] != "running":
if job["state"] != "done":
raise RuntimeError(f"load {model}: {job['state']} {job.get('error')}")
return state
time.sleep(0.3)
raise RuntimeError("load did not finish")
def tree_of(app: App, slots=("app-1",)) -> tuple[list[procs.Ident], list[int]]:
idents, ports = [], []
members = [psutil.Process(app.proc.pid), *psutil.Process(app.proc.pid).children(recursive=True)]
pids = {p.pid for p in members}
for slot in slots:
state = runtime._read_state(slot)
if state:
pids.add(state["host_pid"])
for r in procs.records():
if r.get("profile") in slots: # the app's own slots; a Terminal service is not part of the app's tree
pids.add(r["pid"])
for pid in pids:
token = procs.start_token(pid)
if token:
idents.append(procs.Ident(pid, token, tuple(procs.command_line(pid) or [])))
return idents, ports
def survivors(idents):
return [f"{i.pid} {' '.join(i.cmd)[:80]}" for i in idents if procs.start_token(i.pid) == i.token]
def wait_gone(idents, seconds):
deadline = time.monotonic() + seconds
left = survivors(idents)
while left and time.monotonic() < deadline:
time.sleep(0.25)
left = survivors(idents)
return left
def lease_free() -> bool:
try:
fd = runtime.acquire_activity_lease(shared=False)
except Exception: # noqa: BLE001
return False
runtime._unlock_file(fd)
os.close(fd)
return True
def control(scratch: Scratch, *argv: str, timeout: int = 300) -> dict:
py = scratch.install / "control" / "python" / "python.exe"
out = subprocess.run([str(py), str(scratch.install / "control" / "strata.py"), *argv, "--output", "json"],
env={**scratch.env, "PYTHONNOUSERSITE": "1", "STRATA_NO_ACQUIRE": "1"},
capture_output=True, text=True, timeout=timeout)
try:
return json.loads(out.stdout)
except ValueError:
return {"ok": False, "raw": out.stdout[-300:] + out.stderr[-300:]}
def stop_cli(scratch: Scratch, slot: str) -> dict:
py = scratch.install / "control" / "python" / "python.exe"
out = subprocess.run([str(py), str(scratch.install / "control" / "strata.py"), "runtime", "stop", "--profile", slot, "--wait", "30",
"--output", "json"], env={**scratch.env, "PYTHONNOUSERSITE": "1", "STRATA_NO_ACQUIRE": "1"},
capture_output=True, text=True, timeout=120)
try:
return json.loads(out.stdout)
except ValueError:
return {"state": "?", "raw": out.stdout[-200:] + out.stderr[-200:]}
def case(name: str, install: Path, how: str) -> dict:
with tempfile.TemporaryDirectory(prefix="strata-app-matrix-", ignore_cleanup_errors=True) as tmp:
saved = dict(os.environ)
scratch = Scratch(Path(tmp), install)
app = App(scratch)
problems, note = [], ""
try:
app.wait_up()
slots = ("app-1", "app-2") if how == "two-engines" else ("app-1",)
terminal = None
if how == "window-x-terminal-engine":
# an engine the Terminal started (its own profile, no tie to the app) must survive the app's close
control(scratch, "profile", "create", "terminal-1", "--from-config", str(scratch.models / "strata-stub2.json"))
terminal = control(scratch, "runtime", "start", "--profile", "terminal-1", "--wait", "60")
if terminal.get("state") != "ready":
raise RuntimeError(f"the Terminal's service did not start: {terminal}")
terminal_pids = {r["pid"] for r in procs.records(profile="terminal-1")} | {runtime._read_state("terminal-1")["host_pid"]}
state = app.load("stub")
if how == "two-engines":
state = app.load("stub2", multiple=True)
engine_ports = [i["port"] for i in state["instances"]]
idents, _ = tree_of(app, slots)
kinds = sorted({("engine" if "engine" in " ".join(i.cmd) else "server" if "serve.server" in " ".join(i.cmd) else
"owner" if "strata_control.runtime" in " ".join(i.cmd) else "app" if "strata-desktop" in i.cmd[0].lower()
else "host" if "node" in i.cmd[0].lower() else "other") for i in idents})
note = f"tree {len(idents)} processes: {', '.join(kinds)}; engine ports {engine_ports}"
t0 = time.monotonic()
if how == "normal-close":
subprocess.run(["taskkill", "/PID", str(app.proc.pid)], capture_output=True)
elif how in ("window-x", "two-engines", "window-x-terminal-engine"):
hwnd = main_window(app.proc.pid)
if not hwnd:
raise RuntimeError("no main window")
user32.PostMessageW(hwnd, WM_CLOSE, 0, 0)
elif how == "kill-app":
subprocess.run(["taskkill", "/F", "/PID", str(app.proc.pid)], capture_output=True)
elif how == "host-crash":
host = app.host()
subprocess.run(["taskkill", "/F", "/PID", str(host.pid)], capture_output=True)
elif how == "second-launch":
second = subprocess.Popen([str(install / "strata-desktop.exe")], cwd=install, env=scratch.env,
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
try:
code = second.wait(timeout=30)
except subprocess.TimeoutExpired:
second.kill()
problems.append("the second launch did not exit")
code = None
time.sleep(2)
hosts = [p for p in psutil.process_iter(["name", "pid"]) if p.info["name"].lower() == "node.exe"
and (p.parent() and p.parent().pid == app.proc.pid)]
apps = [p for p in psutil.process_iter(["name"]) if p.info["name"].lower() == "strata-desktop.exe"]
if app.proc.poll() is not None:
problems.append("the first app went away")
if len(apps) != 1:
problems.append(f"{len(apps)} app processes (want 1)")
if len(hosts) != 1:
problems.append(f"{len(hosts)} hosts under the first app (want 1)")
after, _ = tree_of(app, slots)
new = [i for i in after if i.pid not in {x.pid for x in idents}]
if new:
problems.append(f"the second launch started: {[' '.join(i.cmd)[:60] for i in new]}")
note += f"; second launch exit code {code}, app processes {len(apps)}, hosts {len(hosts)}"
subprocess.run(["taskkill", "/PID", str(app.proc.pid)], capture_output=True)
if how == "host-crash":
# the shell and its window stay (nothing here restarts a dead host); the host, Control's owner, the server and
# the engine must not
idents = [i for i in idents if any(k in " ".join(i.cmd) for k in ("strata_control.runtime", "serve.server", "engine", "node.exe"))]
left = wait_gone(idents, 90)
if left:
problems.append(f"processes left: {left}")
for port in engine_ports:
deadline = time.monotonic() + 10
while port_open(port) and time.monotonic() < deadline:
time.sleep(0.2)
if port_open(port):
problems.append(f"engine port {port} still open")
if terminal is None and not lease_free():
problems.append("lease still held")
marks = sorted(p.name for p in scratch.mark.iterdir())
if terminal is not None:
status = control(scratch, "runtime", "status", "--profile", "terminal-1")
alive_pids = [p for p in terminal_pids if procs.start_token(p)]
if status.get("state") != "ready" or len(alive_pids) != len(terminal_pids):
problems.append(f"closing the app disturbed the Terminal's service: {status.get('state')}, alive {alive_pids} of {sorted(terminal_pids)}")
note += f"; the Terminal's service is {status.get('state')} after the app closed"
stopped = control(scratch, "runtime", "stop", "--profile", "terminal-1", "--wait", "30")
if stopped.get("state") != "stopped" or [p for p in terminal_pids if procs.start_token(p)]:
problems.append(f"the Terminal's service did not stop on request: {stopped}")
time.sleep(1)
if not lease_free():
problems.append("lease still held after the Terminal's service stopped")
if how in ("normal-close", "window-x", "two-engines", "second-launch", "window-x-terminal-engine") and not ({"quit", "eof"} & set(marks)):
problems.append(f"the engine was not stopped the clean way (marks {marks})")
seconds = round(time.monotonic() - t0, 1)
if how == "host-crash":
subprocess.run(["taskkill", "/PID", str(app.proc.pid)], capture_output=True)
note += "; the app shell stays open after a host crash (by design today)"
# the slot must be usable again: stopping a lost owner's slot cleans its state; nothing blocks the next load
for slot in slots:
res = stop_cli(scratch, slot)
already = "not managed or is stopped" in json.dumps(res) # nothing was left to stop: the clean outcome
if res.get("state") not in ("stopped",) and not already:
problems.append(f"runtime stop {slot} after the close: {res}")
if runtime._read_state(slot) is not None:
problems.append(f"state of {slot} left behind")
return {"name": f"app-{how}", "ok": not problems, "problems": problems, "seconds": seconds, "marks": marks, "note": note}
except Exception as exc: # noqa: BLE001
return {"name": f"app-{how}", "ok": False, "problems": [f"{type(exc).__name__}: {exc}"] + problems, "note": note}
finally:
for p in psutil.process_iter(["name", "exe"]):
try:
if (p.info["exe"] or "").lower().startswith(str(install).lower()):
p.kill()
except psutil.Error:
pass
for r in procs.records():
try:
psutil.Process(r["pid"]).kill()
except psutil.Error:
pass
os.environ.clear(); os.environ.update(saved)
SCENARIOS = ["normal-close", "window-x", "window-x-terminal-engine", "kill-app", "host-crash", "second-launch", "two-engines"]
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--install", required=True)
ap.add_argument("--scenarios", default="all")
ap.add_argument("--report")
a = ap.parse_args()
names = SCENARIOS if a.scenarios == "all" else a.scenarios.split(",")
results = []
for how in names:
r = case(how, Path(a.install), how)
results.append(r)
print(("PASS " if r["ok"] else "FAIL ") + r["name"], r.get("seconds", ""), r.get("problems") or "", r.get("note", ""), flush=True)
if a.report:
Path(a.report).write_text(json.dumps({"platform": "windows", "engine": "fake", "results": results}, indent=1), encoding="utf-8")
return 0 if all(r["ok"] for r in results) else 1
if __name__ == "__main__":
raise SystemExit(main())
+110
View File
@@ -0,0 +1,110 @@
#!/usr/bin/env node
/**
* The Graphics cards choice of the app, end to end on a box with two cards, through the installed app layout (packaged host + bundled Control + a real
* engine component): the model list carries the options (one rule, strata_control/gpu_fit.py), a model that fits one card recommends that card, then
* load with ONE card -> answers, only that card holds the model; load with BOTH cards (the layer split) -> answers, both cards hold the model.
*
* node cards-e2e.mjs --install <folder with node/ host/ control/> --components <folder holding engine/<package>> --work <scratch> \
* --family qwen --model IQ3_XXS --data <data folder that has the model files> [--smi "nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits"]
*/
import { spawn, spawnSync } from 'node:child_process';
import { existsSync, mkdirSync, readdirSync } from 'node:fs';
import { createServer } from 'node:net';
import { join, resolve } from 'node:path';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]?.startsWith('--') || all[i + 1] === undefined ? true : all[i + 1]]] : acc), []));
for (const need of ['install', 'components', 'work', 'model']) if (!args[need]) { console.error('missing --' + need); process.exit(2); }
const install = resolve(args.install), work = resolve(args.work), components = resolve(args.components), family = String(args.family ?? 'qwen'), modelId = String(args.model);
const node = join(install, 'node', 'node'), hostEntry = join(install, 'host', 'server', 'index.mjs');
const dirs = { models: join(work, 'models'), profiles: join(work, 'profiles'), runtime: join(work, 'runtime'), home: join(work, 'home'), xdg: join(work, 'xdg'), appdata: join(work, 'appdata') };
for (const d of [dirs.models, dirs.home, dirs.xdg, dirs.appdata]) mkdirSync(d, { recursive: true });
const delay = ms => new Promise(r => setTimeout(r, ms));
let failed = false;
const record = (name, ok, detail = '') => { if (!ok) failed = true; console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ' ' + String(detail).slice(0, 500) : ''}`); };
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
const env = { ...process.env, STRATA_COMPONENTS_DIR: components, STRATA_PROFILE_DIR: dirs.profiles, STRATA_RUNTIME_DIR: dirs.runtime, STRATA_INSTALL_DIR: dirs.models,
STRATA_HOME: dirs.home, STRATA_MODELS_DIR: join(work, 'models-unused'), APPDATA: dirs.appdata, XDG_CONFIG_HOME: dirs.xdg, PYTHONDONTWRITEBYTECODE: '1' };
delete env.STRATA_API_KEY;
const port = await freePort();
const base = `http://127.0.0.1:${port}`;
const host = spawn(node, [hostEntry], { cwd: join(install, 'host'), stdio: ['ignore', 'pipe', 'pipe'],
env: { ...env, STRATA_PORT: String(port), STRATA_DIST: join(install, 'host', 'dist'), NODE_ENV: 'production', STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1', STRATA_NO_ACQUIRE: '1',
STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()) } });
let hostLog = '';
host.stdout.on('data', c => { hostLog += c; }); host.stderr.on('data', c => { hostLog += c; });
const call = async (path, method = 'GET', body) => {
const response = await fetch(base + path, { method, headers: { 'content-type': 'application/json' }, body: body === undefined ? undefined : JSON.stringify(body) });
return { status: response.status, body: await response.json().catch(() => null) };
};
async function waitJob(id, seconds) {
let last = '';
for (let i = 0; i < seconds * 2; i++) {
const state = (await call('/api/native')).body;
const job = state.jobs.find(j => j.id === id);
if (job && job.summary !== last) { last = job.summary; console.log(' ... ' + String(last).slice(0, 120)); }
if (job && job.state !== 'running') return { job, state };
await delay(500);
}
throw new Error('operation did not finish');
}
const smi = () => {
const cmd = String(args.smi ?? 'nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits').split(' ');
const r = spawnSync(cmd[0], cmd.slice(1), { encoding: 'utf8' });
return (r.stdout ?? '').split('\n').map(s => Number(s.trim())).filter(Number.isFinite);
};
async function chatPong(instance) {
const chat = await call('/v1/chat/completions', 'POST', { model: instance?.apiModelId ?? modelId.toLowerCase(), max_tokens: 400, stream: false, temperature: 0, messages: [{ role: 'user', content: 'Reply with exactly one word: pong' }] });
const m = chat.body?.choices?.[0]?.message ?? {};
return { ok: chat.status === 200 && /pong/i.test(`${m.content ?? ''}`), text: `${m.content ?? ''}`.slice(0, 60), status: chat.status };
}
async function main() {
let up = false;
for (let i = 0; i < 200 && !up; i++) { try { up = (await fetch(base + '/api/health')).ok; } catch { /* not yet */ } if (!up) await delay(150); }
record('host healthy', up, up ? base : hostLog.slice(-500));
if (!up) return;
const packages = existsSync(join(components, 'engine')) ? readdirSync(join(components, 'engine')) : [];
record('an engine component is installed', packages.length > 0, packages.join(', '));
// ---- the model list carries the choice
const native = (await call('/api/native?refresh=1')).body;
const gpus = native.hardware?.gpus ?? [];
record('the host sees two or more graphics cards', gpus.filter(g => g.usable !== false).length >= 2, JSON.stringify(gpus.map(g => ({ i: g.index, n: g.name, gb: g.vram_gb }))));
const size = native.catalog?.families?.find(f => f.family === family)?.sizes.find(s => s.model === modelId);
const opt = size?.gpu_options;
record('the model list offers the cards choice for this model', !!opt && opt.options.length >= 2, JSON.stringify(opt?.options?.map(o => ({ key: o.key, cards: o.cards, verdict: o.verdict, rec: o.recommended }))));
record('a model that fits one card recommends the single largest card (the split is offered, not forced)', opt?.recommended === 'single' && opt.options.find(o => o.key === 'split'), `recommended=${opt?.recommended}`);
const single = opt?.options.find(o => o.key === 'single')?.cards ?? [0];
const both = opt?.options.find(o => o.key === 'split')?.cards ?? [0, 1];
// ---- Download and prepare with the recommended card
const values = { family, model: modelId, context: 32768, vision: 'no', gpu: single[0], ...(args.data ? { data_dir: resolve(args.data) } : {}) };
const started = await call('/api/native/operations', 'POST', { kind: 'install', args: values });
record('the install operation starts', started.status === 200 || started.status === 202, JSON.stringify(started.body).slice(0, 160));
const done = await waitJob(started.body.id, 3600);
record('Download and prepare finishes', done.job.state === 'done', `${done.job.state} ${done.job.error ?? done.job.summary}`);
const installed = (await call('/api/native?refresh=1')).body.installed?.models?.find(m => String(m.model).toLowerCase() === modelId.toLowerCase());
const model = installed?.model ?? modelId.toLowerCase();
for (const [label, cards] of [['ONE card', single], ['BOTH cards (layer split)', both]]) {
const before = smi();
const load = await call('/api/native/operations', 'POST', { kind: 'load', args: { model, gpus: cards } });
const loaded = await waitJob(load.body.id, 900);
const instance = loaded.state.instances[0];
record(`load with ${label} starts the engine`, loaded.job.state === 'done' && instance?.status === 'ready', `${loaded.job.state} ${loaded.job.error ?? ''}; instance=${instance?.status}; gpus=${JSON.stringify(instance?.gpus)}`);
record(`the instance runs on exactly the chosen card(s)`, JSON.stringify([...(instance?.gpus ?? [])].sort()) === JSON.stringify([...cards].sort()), `want ${JSON.stringify(cards)} got ${JSON.stringify(instance?.gpus)}`);
await delay(3000);
const used = smi();
const holding = used.map((mb, i) => (mb - (before[i] ?? 0)) > 1000 ? i : -1).filter(i => i >= 0);
record(`only the chosen card(s) hold the model (graphics memory in use: ${used.join(', ')} MiB)`, JSON.stringify(holding) === JSON.stringify([...cards].sort()), `holding ${JSON.stringify(holding)}`);
const pong = await chatPong(instance);
record(`it answers (${label})`, pong.ok, `${pong.status} ${JSON.stringify(pong.text)}`);
const unloaded = await call(`/api/native/instances/${instance?.id}/unload`, 'POST', {});
record(`unload (${label})`, unloaded.status === 200, unloaded.status);
await delay(4000);
}
}
try { await main(); } catch (e) { record('run', false, e.stack ?? e); }
host.kill();
await delay(1500);
console.log(failed ? 'CARDS-E2E FAIL' : 'CARDS-E2E PASS');
process.exit(failed ? 1 : 0);
+72
View File
@@ -0,0 +1,72 @@
# Fetch and verify the private Node runtime the Windows desktop bundle ships (win-x64).
#
# The PowerShell twin of fetch-node.sh: same pinned version, same SHA-256 pin, same refusal to
# install anything whose digest does not match. Not committed output: node.exe is a ~90 MB binary.
#
# Usage:
# powershell -NoProfile -ExecutionPolicy Bypass -File apps\desktop\scripts\fetch-node.ps1
# ... -File fetch-node.ps1 -Check # verify what is already installed
# ... -File fetch-node.ps1 -FromFile node.zip # verify and install a zip you already have (offline)
#
# The Node version must satisfy the App Host's own engines.node (>= 24.12.0).
[CmdletBinding()]
param(
[switch]$Check,
[string]$FromFile = '',
[string]$Dest = ''
)
$ErrorActionPreference = 'Stop'
$NodeVersion = if ($env:STRATA_NODE_VERSION) { $env:STRATA_NODE_VERSION } else { '24.12.0' }
$Target = 'win-x64'
# SHA-256 of node-v24.12.0-win-x64.zip from nodejs.org's SHASUMS256.txt; keep equal to fetch-node.sh.
$Expected = '9c125f61ae947b52e779095830f9cac267846a043ef7192183c84016aaad2812'
if (-not $Dest) { $Dest = Join-Path $PSScriptRoot '..\src-tauri\resources\node' }
$Installed = Join-Path $Dest 'node.exe'
function Fail([string]$Message) { [Console]::Error.WriteLine("fetch-node: $Message"); exit 1 }
if ($Check) {
if (-not (Test-Path -LiteralPath $Installed)) { Fail "not installed at $Installed" }
$got = (& $Installed --version).Trim()
Write-Output "fetch-node: $Installed is $got"
if ($got -ne "v$NodeVersion") { Fail "expected v$NodeVersion" }
exit 0
}
$Archive = "node-v$NodeVersion-$Target.zip"
$Work = Join-Path ([IO.Path]::GetTempPath()) ("strata-node-" + [Guid]::NewGuid().ToString('N'))
New-Item -ItemType Directory -Path $Work | Out-Null
try {
$Zip = Join-Path $Work $Archive
if ($FromFile) {
Copy-Item -LiteralPath $FromFile -Destination $Zip
} else {
$Url = "https://nodejs.org/dist/v$NodeVersion/$Archive"
Write-Output "fetch-node: $Url"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest -UseBasicParsing -Uri $Url -OutFile $Zip
}
$Actual = (Get-FileHash -Algorithm SHA256 -LiteralPath $Zip).Hash.ToLowerInvariant()
if ($NodeVersion -ne '24.12.0') {
# another version has no recorded digest: say so rather than implying a verification
[Console]::Error.WriteLine("fetch-node: WARNING - no recorded digest for v$NodeVersion; this download is NOT authenticated.")
[Console]::Error.WriteLine("fetch-node: computed sha256 $Actual - record it in this script and fetch-node.sh.")
} elseif ($Actual -ne $Expected) {
Fail "REFUSING - $Archive hashes $Actual, expected $Expected"
} else {
Write-Output 'fetch-node: sha256 verified against the recorded digest'
}
Add-Type -AssemblyName System.IO.Compression.FileSystem
$zipFile = [IO.Compression.ZipFile]::OpenRead($Zip)
try {
$entry = $zipFile.GetEntry("node-v$NodeVersion-$Target/node.exe")
if (-not $entry) { Fail "node.exe is not inside $Archive" }
New-Item -ItemType Directory -Force -Path $Dest | Out-Null
[IO.Compression.ZipFileExtensions]::ExtractToFile($entry, $Installed, $true)
} finally { $zipFile.Dispose() }
Write-Output "fetch-node: installed at $Installed"
} finally {
Remove-Item -LiteralPath $Work -Recurse -Force -ErrorAction SilentlyContinue
}
+103
View File
@@ -0,0 +1,103 @@
#!/usr/bin/env bash
# Fetch and verify the private Node runtime the desktop bundle ships.
#
# The App Host is an ordinary Node program, and the packaging requirement is that an end user
# supplies no runtime of their own. The desktop bundle therefore carries a pinned standalone Node
# beside the host.
#
# It is NOT committed. It is a ~117 MB platform binary: committing it would bloat every clone, it
# cannot be reviewed in a diff, and it is not source. This script is the reproducible substitute —
# it downloads the official build for the target and refuses to install anything whose SHA-256 does
# not match the recorded digest.
#
# Usage:
# apps/desktop/scripts/fetch-node.sh # host platform
# apps/desktop/scripts/fetch-node.sh --target linux-x64
# apps/desktop/scripts/fetch-node.sh --check # verify what is already installed
#
# The Node version must satisfy the App Host's own `engines.node` (>= 24.12.0).
set -euo pipefail
NODE_VERSION="${STRATA_NODE_VERSION:-24.12.0}"
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DEST="$HERE/../src-tauri/resources/node"
# Published SHA-256 of each official nodejs.org archive, recorded so a tampered or truncated
# download is refused rather than shipped. Add a row when bumping the version.
# Taken from nodejs.org's own SHASUMS256.txt for v24.12.0, which is published over HTTPS beside
# the archives. Recorded here so the fetch is verified against a value that does not come from the
# same response as the payload.
digest_for() {
case "$1" in
darwin-arm64) echo "319f221adc5e44ff0ed57e8a441b2284f02b8dc6fc87b8eb92a6a93643fd8080" ;;
linux-x64) echo "6159227e0af7d7c3c6bb2fa900452b04a6cb8841a702a79acc613209d70b04d0" ;;
linux-arm64) echo "9b2a2eeb98a8eb37361224e2a1d060300ad2dd143af58dfdb16de785df0f1228" ;;
win-x64) echo "9c125f61ae947b52e779095830f9cac267846a043ef7192183c84016aaad2812" ;;
*) echo "" ;;
esac
}
target() {
if [ "${1:-}" = "--target" ]; then echo "${2:?--target needs a value}"; return; fi
case "$(uname -s)-$(uname -m)" in
Darwin-arm64) echo darwin-arm64 ;;
Linux-x86_64) echo linux-x64 ;;
Linux-aarch64) echo linux-arm64 ;;
MINGW*|MSYS*|CYGWIN*) echo win-x64 ;;
*) echo "unsupported host: $(uname -s)-$(uname -m)" >&2; exit 2 ;;
esac
}
TARGET="$(target "$@")"
EXE="node"
case "$TARGET" in win-*) EXE="node.exe" ;; esac
INSTALLED="$DEST/$EXE"
if [ "${1:-}" = "--check" ]; then
if [ ! -x "$INSTALLED" ]; then echo "fetch-node: not installed at $INSTALLED" >&2; exit 1; fi
got="$("$INSTALLED" --version)"
echo "fetch-node: $INSTALLED is $got"
case "$got" in "v$NODE_VERSION") exit 0 ;; *) echo "fetch-node: expected v$NODE_VERSION" >&2; exit 1 ;; esac
fi
# The official archive for this target.
case "$TARGET" in
darwin-arm64) ARCHIVE="node-v$NODE_VERSION-darwin-arm64.tar.gz" ;;
darwin-x64) ARCHIVE="node-v$NODE_VERSION-darwin-x64.tar.gz" ;;
linux-x64) ARCHIVE="node-v$NODE_VERSION-linux-x64.tar.gz" ;;
linux-arm64) ARCHIVE="node-v$NODE_VERSION-linux-arm64.tar.gz" ;;
win-x64) ARCHIVE="node-v$NODE_VERSION-win-x64.zip" ;;
*) echo "fetch-node: no archive known for $TARGET" >&2; exit 2 ;;
esac
URL="https://nodejs.org/dist/v$NODE_VERSION/$ARCHIVE"
EXPECTED="$(digest_for "$TARGET")"
WORK="$(mktemp -d "${TMPDIR:-/tmp}/strata-node.XXXXXX")" # GNU mktemp needs the X template; BSD accepts it too
trap 'rm -rf "$WORK"' EXIT
echo "fetch-node: $URL"
curl -fsSL -o "$WORK/$ARCHIVE" "$URL"
ACTUAL="$(shasum -a 256 "$WORK/$ARCHIVE" 2>/dev/null | cut -d' ' -f1 || sha256sum "$WORK/$ARCHIVE" | cut -d' ' -f1)"
if [ -n "$EXPECTED" ]; then
if [ "$ACTUAL" != "$EXPECTED" ]; then
echo "fetch-node: REFUSING — $ARCHIVE hashes $ACTUAL, expected $EXPECTED" >&2
exit 1
fi
echo "fetch-node: sha256 verified against the recorded digest"
else
# Without a recorded digest the download cannot be authenticated. Say so loudly and keep the
# computed value so it can be recorded, rather than implying a verification that did not happen.
echo "fetch-node: WARNING — no recorded digest for $TARGET; this download is NOT authenticated." >&2
echo "fetch-node: computed sha256 $ACTUAL — record it in $(basename "${BASH_SOURCE[0]}") to enable verification." >&2
fi
mkdir -p "$DEST"
case "$ARCHIVE" in
*.zip) unzip -q -j "$WORK/$ARCHIVE" "node-v$NODE_VERSION-win-x64/node.exe" -d "$DEST" ;;
*) tar xzf "$WORK/$ARCHIVE" -C "$WORK"
cp "$WORK/node-v$NODE_VERSION-$TARGET/bin/node" "$DEST/node" ;;
esac
chmod +x "$DEST/$EXE"
echo "fetch-node: installed $("$DEST/$EXE" --version) at $DEST/$EXE"
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env node
/**
* The first run of the installed app, headless: the host as the Tauri shell starts it (STRATA_PACKAGED=1, STRATA_NO_ACQUIRE=1,
* the staged Control, nothing installed, nothing in the per-user folders) and Chrome on its onboarding.
*
* node apps/desktop/scripts/first-run-ui.mjs [--control <staged control>] [--node <node.exe>] [--work <scratch>] [--shots <dir>]
*
* Checks, one line each (exit 1 if any fails):
* the host answers; /api/native has a real catalog (families with sizes), a recommendation and the GPUs of this PC
* (from Control's bundled installer on Control's Python); the page shows "Let's get started", and after it the model
* list with sizes (no "couldn't show this screen"); no page error; the install plan for the recommended model is a
* plan (nothing is downloaded or started: a plan changes nothing); the spy sees no pip/git/compiler/engine fetch.
* Per-user folders (STRATA_HOME, APPDATA, LOCALAPPDATA, the components and install folders) are all under --work.
* No model is downloaded and no engine runs.
*/
import { spawn } from 'node:child_process';
import { existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
import { createServer } from 'node:net';
import { tmpdir } from 'node:os';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]?.startsWith('--') ? true : all[i + 1]]] : acc), []));
const here = dirname(fileURLToPath(import.meta.url));
const repoHost = resolve(here, '../../host');
const staged = resolve(here, '../src-tauri/resources');
const control = resolve(args.control ?? join(staged, 'control'));
const nodeExe = resolve(args.node ?? join(staged, 'node', process.platform === 'win32' ? 'node.exe' : 'node'));
const work = resolve(args.work ?? mkdtempSync(join(tmpdir(), 'strata-first-run-')));
const shots = resolve(args.shots ?? join(work, 'shots'));
mkdirSync(work, { recursive: true }); mkdirSync(shots, { recursive: true });
const spy = pathToFileURL(join(repoHost, 'tests/helpers/acquire-spy.mjs')).href;
const spyLog = join(work, 'spy.log'); writeFileSync(spyLog, '');
const { chromium } = await import(pathToFileURL(join(repoHost, 'node_modules/playwright-core/index.mjs')).href).catch(() => import('playwright-core'));
const CHROME = process.env.CHROME ?? ['C:/Program Files/Google/Chrome/Application/chrome.exe', '/usr/bin/google-chrome', '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'].find(existsSync);
let failed = false;
const check = (ok, label, detail = '') => { console.log(`${ok ? 'ok ' : 'FAIL'} ${label}${detail ? ' ' + detail : ''}`); if (!ok) failed = true; };
const delay = ms => new Promise(r => setTimeout(r, ms));
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
for (const need of [nodeExe, join(control, 'strata.py'), join(repoHost, 'dist', 'index.html')]) if (!existsSync(need)) { console.error(`missing ${need} (run stage-control.py, fetch-node and "npm run build" in apps/host first)`); process.exit(2); }
const dirs = { home: join(work, 'home'), appdata: join(work, 'appdata'), local: join(work, 'local'), components: join(work, 'local', 'Strata', 'components') };
for (const d of Object.values(dirs)) mkdirSync(d, { recursive: true });
const port = await freePort();
const env = {
...process.env,
STRATA_PORT: String(port), STRATA_DIST: join(repoHost, 'dist'), STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1', STRATA_NO_ACQUIRE: '1',
STRATA_CONTROL_DIR: control, STRATA_HOME: dirs.home, STRATA_MODELS_DIR: join(work, 'models'), STRATA_COMPONENTS_DIR: dirs.components,
APPDATA: dirs.appdata, LOCALAPPDATA: dirs.local, XDG_CONFIG_HOME: join(work, 'xdg'), XDG_DATA_HOME: join(work, 'xdgdata'),
STRATA_PROFILE_DIR: join(work, 'profiles'), STRATA_RUNTIME_DIR: join(work, 'runtime'), ACQUIRE_SPY_LOG: spyLog,
};
delete env.STRATA_INSTALL_DIR;
mkdirSync(env.STRATA_MODELS_DIR, { recursive: true });
const host = spawn(nodeExe, [...(args['no-spy'] === undefined ? ['--import', spy] : []), join(repoHost, 'server', 'index.mjs')], { cwd: repoHost, env, stdio: ['ignore', 'pipe', 'pipe'] });
let hostLog = '';
host.stdout.on('data', c => { hostLog += c; }); host.stderr.on('data', c => { hostLog += c; });
const base = `http://127.0.0.1:${port}`;
const api = async (path, init) => { const r = await fetch(base + path, init); return { status: r.status, body: await r.json().catch(() => null) }; };
let browser;
try {
let up = false;
for (let i = 0; i < 150 && !up; i++) { try { up = (await api('/api/health')).status === 200; } catch { await delay(200); } }
check(up, 'the packaged host starts', `port ${port}`);
if (!up) throw new Error(hostLog.slice(-2000));
const native = await api('/api/native?refresh=1');
const state = native.body;
check(native.status === 200 && state?.available === true, 'the host reads Control', state?.error ?? '');
const sizes = (state?.catalog?.families ?? []).reduce((n, f) => n + (f.sizes?.length ?? 0), 0);
check((state?.catalog?.families?.length ?? 0) > 0 && sizes >= 3, 'the catalog has real families and sizes', `${state?.catalog?.families?.length} families, ${sizes} sizes`);
check(!!state?.catalog?.recommendation, 'the catalog has a recommendation', JSON.stringify(state?.catalog?.recommendation ?? null).slice(0, 120));
check(Array.isArray(state?.hardware?.gpus), 'the hardware lists GPUs', (state?.hardware?.gpus ?? []).map(g => `${g.name} ${g.vram_gb} GB`).join('; ') || 'none on this PC');
check(state?.installDir && realpathSync.native(state.installDir).toLowerCase().startsWith(realpathSync.native(work).toLowerCase()), 'models go to a writable per-user folder, not the bundle', state?.installDir);
const hw = (await api('/api/hardware')).body;
console.log('info host hardware probe:', JSON.stringify((hw?.gpus ?? []).map(g => ({ name: g.name, backend: g.backend, vramGb: g.vramGb, unified: g.unified }))));
check(!state?.installerError, 'the bundled installer answers', state?.installerError ?? '');
browser = await chromium.launch({ executablePath: CHROME, headless: true });
const page = await browser.newPage({ viewport: { width: 1400, height: 950 } });
const problems = [];
page.on('pageerror', e => problems.push(`pageerror: ${e.message}`));
await page.goto(base, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('text=/Let.s get started|Choose a version|Connect your Strata/i', { timeout: 30000 }).catch(() => {});
await page.screenshot({ path: join(shots, '1-welcome.png') });
const broken = await page.locator('text=/couldn.t show this screen/i').count();
check(broken === 0, 'the first screen renders (not "Strata couldn\'t show this screen")');
const start = page.getByRole('button', { name: /get started/i });
check(await start.count() > 0, 'the welcome step offers "Let\'s get started"');
if (await start.count()) await start.first().click();
await page.waitForSelector('text=/Choose a version/i', { timeout: 30000 }).catch(() => {});
await delay(1200);
await page.screenshot({ path: join(shots, '2-models.png') });
check(await page.locator('text=/couldn.t show this screen/i').count() === 0, 'the model step renders');
check(await page.getByText(/Choose a version/i).count() > 0, 'the model step lists the versions');
const labels = await page.locator('[role=radio]').allInnerTexts();
check(labels.length >= 3, 'the versions are the real catalog\'s sizes', labels.map(l => l.replace(/\s+/g, ' ').slice(0, 40)).join(' | '));
check(labels.some(l => /\d+\.\d\s*GB/.test(l)), 'each version shows its download size');
check(problems.length === 0, 'no page error', problems.join('; '));
const recommended = state?.catalog?.recommendation;
if (recommended?.model) {
const plan = await api('/api/native/plan', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ family: recommended.family, model: recommended.model, context: recommended.context, data_dir: join(work, 'Strata-data') }) });
check(plan.status === 200 && !!plan.body?.plan, 'the install plan for the recommended model is a plan (nothing downloaded)', plan.body?.summary?.slice(0, 100) ?? JSON.stringify(plan.body).slice(0, 150));
check(!existsSync(join(work, 'Strata-data', 'models')), 'a plan wrote no model files');
}
} catch (error) {
check(false, 'the run', String(error?.message ?? error).slice(0, 600));
} finally {
await browser?.close().catch(() => {});
host.kill();
await new Promise(r => host.once('exit', r)); // only the PID this script started
const attempts = readFileSync(spyLog, 'utf8').split('\n').filter(Boolean);
check(attempts.length === 0, 'the spy saw no pip, git, compiler, engine or internet access', attempts.slice(0, 3).join(' '));
console.log(`shots in ${shots}`);
if (!args.work) rmSync(work, { recursive: true, force: true, maxRetries: 3 });
}
process.exit(failed ? 1 : 0);
+133
View File
@@ -0,0 +1,133 @@
/**
* Turn a pnpm "isolated" node_modules (a virtual store in `.pnpm` full of links) into an ordinary hoisted tree of real folders.
*
* Why: the installer copies the Harness folder file by file. pnpm's layout is thousands of links (junctions on Windows); a copy
* that follows them duplicates every package once per dependent and one that does not breaks on the first dangling link. A hoisted
* tree has each package once (nested only where two versions of the same name are needed), no links, and Node resolves it
* exactly as it resolves any npm install.
*
* Algorithm (the one npm uses, in short): the packages the root names come first, at the top. Then, walking each placed package's
* own dependencies: look the name up from the package's own folder upwards; if the same instance is already visible, nothing to
* do; if a DIFFERENT instance of that name is visible, nest this one inside the dependent; if none is visible, hoist it to the top.
*
* @param {string} out folder holding `node_modules`
* @param {{skip?: RegExp}} [options] package names never copied (the caller removes them anyway)
* @returns {{packages:number, copied:number, nested:number}}
*/
import { cpSync, existsSync, lstatSync, mkdirSync, readFileSync, readdirSync, realpathSync, renameSync, rmSync } from 'node:fs';
import { dirname, join, relative, sep } from 'node:path';
export function flattenPnpm(out, { skip = null } = {}) {
const nm = join(out, 'node_modules');
const store = join(nm, '.pnpm');
const flatRoot = join(out, 'node_modules.flat');
rmSync(flatRoot, { recursive: true, force: true });
mkdirSync(join(flatRoot, 'node_modules'), { recursive: true });
const manifest = dir => { try { return JSON.parse(readFileSync(join(dir, 'package.json'), 'utf8')); } catch { return null; } };
const nameOf = dir => manifest(dir)?.name ?? null;
/**
* What makes two instances "the same package": name and version. pnpm keeps one copy per set of peer packages (the same workspace
* package under many hashed folders); in one hoisted tree they are one package.
*/
const identity = dir => { const m = manifest(dir); return m ? `${m.name}@${m.version}` : dir; };
/** The sibling links of a store instance: its dependencies, by name -> real instance folder. */
function storeDeps(instance, name) {
const container = name.startsWith('@') ? dirname(dirname(instance)) : dirname(instance);
const found = new Map();
for (const e of readdirSync(container, { withFileTypes: true })) {
if (e.name === '.bin') continue;
const entries = e.name.startsWith('@') ? readdirSync(join(container, e.name), { withFileTypes: true }).map(x => [`${e.name}/${x.name}`, join(container, e.name, x.name)]) : [[e.name, join(container, e.name)]];
for (const [dep, p] of entries) {
if (dep === name) continue;
try { if (lstatSync(p).isSymbolicLink() || existsSync(p)) found.set(dep, realpathSync(p)); } catch { /* a dangling link: the package was removed on purpose */ }
}
}
return found;
}
/** Every instance in the store by package name, for packages that arrived without a store entry (copied peers). */
const byName = new Map();
if (existsSync(store)) {
for (const id of readdirSync(store, { withFileTypes: true })) {
if (!id.isDirectory()) continue;
const container = join(store, id.name, 'node_modules');
if (!existsSync(container)) continue;
for (const e of readdirSync(container, { withFileTypes: true })) {
if (e.name === '.bin') continue;
const dirs = e.name.startsWith('@') ? readdirSync(join(container, e.name)).map(x => join(container, e.name, x)) : [join(container, e.name)];
for (const d of dirs) {
let real; try { real = realpathSync(d); } catch { continue; }
if (!real.startsWith(store)) continue; // a link to a workspace package outside the store is handled through the instance table
const n = nameOf(real);
if (n && !byName.has(n)) byName.set(n, real);
}
}
}
}
const placed = new Map(); // folder (in the flat tree) -> real instance folder it holds
let copied = 0, nested = 0;
const visibleFrom = (chain, name) => {
for (const dir of chain) {
const slot = join(dir, 'node_modules', ...name.split('/'));
if (placed.has(slot)) return { slot, instance: placed.get(slot) };
}
return null;
};
const copyInto = (slot, instance) => {
mkdirSync(dirname(slot), { recursive: true });
cpSync(instance, slot, { recursive: true, dereference: true, filter: f => !relative(instance, f).split(sep).includes('node_modules') });
placed.set(slot, identity(instance));
copied++;
};
const queue = []; // { slotDir (folder of the placed package), chain (folders searched upwards), instance, name }
const rootChain = [flatRoot];
const place = (name, instance, chain, forceHere) => {
if (skip?.test(name)) return;
const seen = visibleFrom(chain, name);
if (seen && seen.instance === identity(instance)) return;
const hoist = !seen && !forceHere;
const base = hoist ? flatRoot : chain[0];
const slot = join(base, 'node_modules', ...name.split('/'));
if (placed.has(slot)) return; // another instance already sits where this one would go (it nests below, handled by the caller's chain)
copyInto(slot, instance);
if (!hoist && base !== flatRoot) nested++;
queue.push({ slotDir: slot, chain: [slot, ...(hoist ? rootChain : chain)], instance, name });
};
// 1. the packages the root names
const rootEntries = [];
for (const e of readdirSync(nm, { withFileTypes: true })) {
if (e.name === '.pnpm' || e.name === '.bin' || e.name.startsWith('.')) continue;
const entries = e.name.startsWith('@') ? readdirSync(join(nm, e.name)).map(x => [`${e.name}/${x}`, join(nm, e.name, x)]) : [[e.name, join(nm, e.name)]];
for (const [name, p] of entries) rootEntries.push([name, p]);
}
for (const [name, p] of rootEntries) {
let real; try { real = realpathSync(p); } catch { continue; }
place(name, real, rootChain, true);
}
// 2. their dependencies, breadth first
while (queue.length) {
const { slotDir, chain, instance, name } = queue.shift();
let wants;
if (instance.startsWith(store)) wants = storeDeps(instance, name);
else {
// a package copied in without a store entry: its declared requirements, taken from the store by name when not visible yet
const m = manifest(instance) ?? {};
wants = new Map();
for (const dep of [...Object.keys(m.dependencies ?? {}), ...Object.keys(m.peerDependencies ?? {}), ...Object.keys(m.optionalDependencies ?? {})]) {
if (visibleFrom(chain, dep)) continue;
const real = byName.get(dep);
if (real) wants.set(dep, real);
}
}
for (const [dep, real] of wants) place(dep, real, chain, false);
}
rmSync(nm, { recursive: true, force: true });
renameSync(join(flatRoot, 'node_modules'), nm);
rmSync(flatRoot, { recursive: true, force: true });
return { packages: placed.size, copied, nested };
}
+79
View File
@@ -0,0 +1,79 @@
/**
* Which files and folders of the staged Harness are dropped, and the check that nothing the Harness imports is gone.
*
* The 2026-10-17 tester report: on Linux and Mac every agent chat failed ("session/create ... waiting for workflowEngine").
* Cause: the old rule "drop any folder named after another operating system" also removed the Harness's own JavaScript package
* `@deepseek-ai/dsh-win32-process`. It is plain JS (no binary), and `dsh-subprocess-local` and `dsh-sandbox-local` import it by
* name on every platform, so those two plugins failed to load, and everything waiting on them (shell, subprocess, ptcRuntime,
* workflowEngine) never started. Windows kept it (the rule differs there), so the Windows tests passed.
*/
import { existsSync, readFileSync, readdirSync } from 'node:fs';
import { join, sep } from 'node:path';
export const OTHER_OS = {
win32: /(^|[\\/._-])(darwin|linux|android|freebsd|musl)([\\/._-]|$)/i,
darwin: /(^|[\\/._-])(win32|linux|android|freebsd|musl)([\\/._-]|$)/i,
// Linux: the AppImage bundler (linuxdeploy) resolves the libraries of every ELF file under resources and stops on one it
// cannot satisfy (koffi's musl build wants libc.musl). The app is glibc x86_64, so musl and other-CPU builds are dropped.
linux: /(^|[\\/._-])(win32|darwin|android|freebsd|musl|arm64|aarch64|armv7|arm|ia32|riscv64|ppc64|s390x|loong64)([\\/._-]|$)/i,
};
const NATIVE_FOLDER = /prebuild|bin|native|platform|-(win32|darwin|linux)/i;
/** The Harness's own JavaScript packages (`@deepseek-ai/dsh-*`) are code every platform imports by name; never drop one. */
export function isHarnessCodePackage(path) {
const parts = path.split(sep);
const i = parts.lastIndexOf('@deepseek-ai');
return i >= 0 && i === parts.length - 2 && parts[i + 1].startsWith('dsh-');
}
/** True when the folder `path` (a directory named `name`) is another operating system's prebuilt binary and can go. */
export function isOtherOsBinaryFolder(platform, name, path) {
const rule = OTHER_OS[platform] ?? OTHER_OS.linux;
if (isHarnessCodePackage(path)) return false;
return rule.test(name) && NATIVE_FOLDER.test(path);
}
/** Imports the Windows copy (which works) is also without: browser-side or dev-only bundles, and the document converter the app drops on purpose. */
const KNOWN_ABSENT = new Set([
'@deepseek-ai/libreoffice-kit', '@deepseek-ai/dsh-client-ui-dockkit', '@deepseek-ai/dsh-client-web',
'@deepseek-ai/dsh-experimental-webworker-runtime', '@deepseek-ai/dsh-remote-mock', '@deepseek-ai/dsh-client-test-runtime',
]);
function jsFiles(dir, out = []) {
for (const e of readdirSync(dir, { withFileTypes: true })) {
const p = join(dir, e.name);
if (e.isSymbolicLink()) continue;
if (e.isDirectory()) jsFiles(p, out);
else if (/\.(m?js|cjs)$/.test(e.name)) out.push(p);
}
return out;
}
/**
* Every bare `@deepseek-ai/<package>` the staged Harness's own packages import (static `from "x"` and `import("x")`) must exist
* in the staged node_modules. Returns the list of `{from, missing}` problems (empty = good). Plugins that are only loaded on
* another platform still count: Node resolves a static import whether or not the code runs.
*/
export function missingHarnessImports(harnessDir) {
const scope = join(harnessDir, 'node_modules', '@deepseek-ai');
if (!existsSync(scope)) return [{ from: scope, missing: '@deepseek-ai (no installed packages)' }];
const problems = [];
const seen = new Set();
const re = /(?:from|import)\s*\(?\s*["'](@deepseek-ai\/[^"'/]+)(?:\/[^"']*)?["']/g;
for (const pkg of readdirSync(scope)) {
const lib = join(scope, pkg, 'lib');
if (!existsSync(lib)) continue;
for (const file of jsFiles(lib)) {
const text = readFileSync(file, 'utf8');
for (const m of text.matchAll(re)) {
const name = m[1];
const key = `${pkg}>${name}`;
if (seen.has(key) || KNOWN_ABSENT.has(name)) continue;
seen.add(key);
if (!existsSync(join(harnessDir, 'node_modules', ...name.split('/')))
&& !existsSync(join(scope, pkg, 'node_modules', ...name.split('/')))) problems.push({ from: `${pkg}`, missing: name });
}
}
}
return problems;
}
+310
View File
@@ -0,0 +1,310 @@
#!/usr/bin/env node
/**
* Prove the INSTALLED app layout can start and stop an engine with nothing acquired, using a stub engine.
*
* node apps/desktop/scripts/installed-flow.mjs --install <folder the installer wrote> [--work <scratch>] [--spy <acquire-spy.mjs>]
*
* It starts the installed host with the installed node.exe exactly as the Tauri shell does (STRATA_PACKAGED=1,
* STRATA_NO_ACQUIRE=1; Control found beside the host), against a scratch component folder holding a fake engine
* package and a scratch install folder holding one run config. Everything per-user (profiles, runtime state, host data,
* components) lives under --work, so the real profile of whoever runs this is never read or written.
*
* Steps (each one printed, the run exits 1 if any fails):
* host health, Control status (engine usable, acquiring forbidden), models from Control, load through the host
* (profile + `runtime start` under the owner runtime), a chat answer from the stub engine, a second client attaches to
* the same owner/instance, unload stops it, no engine for this PC is a notice with nothing started, the Tauri-style
* clean shutdown stops the engines the host started, a hard-killed host takes its engine with it.
* With --spy the host runs under tests/helpers/acquire-spy.mjs and the run fails on any attempt to reach the internet or
* run an installer.
*
* REAL mode (a test box with a GPU): --components <folder holding engine/<package>> --models <folder with strata-<id>.json
* whose engine is "component:strata-engine"> --model <id> [--backend cuda]. The same host and Control, a real engine and a
* real model: load, a real answer, ownership, unload, the missing-engine notice, load again, clean shutdown. The stub-only
* steps (the marker file, the hard-kill report) are left out.
*/
import { spawn, spawnSync } from 'node:child_process';
import { createHash, randomBytes } from 'node:crypto';
import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, renameSync, rmSync, writeFileSync, chmodSync } from 'node:fs';
import { createServer } from 'node:net';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]?.startsWith('--') || all[i + 1] === undefined ? true : all[i + 1]]] : acc), []));
if (!args.install) { console.error('usage: installed-flow.mjs --install <folder> [--work <dir>] [--spy <acquire-spy.mjs>]'); process.exit(2); }
const install = resolve(args.install);
const isWin = process.platform === 'win32';
const node = join(install, 'node', isWin ? 'node.exe' : 'node');
const hostEntry = join(install, 'host', 'server', 'index.mjs');
const controlDir = join(install, 'control');
const python = isWin ? join(controlDir, 'python', 'python.exe') : join(controlDir, 'python', 'bin', 'python3');
const real = !!args.components;
const modelId = String(args.model ?? 'stub');
const work = resolve(args.work ?? mkdtempSync(join(tmpdir(), 'strata-installed-flow-')));
mkdirSync(work, { recursive: true });
const dirs = { components: real ? resolve(args.components) : join(work, 'components'), models: real ? resolve(args.models) : join(work, 'models'), profiles: join(work, 'profiles'), runtime: join(work, 'runtime'), home: join(work, 'home') };
// the runtime and profile folders are created by Control itself, with the owner-only ACL it insists on
for (const [name, d] of Object.entries(dirs)) if (!['runtime', 'profiles'].includes(name) && !(real && ['components', 'models'].includes(name))) mkdirSync(d, { recursive: true });
const steps = [];
const record = (name, ok, detail = '') => { steps.push({ name, ok: !!ok, detail: String(detail) }); console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ' ' + detail : ''}`); };
const delay = ms => new Promise(r => setTimeout(r, ms));
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
const sha = buf => createHash('sha256').update(buf).digest('hex');
// ------------------------------------------------------------------------------ fixtures
const baseEnv = {
...process.env,
STRATA_COMPONENTS_DIR: dirs.components, STRATA_PROFILE_DIR: dirs.profiles, STRATA_RUNTIME_DIR: dirs.runtime,
STRATA_INSTALL_DIR: dirs.models, STRATA_ENGINE_BACKEND: String(args.backend ?? 'cuda'), STRATA_HOME: dirs.home, STRATA_MODELS_DIR: real ? join(work, 'models-unused') : dirs.models,
APPDATA: join(work, 'appdata'), XDG_CONFIG_HOME: join(work, 'xdg'), PYTHONDONTWRITEBYTECODE: '1',
};
delete baseEnv.STRATA_API_KEY;
mkdirSync(baseEnv.APPDATA, { recursive: true });
const mark = join(work, 'engine-ran.json');
const FAKE_ENGINE = `import os, sys, json
from pathlib import Path
Path(os.environ['FAKE_ENGINE_MARK']).write_text(json.dumps({'pid': os.getpid(), 'script': os.path.abspath(__file__), 'argv': sys.argv[1:]}))
print('READY 4096 stop', flush=True)
for line in sys.stdin:
if line.strip() == 'QUIT': break
if line.startswith(('GEN ', 'BGEN ')):
slot = int(line.split()[1]) if line.startswith('BGEN ') else None
print('T 97', flush=True)
print('DONE 1 1 1.0 1.0 length 0 0 0', flush=True)
if slot is not None: print(f'BADM {slot} 0', flush=True)
`;
function installFakeEngine() {
const platform = isWin ? 'windows-x86_64' : 'linux-x86_64';
const folder = join(dirs.components, 'engine', `0.1.45-${platform}-cuda`);
mkdirSync(folder, { recursive: true });
writeFileSync(join(folder, 'engine_main.py'), FAKE_ENGINE);
const entry = isWin ? 'engine.cmd' : 'engine';
writeFileSync(join(folder, entry), isWin
? `@echo off\r\n"${python}" "%~dp0engine_main.py" %*\r\n`
: `#!/bin/sh\nexec "${python}" "$(dirname "$0")/engine_main.py" "$@"\n`);
if (!isWin) chmodSync(join(folder, entry), 0o755);
const files = [entry, 'engine_main.py'].map(p => { const b = readFileSync(join(folder, p)); return { path: p, sha256: sha(b), bytes: b.length }; });
writeFileSync(join(folder, 'engine-component.json'), JSON.stringify({ schema_version: 1, id: 'strata-engine', version: '0.1.45', platform, backend: 'cuda', entrypoint: entry, files }));
return folder;
}
function makeTokenizerAndConfig() {
const tok = join(dirs.models, 'tokenizer');
mkdirSync(tok, { recursive: true });
const script = `import json, sys\nfrom tools.strata_tokenizer import BYTE_TO_UNICODE\ntok = sys.argv[1]\ntokens = [BYTE_TO_UNICODE[i] for i in range(256)] + ['ab', '<|im_start|>', '<|im_end|>', '<|endoftext|>']\njson.dump({t: i for i, t in enumerate(tokens)}, open(tok + '/vocab.json', 'w'))\nopen(tok + '/merges.txt', 'w').write('a b')\njson.dump([0] * 257 + [3, 3, 3], open(tok + '/token_type.json', 'w'))\n`;
const done = spawnSync(python, ['-c', script, tok], { cwd: controlDir, env: { ...baseEnv, PYTHONNOUSERSITE: '1' }, encoding: 'utf8' });
if (done.status !== 0) throw new Error('tokenizer fixture failed: ' + done.stderr);
writeFileSync(join(dirs.models, 'strata-stub.json'), JSON.stringify({ exe: 'component:strata-engine', backend: 'cuda', tokenizer: tok, args: [], cwd: work, model_name: 'stub', host: '127.0.0.1', port: 0 }));
}
const control = (...argv) => spawnSync(python, [join(controlDir, 'strata.py'), ...argv, '--output', 'json'], { cwd: controlDir, env: { ...baseEnv, PYTHONNOUSERSITE: '1', STRATA_NO_ACQUIRE: '1', FAKE_ENGINE_MARK: mark }, encoding: 'utf8', timeout: 120000 });
const controlJson = (...argv) => { const r = control(...argv); let json = null; try { json = JSON.parse(r.stdout.slice(r.stdout.indexOf('{'))); } catch { /* not json */ } return { code: r.status, json, text: r.stdout + r.stderr }; };
const alive = pid => { try { process.kill(pid, 0); return true; } catch { return false; } };
// ------------------------------------------------------------------------------ the host
let host = null, port = 0, base = '', spyLog = null, shutdownToken = randomBytes(32).toString('hex');
async function startHost() {
port = await freePort(); base = `http://127.0.0.1:${port}`;
const hostArgs = [...(args.spy ? ['--import', pathToFileURL(resolve(args.spy)).href] : []), hostEntry];
if (args.spy) { spyLog = join(work, 'acquire-spy.log'); writeFileSync(spyLog, ''); }
host = spawn(node, hostArgs, {
cwd: join(install, 'host'), stdio: ['ignore', 'pipe', 'pipe'], windowsHide: true,
env: { ...baseEnv, STRATA_PORT: String(port), STRATA_DIST: join(install, 'host', 'dist'), NODE_ENV: 'production', STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1',
STRATA_NO_ACQUIRE: '1', STRATA_SHUTDOWN_TOKEN: shutdownToken, FAKE_ENGINE_MARK: mark, STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()),
...(spyLog ? { ACQUIRE_SPY_LOG: spyLog } : {}) },
});
let log = '';
host.stdout.on('data', c => { log += c; }); host.stderr.on('data', c => { log += c; });
host.hostLog = () => log;
for (let i = 0; i < 200; i++) { try { if ((await fetch(base + '/api/health')).ok) return true; } catch { /* not yet */ } await delay(150); }
return false;
}
/** The host started the way the Tauri shell starts it, by a stand-in shell (a node process) that can then be killed hard. */
async function startHostViaShell() {
port = await freePort(); base = `http://127.0.0.1:${port}`;
const spec = { command: node, args: [hostEntry], options: {
cwd: join(install, 'host'), stdio: 'ignore', windowsHide: true,
env: { ...baseEnv, STRATA_PORT: String(port), STRATA_DIST: join(install, 'host', 'dist'), NODE_ENV: 'production', STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1',
STRATA_NO_ACQUIRE: '1', STRATA_SHUTDOWN_TOKEN: shutdownToken, STRATA_EXIT_WITH_PARENT: '1', FAKE_ENGINE_MARK: mark,
STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()) } } };
const shell = spawn(node, ['-e', "const s=JSON.parse(process.env.HOST_SPEC);const c=require('node:child_process').spawn(s.command,s.args,s.options);console.log(c.pid);setInterval(()=>{},1000);"],
{ env: { ...process.env, HOST_SPEC: JSON.stringify(spec) }, stdio: ['ignore', 'pipe', 'inherit'], windowsHide: true });
const hostPid = await new Promise(r => shell.stdout.once('data', c => r(Number(String(c).trim()))));
for (let i = 0; i < 200; i++) { try { if ((await fetch(base + '/api/health')).ok) return { shell, hostPid }; } catch { /* not yet */ } await delay(150); }
return { shell, hostPid, failed: true };
}
const call = async (path, method = 'GET', body) => {
const response = await fetch(base + path, { method, headers: { 'content-type': 'application/json' }, body: body === undefined ? undefined : JSON.stringify(body) });
return { status: response.status, body: await response.json().catch(() => null) };
};
async function waitJob(id, seconds = 120) {
for (let i = 0; i < seconds * 5; i++) {
const state = (await call('/api/native')).body;
const job = state.jobs.find(j => j.id === id);
if (job && job.state !== 'running') return { job, state };
await delay(200);
}
throw new Error('operation did not finish');
}
async function stopHost(hard = false) {
if (!host || host.exitCode !== null) return;
if (hard) { if (isWin) spawnSync('taskkill', ['/PID', String(host.pid), '/T', '/F'], { windowsHide: true }); else host.kill('SIGKILL'); }
else host.kill();
for (let i = 0; i < 100 && host.exitCode === null; i++) await delay(100);
}
async function main() {
for (const need of [node, hostEntry, python, join(controlDir, 'strata.py')]) if (!existsSync(need)) { record('installed layout', false, `missing ${need}`); return; }
record('installed layout', true, install);
if (real) {
const packages = existsSync(join(dirs.components, 'engine')) ? readdirSync(join(dirs.components, 'engine')) : [];
record('fixtures', packages.length > 0 && existsSync(join(dirs.models, `strata-${modelId}.json`)), `engine package(s) ${packages.join(', ')} in ${dirs.components}; run config strata-${modelId}.json in ${dirs.models}`);
} else {
installFakeEngine();
makeTokenizerAndConfig();
record('fixtures', true, `stub engine package in ${dirs.components}; run config strata-stub.json`);
}
if (!(await startHost())) { record('host healthy', false, host.hostLog().slice(-600)); return; }
record('host healthy', true, base);
const status = (await call('/api/control/status')).body;
record('Control status: bundled, engine usable, acquiring forbidden', status.control?.python?.startsWith(controlDir) && status.engine?.usable === true && status.acquisition?.forbidden === true,
`python=${status.control?.python}; engine=${status.engine?.engine?.version}; forbidden=${status.acquisition?.forbidden}`);
const native = (await call('/api/native?refresh=1')).body;
const model = native.installed?.models?.find(m => m.model === modelId);
record('models come from Control', native.available && model?.ready === true && native.packaged === true, `available=${native.available}; model ready=${model?.ready}; ${native.summary ?? native.error}`);
// ---- load
const started = (await call('/api/native/operations', 'POST', { kind: 'load', args: { model: modelId } })).body;
const loaded = await waitJob(started.id, 400);
const instance = loaded.state.instances[0];
record('load through the host starts the engine under Control', loaded.job.state === 'done' && instance?.status === 'ready', `job=${loaded.job.state} ${loaded.job.error ?? ''}; instance=${instance?.status}`);
let ran = existsSync(mark) ? JSON.parse(readFileSync(mark, 'utf8')) : null;
if (real) {
const listed = spawnSync('ps', ['-eo', 'pid,args'], { encoding: 'utf8' }).stdout.split('\n').filter(l => l.includes(`${dirs.components}/engine/`) && l.includes('--serve'));
ran = listed.length ? { pid: Number(listed[0].trim().split(/\s+/)[0]), script: listed[0].trim().split(/\s+/)[1] } : null;
record('the engine process running is the binary from the component folder', !!ran, listed[0] ? listed[0].trim().slice(0, 160) : 'no engine process under the component folder');
} else {
record('the engine that ran is the one in the component folder', ran && ran.script.toLowerCase().startsWith(dirs.components.toLowerCase()), ran?.script ?? 'engine never ran');
}
const runtime = controlJson('runtime', 'status', '--profile', 'app-1');
record('a second client sees the same owner and instance (ownership)', runtime.json?.state === 'ready' && !!runtime.json.owner_id && !!runtime.json.instance_id && runtime.json.host_pid !== host.pid,
`state=${runtime.json?.state}; owner=${String(runtime.json?.owner_id).slice(0, 8)}; instance=${String(runtime.json?.instance_id).slice(0, 8)}; owner host pid ${runtime.json?.host_pid} is not the app host (${host.pid})`);
// ---- answer
const chat = await call('/v1/chat/completions', 'POST', real
? { model: instance?.apiModelId ?? modelId, messages: [{ role: 'user', content: 'Reply with exactly one word: pong' }], max_tokens: 200, temperature: 0, reasoning_effort: 'none', stream: false }
: { model: instance?.apiModelId ?? modelId, messages: [{ role: 'user', content: 'hi' }], max_tokens: 4, stream: false });
const message = chat.body?.choices?.[0]?.message ?? {};
const answer = `${message.content ?? ''}${message.reasoning_content ?? ''}`; // the stub emits token 97 ("a"); the server files it as reasoning
record(real ? 'the real model answers through the host (the answer text is checked)' : 'the engine answers through the host (the stub token, not an error)',
chat.status === 200 && (real ? /pong/i.test(answer) : answer === 'a'), `status=${chat.status}; answer=${JSON.stringify(answer.slice(0, 200))}${chat.status === 200 ? '' : ' ' + JSON.stringify(chat.body).slice(0, 160)}`);
// ---- unload
const unloaded = await call(`/api/native/instances/${instance.id}/unload`, 'POST', {});
await delay(500);
const after = controlJson('runtime', 'status', '--profile', 'app-1');
for (let i = 0; i < 100 && ran && alive(ran.pid); i++) await delay(100); // a real engine takes a moment to release the GPU
record('unload stops the engine', unloaded.status === 200 && after.json?.state === 'stopped' && (!ran || !alive(ran.pid)), `status=${unloaded.status}; runtime=${after.json?.state}; engine pid ${ran?.pid} alive=${ran ? alive(ran.pid) : 'n/a'}`);
// ---- no engine for this PC
const hidden = join(dirs.components, real ? 'engine.hidden-by-flow' : 'engine.gone');
renameSync(join(dirs.components, 'engine'), hidden);
rmSync(mark, { force: true });
const missing = await waitJob((await call('/api/native/operations', 'POST', { kind: 'load', args: { model: modelId } })).body.id);
record('no engine for this PC is a notice and nothing is started', missing.job.state === 'error' && /No engine for this PC/.test(missing.job.error) && /never downloads/.test(missing.job.error) && !existsSync(mark),
`${missing.job.state}: ${String(missing.job.error).slice(0, 160)}`);
const refused = await waitJob((await call('/api/native/operations', 'POST', { kind: 'install', args: { family: 'qwen', model: 'IQ2_XS' } })).body.id);
record('installing from the app is refused (Terminal only)', refused.job.state === 'error' && /Terminal/.test(refused.job.error), refused.job.error);
const binary = await call('/api/engine/binary', 'POST');
record('the llama.cpp download route is refused', binary.status === 409 && /never downloads/.test(binary.body?.error ?? ''), `${binary.status} ${binary.body?.error}`);
// ---- clean shutdown (what the Tauri shell does on exit)
renameSync(hidden, join(dirs.components, 'engine'));
rmSync(mark, { force: true });
const again = await waitJob((await call('/api/native/operations', 'POST', { kind: 'load', args: { model: modelId } })).body.id, 400);
let ran2 = existsSync(mark) ? JSON.parse(readFileSync(mark, 'utf8')) : null;
if (real) {
const listed = spawnSync('ps', ['-eo', 'pid,args'], { encoding: 'utf8' }).stdout.split('\n').filter(l => l.includes(`${dirs.components}/engine/`) && l.includes('--serve'));
ran2 = listed.length ? { pid: Number(listed[0].trim().split(/\s+/)[0]) } : null;
}
record('load again after the engine package returns', again.job.state === 'done', again.job.error ?? '');
const wrong = await fetch(base + '/api/control/shutdown', { method: 'POST', headers: { 'x-strata-shutdown': 'nope' } });
record('shutdown without the shell\'s secret is refused', wrong.status === 403, String(wrong.status));
const exited = new Promise(r => host.once('exit', code => r(code)));
const stopping = await fetch(base + '/api/control/shutdown', { method: 'POST', headers: { 'x-strata-shutdown': shutdownToken } });
const code = await Promise.race([exited, delay(90000).then(() => 'timeout')]);
for (let i = 0; i < 100 && ran2 && alive(ran2.pid); i++) await delay(100);
const final = controlJson('runtime', 'status', '--profile', 'app-1');
record('clean shutdown stops the engine the app started and exits', stopping.status === 202 && code === 0 && final.json?.state === 'stopped' && (!ran2 || !alive(ran2.pid)), `status=${stopping.status}; host exit=${code}; runtime=${final.json?.state}; engine alive=${ran2 ? alive(ran2.pid) : 'n/a'}`);
// ---- hard kill: an engine whose app host dies ends with it (the owner follows the host: runtime start --die-with-pid)
if (!(await startHost())) { record('host restarts', false, host.hostLog().slice(-400)); return; }
await call('/api/native?refresh=1');
rmSync(mark, { force: true });
const third = await waitJob((await call('/api/native/operations', 'POST', { kind: 'load', args: { model: modelId } })).body.id, 400);
let ran3 = existsSync(mark) ? JSON.parse(readFileSync(mark, 'utf8')) : null;
if (real) {
const listed = spawnSync('ps', ['-eo', 'pid,args'], { encoding: 'utf8' }).stdout.split('\n').filter(l => l.includes(`${dirs.components}/engine/`) && l.includes('--serve'));
ran3 = listed.length ? { pid: Number(listed[0].trim().split(/\s+/)[0]) } : null;
record('(real) the engine is up before the host is killed', !!ran3, listed[0] ? listed[0].trim().slice(0, 120) : 'no engine process');
}
await stopHost(true);
for (let i = 0; i < 600 && ran3 && alive(ran3.pid); i++) await delay(100); // the owner notices within a second, drains, stops
const orphan = controlJson('runtime', 'status', '--profile', 'app-1');
record('a hard-killed app host takes its engine with it', third.job.state === 'done' && (!ran3 || !alive(ran3.pid)), `engine alive=${ran3 ? alive(ran3.pid) : 'n/a'}; runtime=${orphan.json?.state ?? orphan.json?.error?.message}`);
const cleanup = controlJson('runtime', 'stop', '--profile', 'app-1');
record('nothing is left to stop afterwards', /stopped|not managed/.test(JSON.stringify(cleanup.json)) && controlJson('runtime', 'status', '--profile', 'app-1').json?.state === 'stopped', JSON.stringify(cleanup.json).slice(0, 160));
// ---- the shell itself is killed (what the desktop shell's crash or kill looks like to the host): the host leaves, and
// its engine with it, with nothing announced. Windows also has the job object; this is the path that works everywhere.
const started2 = await startHostViaShell();
if (started2.failed) { record('host starts under a stand-in shell', false, ''); try { started2.shell.kill('SIGKILL'); } catch { /* gone */ } }
else {
await call('/api/native?refresh=1');
rmSync(mark, { force: true });
const fourth = await waitJob((await call('/api/native/operations', 'POST', { kind: 'load', args: { model: modelId } })).body.id, 400);
let ran4 = existsSync(mark) ? JSON.parse(readFileSync(mark, 'utf8')) : null;
if (real) {
const listed = spawnSync('ps', ['-eo', 'pid,args'], { encoding: 'utf8' }).stdout.split('\n').filter(l => l.includes(`${dirs.components}/engine/`) && l.includes('--serve'));
ran4 = listed.length ? { pid: Number(listed[0].trim().split(/\s+/)[0]) } : null;
}
if (isWin) spawnSync('taskkill', ['/PID', String(started2.shell.pid), '/F'], { windowsHide: true }); else started2.shell.kill('SIGKILL');
for (let i = 0; i < 100 && alive(started2.hostPid); i++) await delay(100);
const hostGone = !alive(started2.hostPid);
for (let i = 0; i < 900 && ran4 && alive(ran4.pid); i++) await delay(100);
const engineGone = !ran4 || !alive(ran4.pid);
record('a killed shell takes the host and its engine with it', fourth.job.state === 'done' && hostGone && engineGone,
`host pid ${started2.hostPid} gone=${hostGone}; engine pid ${ran4?.pid} gone=${engineGone}; runtime=${controlJson('runtime', 'status', '--profile', 'app-1').json?.state}`);
controlJson('runtime', 'stop', '--profile', 'app-1');
}
await finishSpy();
}
async function finishSpy() {
if (spyLog) {
const attempts = readFileSync(spyLog, 'utf8').split('\n').filter(Boolean);
record('no acquisition attempt by the host during the whole run (spy)', attempts.length === 0, attempts.join(' | '));
}
}
try { await main(); }
catch (error) { record('flow', false, error.stack ?? error.message); }
finally {
try { const hid = join(dirs.components, 'engine.hidden-by-flow'); if (real && existsSync(hid) && !existsSync(join(dirs.components, 'engine'))) renameSync(hid, join(dirs.components, 'engine')); } catch { /* reported by the run */ }
await stopHost(true);
controlJson('runtime', 'stop', '--profile', 'app-1');
controlJson('runtime', 'stop', '--profile', 'app-2');
}
const failed = steps.filter(s => !s.ok);
writeFileSync(join(work, 'installed-flow.json'), JSON.stringify({ ok: failed.length === 0, install, work, steps }, null, 1));
console.log(`\n${steps.length - failed.length}/${steps.length} ok; report ${join(work, 'installed-flow.json')}`);
if (!args.keep && !args.work && failed.length === 0) { try { rmSync(work, { recursive: true, force: true }); } catch { /* leave it */ } }
process.exit(failed.length ? 1 : 0);
@@ -0,0 +1,73 @@
#!/usr/bin/env node
/**
* Mock-ups of the Windows installer in Strata's own look: four screens (welcome, where, installing, done), light and dark, drawn with
* the app's built CSS (apps/host/dist) and brand files, then photographed with Chrome. Nothing here is built or shipped: it is what
* the owner is shown before an installer is chosen.
*
* node apps/desktop/scripts/installer-mockups.mjs [--out <folder>]
*/
import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]]] : acc), []));
const here = dirname(fileURLToPath(import.meta.url));
const host = resolve(here, '../../host');
const out = resolve(args.out ?? join(here, '../installer-mockups'));
mkdirSync(out, { recursive: true });
const dist = join(host, 'dist');
const css = readdirSync(join(dist, 'assets')).filter(f => f.endsWith('.css')).map(f => readFileSync(join(dist, 'assets', f), 'utf8')).join('\n');
const logo = theme => readFileSync(join(dist, 'brand', `strata-lockup-${theme}.svg`), 'utf8');
const { chromium } = await import(pathToFileURL(join(host, 'node_modules/playwright-core/index.mjs')).href).catch(() => import('playwright-core'));
const CHROME = process.env.CHROME ?? ['C:/Program Files/Google/Chrome/Application/chrome.exe', '/usr/bin/google-chrome'].find(existsSync);
const check = '<svg class="st-icon" width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round" stroke-linejoin="round"><path d="M5 12.5l4.5 4.5L19 7.5"/></svg>';
const folder = '<svg class="st-icon" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="M3 7a2 2 0 0 1 2-2h4l2 2h8a2 2 0 0 1 2 2v8a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2z"/></svg>';
const screens = {
welcome: theme => `<header class="onboard-header"><span class="mock-logo">${logo(theme)}</span></header>
<div class="onboard-welcome-copy"><h1>State of the art. At home.</h1><p>Strata runs AI on your own computer.<br>Setting it up takes a few minutes.</p>
<div class="onboard-benefits"><span>${check}Private</span><span>${check}Fast</span><span>${check}Easy to use</span></div></div>
<footer class="onboard-footer onboard-footer--welcome"><button class="st-btn st-btn--primary">Install Strata</button><small>Installs for you only. No administrator needed.</small></footer>`,
where: theme => `<header class="onboard-header"><span class="mock-logo">${logo(theme)}</span></header>
<div class="mock-body"><h2 class="mock-title">Where should Strata go?</h2><p class="mock-sub">This is the right place for most people.</p>
<div class="onboard-folder">${folder}<div><strong>Your Strata folder</strong><small>C:\\Users\\Alex\\AppData\\Local\\Strata</small></div><button class="st-btn st-btn--sm">Change</button></div>
<p class="mock-note">About 310 MB. Your models and chats are kept somewhere else and are never removed by an update.</p></div>
<footer class="onboard-footer"><button class="st-btn st-btn--ghost">Back</button><span></span><button class="st-btn st-btn--primary">Install</button></footer>`,
installing: theme => `<header class="onboard-header"><span class="mock-logo">${logo(theme)}</span></header>
<div class="mock-body"><h2 class="mock-title">Installing Strata…</h2><p class="mock-sub">Copying the app and its engine room. You can keep working meanwhile.</p>
<div class="onboard-progress" style="--p:62%"><progress max="100" value="62"></progress></div>
<div class="onboard-progress-meta"><span>62% · 190 / 310 MB</span><div class="onboard-progress-actions"><span>About 1 min left</span><button>Cancel</button></div></div></div>
<footer class="onboard-footer"><span></span><span class="mock-hint">Next: choose your first model inside Strata.</span><button class="st-btn st-btn--primary" disabled>Installing…</button></footer>`,
done: theme => `<header class="onboard-header"><span class="mock-logo">${logo(theme)}</span></header>
<div class="onboard-welcome-copy"><h1>Strata is ready.</h1><p>Open it to pick your first model. Strata downloads it for you and gets it ready.</p></div>
<footer class="onboard-footer onboard-footer--welcome"><button class="st-btn st-btn--primary">Open Strata</button><small>A shortcut was added to your Start menu and desktop.</small></footer>`,
};
const page = (name, theme) => `<!doctype html><html data-theme="${theme}"><head><meta charset="utf-8"><style>${css}
body{background:${theme === 'dark' ? '#0b0d0e' : '#e9edef'};margin:0;display:grid;place-items:center;min-height:100vh}
.mock-window{width:720px;border-radius:14px;overflow:hidden;background:var(--st-surface,#fff);border:1px solid var(--st-line,#dde);box-shadow:0 30px 80px #0003}
.mock-titlebar{height:34px;display:flex;align-items:center;justify-content:space-between;padding:0 14px;font-size:12px;color:var(--st-ink-muted);background:var(--st-surface-2,#f3f5f6);border-bottom:1px solid var(--st-line-soft,#e6e9eb)}
.mock-logo svg{height:22px;width:auto;display:block}
.mock-body{padding:8px 36px 28px}.mock-title{font-size:28px;margin:16px 0 8px}.mock-sub{color:var(--st-ink-muted);margin:0 0 22px;line-height:1.6}
.mock-note{color:var(--st-ink-muted);font-size:12px;margin-top:16px;line-height:1.6}.mock-hint{font-size:12px;color:var(--st-ink-muted)}
.onboard-progress progress{width:100%}
.guided-onboarding{min-height:430px;display:flex;flex-direction:column}.onboard-footer{margin-top:auto}
</style></head><body><div class="mock-window"><div class="mock-titlebar"><span>Strata setup</span><span>– ☐ ✕</span></div>
<div class="setup guided-onboarding" data-step="${name === 'welcome' || name === 'done' ? 1 : 2}">${screens[name](theme)}</div></div></body></html>`;
const browser = await chromium.launch({ executablePath: CHROME, headless: true });
const written = [];
for (const theme of ['light', 'dark']) {
for (const name of Object.keys(screens)) {
const tab = await browser.newPage({ viewport: { width: 860, height: 620 }, deviceScaleFactor: 1 });
await tab.setContent(page(name, theme));
await tab.waitForTimeout(250);
const file = join(out, `${name}-${theme}.png`);
await tab.screenshot({ path: file });
written.push(file);
await tab.close();
}
}
await browser.close();
console.log(written.join('\n'));
+126
View File
@@ -0,0 +1,126 @@
#!/usr/bin/env node
/**
* "Use your own GGUF" end to end on a box, through the installed app layout (packaged host + bundled Control + a real engine component):
* preview a folder of a supported model, preview an unsupported GGUF (must be refused in plain terms), prepare the supported one
* (pack, from the folder, nothing copied), load it, get a chat answer, unload, shut down, and check nothing is left running.
*
* node local-gguf-e2e.mjs --root /data/ui4-e2e --good /data/141-gyro/models/IQ2_XS --bad /data/ui4-e2e/bad [--context 8192] [--backend cuda|metal]
* (the app folder is <root>/app: node/, host/, control/ ...; <root>/components gets the engine, from <root>/app/engine-bundle when the app carries one)
*/
import { spawn, spawnSync } from 'node:child_process';
import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
import { createServer } from 'node:net';
import { join, resolve } from 'node:path';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]]] : acc), []));
const root = resolve(args.root);
const good = args.good, bad = args.bad;
const context = Number(args.context ?? 8192);
const app = join(root, 'app');
const dirs = { components: join(root, 'components'), install: join(root, 'install'), profiles: join(root, 'profiles'), runtime: join(root, 'runtime'), home: join(root, 'home'), xdg: join(root, 'xdg'), appdata: join(root, 'appdata') };
for (const d of [dirs.install, dirs.home, dirs.xdg, dirs.appdata]) mkdirSync(d, { recursive: true });
const delay = ms => new Promise(r => setTimeout(r, ms));
const steps = [];
let failed = false;
const record = (name, ok, detail = '') => { steps.push({ name, ok: !!ok, detail: String(detail).slice(0, 400) }); console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ' ' + String(detail).slice(0, 300) : ''}`); if (!ok) failed = true; };
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
const env = { ...process.env, STRATA_COMPONENTS_DIR: dirs.components, STRATA_PROFILE_DIR: dirs.profiles, STRATA_RUNTIME_DIR: dirs.runtime, STRATA_INSTALL_DIR: dirs.install, STRATA_HOME: dirs.home, STRATA_MODELS_DIR: join(root, 'Strata-data'), STRATA_ENGINE_BACKEND: args.backend ?? 'cuda', XDG_CONFIG_HOME: dirs.xdg, APPDATA: dirs.appdata, PYTHONDONTWRITEBYTECODE: '1' };
delete env.STRATA_API_KEY;
const port = await freePort();
const base = `http://127.0.0.1:${port}`;
const host = spawn(join(app, 'node', 'node'), [join(app, 'host', 'server', 'index.mjs')], { cwd: join(app, 'host'), stdio: ['ignore', 'pipe', 'pipe'], detached: false,
env: { ...env, STRATA_PORT: String(port), STRATA_DIST: join(app, 'host', 'dist'), NODE_ENV: 'production', STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1', STRATA_NO_ACQUIRE: '1', STRATA_EXIT_WITH_PARENT: '1',
STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()), STRATA_WORKSPACE: join(root, 'workspace') } });
mkdirSync(join(root, 'workspace'), { recursive: true });
let hostLog = ''; host.stdout.on('data', c => { hostLog += c; }); host.stderr.on('data', c => { hostLog += c; });
const call = async (path, method = 'GET', body) => { const r = await fetch(base + path, { method, headers: { 'content-type': 'application/json' }, body: body === undefined ? undefined : JSON.stringify(body) }); return { status: r.status, body: await r.json().catch(() => null) }; };
async function waitJob(id, seconds) {
const until = Date.now() + seconds * 1000;
let last = '';
while (Date.now() < until) {
const state = (await call('/api/native')).body;
const job = state.jobs.find(j => j.id === id);
if (job && job.state !== 'running') return { job, state };
if (job && job.summary !== last) { last = job.summary; console.log(' ...', String(job.summary).slice(0, 140)); }
await delay(2000);
}
throw new Error('operation did not finish in time');
}
const engineProcs = () => spawnSync('ps', ['-eo', 'pid,args'], { encoding: 'utf8' }).stdout.split('\n').filter(l => l.includes(join(dirs.components, 'engine')) && /\/strata( |$)/.test(l));
async function main() {
let up = false;
for (let i = 0; i < 200 && !up; i++) { try { up = (await fetch(base + '/api/health')).ok; } catch { await delay(150); } }
record('host healthy', up, up ? base : hostLog.slice(-500));
if (!up) return;
let status = (await call('/api/control/status')).body;
for (let i = 0; i < 180 && status?.engineSetup; i++) { await delay(1000); status = (await call('/api/control/status')).body; } // the engine that came with the app is copied on the first start
record('Control status: bundled, engine usable', status?.engine?.usable === true, `engine=${status?.engine?.engine?.version} ${status?.engine?.notice ?? ''}`);
await call('/api/native?refresh=1');
// ---- the unsupported GGUF: a plain refusal
const badPreview = await call('/api/native/local/preview', 'POST', { path: bad });
const badIssues = badPreview.body?.preview?.issues ?? badPreview.body?.issues ?? [];
const badStatus = badPreview.body?.preview?.file_set_status;
console.log(' bad preview:', badPreview.status, JSON.stringify(badPreview.body).slice(0, 400));
record('an unsupported GGUF is refused, nothing adopted', badPreview.status === 200 ? (badStatus !== 'plausible' || !badPreview.body.preview.candidate_selection) : badPreview.status >= 400, `status=${badPreview.status}; file_set_status=${badStatus}; issues=${JSON.stringify(badIssues)}`);
// ---- the supported folder
const goodPreview = await call('/api/native/local/preview', 'POST', { path: good });
const selection = goodPreview.body?.preview?.candidate_selection;
console.log(' good preview:', JSON.stringify(goodPreview.body).slice(0, 500));
record('a folder of a supported model is recognised', goodPreview.status === 200 && goodPreview.body?.preview?.file_set_status === 'plausible' && !!selection, `family=${selection?.family}; model=${selection?.model}; files=${goodPreview.body?.preview?.file_count}`);
if (!selection || args.only === 'preview') return;
// ---- prepare (what the page's button does): install from the folder, nothing copied or downloaded
const native0 = (await call('/api/native?refresh=1')).body;
const catalogSize = native0.catalog?.families?.find(f => f.family === selection.family)?.sizes.find(s => s.model === selection.model);
record('the catalog lists the recognised version', !!catalogSize, catalogSize ? `id=${catalogSize.id}` : `no ${selection.family}/${selection.model} in the catalog`);
const started = await call('/api/native/operations', 'POST', { kind: 'install', args: { family: selection.family, model: selection.model, gguf_dir: good, context, vision: 'no' } });
record('prepare starts', started.status < 300 && !!started.body?.id, `status=${started.status} ${JSON.stringify(started.body).slice(0, 200)}`);
if (!started.body?.id) return;
const prepared = await waitJob(started.body.id, 3600);
record('prepare finishes', prepared.job.state === 'done', `${prepared.job.state}: ${prepared.job.error ?? prepared.job.summary}`);
if (prepared.job.state !== 'done') return;
const native1 = (await call('/api/native?refresh=1')).body;
const modelId = catalogSize?.id ?? `${selection.family === 'qwen' ? '' : selection.family + '-'}${String(selection.model).toLowerCase()}`;
const entry = native1.installed?.models?.find(m => m.model === modelId);
record('the model is ready and points at the folder, not a copy', entry?.ready === true, `model=${modelId}; ready=${entry?.ready}; dataFolder=${entry?.dataFolder}`);
if (!entry?.ready) return;
// ---- load
const load = await call('/api/native/operations', 'POST', { kind: 'load', args: { model: modelId } });
const loaded = await waitJob(load.body.id, 1800);
const instance = loaded.state.instances[0];
record('load starts the engine from the component folder', loaded.job.state === 'done' && instance?.status === 'ready', `job=${loaded.job.state} ${loaded.job.error ?? ''}; instance=${instance?.status}`);
if (instance?.status !== 'ready') return;
record('one engine process is running', engineProcs().length === 1, engineProcs().map(l => l.trim().slice(0, 140)).join(' | '));
// ---- a chat answer, checked
const chat = await fetch(base + '/api/chat/stream', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ sessionId: 'e2e-chat', message: 'Reply with exactly one word: pong', history: [], nativeModelId: instance.id, thinking: 'Off', agent: false, workspaceId: null }) });
let text = '', done = null;
for await (const chunk of chat.body) {
for (const line of Buffer.from(chunk).toString('utf8').split('\n')) {
if (!line.trim()) continue;
try { const f = JSON.parse(line); if (f.type === 'delta') text += f.text ?? ''; if (f.type === 'done') done = f; if (f.type === 'error') text += ` [error: ${f.message}]`; } catch { /* partial line */ }
}
}
record('the model answers through the app (the answer text is checked)', chat.status === 200 && /pong/i.test(text), `status=${chat.status}; answer=${JSON.stringify(text.slice(0, 120))}; tokensOut=${done?.tokensOut}; tps=${done?.tps}`);
// ---- unload, shutdown, nothing left
const unload = await call(`/api/native/instances/${instance.id}/unload`, 'POST');
await delay(3000);
record('unload stops the engine', unload.status < 300 && engineProcs().length === 0, `status=${unload.status}; engines left=${engineProcs().length}`);
}
try { await main(); } catch (e) { record('the flow', false, e?.stack ?? e); }
host.kill();
for (let i = 0; i < 100 && host.exitCode === null; i++) await delay(100);
if (host.exitCode === null) host.kill('SIGKILL');
await delay(2000);
const left = spawnSync('ps', ['-eo', 'pid,args'], { encoding: 'utf8' }).stdout.split('\n').filter(l => (l.includes(dirs.install) || l.includes(join(app, 'host'))) && !l.includes('ps -eo'));
record('nothing of this run is left running', left.length === 0, left.join(' | ').slice(0, 300));
writeFileSync(join(root, 'e2e-result.json'), JSON.stringify({ steps, hostLogTail: hostLog.slice(-2000) }, null, 1));
console.log(failed ? 'FAILED' : 'ALL OK');
process.exit(failed ? 1 : 0);
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env bash
# Build the Strata DMG without a Finder session.
#
# Tauri's default DMG step runs an AppleScript that asks Finder to arrange the window (icon
# positions, background). Finder AppleEvents time out in a non-GUI or sandboxed context — observed
# here as `Finder got an error: AppleEvent timed out. (-1712)` — so the default path cannot be
# relied on for an automated build. The upstream `create-dmg` script already supports exactly this
# case via `--skip-jenkins`, which produces a plain, correct DMG with the app and the Applications
# drop link and no cosmetic arrangement.
#
# Usage: scripts/make-dmg.sh [output.dmg]
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DESKTOP="$(cd "$HERE/.." && pwd)"
BUNDLE="$DESKTOP/src-tauri/target/release/bundle"
APP="$BUNDLE/macos/Strata.app"
VERSION="$(grep -m1 '^version = ' "$DESKTOP/src-tauri/Cargo.toml" | cut -d'"' -f2)" # the one place the app version is written
OUT="${1:-$BUNDLE/dmg/Strata_${VERSION}_aarch64.dmg}"
if [ ! -d "$APP" ]; then
echo "error: $APP does not exist; run 'cargo tauri build --bundles app' first" >&2
exit 1
fi
WORK="$(mktemp -d -t strata-dmg)"
trap 'rm -rf "$WORK"' EXIT
# A staging folder is what create-dmg copies into the image: the app, and nothing else.
#
# The Applications drop link is created by `--app-drop-link` below. Adding one here as well makes
# create-dmg fail with "File exists", because it links into the volume it is assembling.
mkdir -p "$WORK/stage"
cp -R "$APP" "$WORK/stage/"
# Reuse Tauri's vendored create-dmg if the bundler left it behind, otherwise take it from the
# bundle cache. Building it yourself is not necessary and not expected.
SCRIPT="$BUNDLE/dmg/bundle_dmg.sh"
if [ ! -x "$SCRIPT" ]; then
echo "error: $SCRIPT not found; run 'cargo tauri build --bundles dmg' once to vendor it" >&2
exit 1
fi
mkdir -p "$(dirname "$OUT")"
rm -f "$OUT"
# --skip-jenkins: no Finder AppleScript. Everything else is the same code path Tauri uses.
"$SCRIPT" \
--volname "Strata $VERSION" \
--volicon "$BUNDLE/macos/Strata.app/Contents/Resources/Strata.icns" \
--app-drop-link 400 200 \
--skip-jenkins \
--no-internet-enable \
"$OUT" "$WORK/stage"
echo
echo "built: $OUT"
ls -lh "$OUT" | awk '{print "size: " $5}'
shasum -a 256 "$OUT"
+78
View File
@@ -0,0 +1,78 @@
#!/usr/bin/env node
/**
* The app and installer icon, refined from the existing Strata mark (apps/host/public/brand/strata-mark-*.svg and the current
* icons/icon.png): the same three rounded bars on the same dark tile, in the same colours. Nothing is redrawn. What changes is
* how it is cut for small sizes: at 16-48 px the bars are snapped to whole pixels and made a little bolder and wider so they
* stay crisp instead of blurring; from 64 px up the geometry is the mark's own proportions.
*
* node apps/desktop/scripts/make-icons.mjs [--out <folder>] writes icon-<size>.png for every size and icon.ico
*/
import { existsSync, mkdirSync, writeFileSync } from 'node:fs';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]]] : acc), []));
const here = dirname(fileURLToPath(import.meta.url));
const host = resolve(here, '../../host');
const out = resolve(args.out ?? join(here, '../src-tauri/icons'));
mkdirSync(out, { recursive: true });
const { chromium } = await import(pathToFileURL(join(host, 'node_modules/playwright-core/index.mjs')).href).catch(() => import('playwright-core'));
const CHROME = process.env.CHROME ?? ['C:/Program Files/Google/Chrome/Application/chrome.exe', '/usr/bin/google-chrome'].find(existsSync);
const TILE = '#161a1d', LIGHT = '#eef1f3', GREEN = '#34d399'; // the colours of the current icon.png
/** Integer geometry for the small sizes: bars on whole pixels, centred. Returns [{x,y,w,h,fill}]. */
function hinted(size) {
const bar = Math.max(2, Math.round(size * 0.14)), gap = Math.max(2, Math.round(size * 0.1)), mid = bar + 1;
const total = 2 * bar + mid + 2 * gap, top = Math.floor((size - total) / 2);
const even = w => (w % 2 === size % 2 ? w : w + 1);
const wide = even(Math.round(size * 0.5)), long = even(Math.round(size * 0.76));
return [
{ x: (size - wide) / 2, y: top, w: wide, h: bar, fill: LIGHT },
{ x: (size - long) / 2, y: top + bar + gap, w: long, h: mid, fill: GREEN },
{ x: (size - wide) / 2, y: top + bar + gap + mid + gap, w: wide, h: bar, fill: LIGHT },
];
}
/** The mark's own proportions (from icon.png at 512): bars 34% / 50% / 34% wide, 9% high, on a tile with 21.5% corners. */
function proportional(size) {
const k = size / 512;
return [
{ x: 170 * k, y: 156 * k, w: 173 * k, h: 46 * k, fill: LIGHT },
{ x: 128 * k, y: 232 * k, w: 256 * k, h: 48 * k, fill: GREEN },
{ x: 170 * k, y: 309 * k, w: 173 * k, h: 46 * k, fill: LIGHT },
];
}
function svg(size) {
const small = size <= 48;
const bars = (small ? hinted(size) : proportional(size));
const radius = small ? Math.round(size * 0.22) : size * 0.215;
const rects = bars.map(b => `<rect x="${b.x}" y="${b.y}" width="${b.w}" height="${b.h}" rx="${small ? Math.min(b.h / 2, 1.5 + size / 32) : b.h / 2}" fill="${b.fill}"${small ? ' shape-rendering="crispEdges"' : ''}/>`).join('');
return `<svg xmlns="http://www.w3.org/2000/svg" width="${size}" height="${size}" viewBox="0 0 ${size} ${size}"><rect width="${size}" height="${size}" rx="${radius}" fill="${TILE}"/>${rects}</svg>`;
}
const SIZES = [16, 20, 24, 32, 40, 48, 64, 96, 128, 256, 512, 1024];
const browser = await chromium.launch({ executablePath: CHROME, headless: true });
const png = {};
for (const size of SIZES) {
const page = await browser.newPage({ viewport: { width: size, height: size }, deviceScaleFactor: 1 });
await page.setContent(`<!doctype html><body style="margin:0;background:transparent">${svg(size)}</body>`);
png[size] = await page.screenshot({ omitBackground: true, clip: { x: 0, y: 0, width: size, height: size } });
await page.close();
writeFileSync(join(out, `icon-${size}.png`), png[size]);
}
await browser.close();
// The .ico: one PNG image per size (16-256), the sizes Windows asks for at 100-250% scaling.
const ICO = [16, 20, 24, 32, 40, 48, 64, 96, 128, 256];
const head = Buffer.alloc(6); head.writeUInt16LE(0, 0); head.writeUInt16LE(1, 2); head.writeUInt16LE(ICO.length, 4);
let offset = 6 + 16 * ICO.length;
const entries = [], blobs = [];
for (const size of ICO) {
const data = png[size], e = Buffer.alloc(16);
e.writeUInt8(size === 256 ? 0 : size, 0); e.writeUInt8(size === 256 ? 0 : size, 1); e.writeUInt8(0, 2); e.writeUInt8(0, 3);
e.writeUInt16LE(1, 4); e.writeUInt16LE(32, 6); e.writeUInt32LE(data.length, 8); e.writeUInt32LE(offset, 12);
offset += data.length; entries.push(e); blobs.push(data);
}
writeFileSync(join(out, 'icon.ico'), Buffer.concat([head, ...entries, ...blobs]));
writeFileSync(join(out, 'icon.png'), png[512]);
console.log('icons in', out, ICO.join(','));
@@ -0,0 +1,66 @@
#!/usr/bin/env node
/**
* The installer's pictures, drawn from the app's own design: the light surface and ink colours of the app (src/styles tokens), the
* brand lockup and mark files, GT Walsheim Bold for the words (baked into the pictures, because NSIS cannot use a web font).
*
* node apps/desktop/scripts/make-installer-images.mjs [--out <folder>] [--preview <folder>]
*
* Writes sidebar.bmp (164x314, the welcome and finish pages) and header.bmp (150x57, the other pages) as 24-bit BMPs, and with
* --preview the same as PNGs at 3x.
*/
import { spawnSync } from 'node:child_process';
import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { tmpdir } from 'node:os';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]]] : acc), []));
const here = dirname(fileURLToPath(import.meta.url));
const host = resolve(here, '../../host');
const out = resolve(args.out ?? join(here, '../src-tauri/nsis'));
mkdirSync(out, { recursive: true });
const { chromium } = await import(pathToFileURL(join(host, 'node_modules/playwright-core/index.mjs')).href).catch(() => import('playwright-core'));
const CHROME = process.env.CHROME ?? ['C:/Program Files/Google/Chrome/Application/chrome.exe', '/usr/bin/google-chrome'].find(existsSync);
const font = pathToFileURL(join(host, 'public/fonts/GTWalsheimPro-Bold.woff2')).href;
const fontMedium = pathToFileURL(join(host, 'public/fonts/GTWalsheimPro-Medium.woff2')).href;
const lockup = readFileSync(join(host, 'public/brand/strata-lockup-light.svg'), 'utf8');
const mark = readFileSync(join(host, 'public/brand/strata-mark-light.svg'), 'utf8');
// the app's light tokens
const SURFACE = '#f9fafb', INK = '#171b1e', MUTED = '#5b646b', ACCENT = '#10b981', LINE = '#e3e8ea';
const css = `@font-face{font-family:GT;font-weight:700;src:url(${font})}@font-face{font-family:GT;font-weight:500;src:url(${fontMedium})}
*{box-sizing:border-box}body{margin:0;font-family:GT,'Segoe UI',sans-serif;color:${INK};background:${SURFACE}}`;
const pages = {
sidebar: { w: 164, h: 314, html: `<div style="width:164px;height:314px;padding:22px 18px;display:flex;flex-direction:column;background:linear-gradient(180deg,#f9fafb 0%,#eef6f2 100%);border-right:1px solid ${LINE}">
<div style="height:26px">${lockup.replace('<svg', '<svg style="height:26px;width:auto;display:block"')}</div>
<div style="margin-top:auto">
<div style="font-weight:700;font-size:22px;line-height:1.12;letter-spacing:-.01em">State of<br>the art.<br>At home.</div>
<div style="margin-top:10px;font-weight:500;font-size:11px;line-height:1.5;color:${MUTED}">AI on your own computer.<br>Private. Fast. Easy.</div>
<div style="margin-top:16px;height:3px;width:34px;border-radius:2px;background:${ACCENT}"></div>
</div></div>` },
header: { w: 150, h: 57, html: `<div style="width:150px;height:57px;display:flex;align-items:center;justify-content:flex-end;padding-right:12px;background:${SURFACE}">${lockup.replace('<svg', '<svg style="height:22px;width:auto;display:block"')}</div>` },
};
const browser = await chromium.launch({ executablePath: CHROME, headless: true });
const work = join(tmpdir(), 'strata-installer-images'); mkdirSync(work, { recursive: true });
for (const [name, page] of Object.entries(pages)) {
for (const scale of [1, 3]) {
const file = join(work, `${name}.html`);
writeFileSync(file, `<!doctype html><meta charset="utf-8"><style>${css}</style><body>${page.html}</body>`);
const tab = await browser.newPage({ viewport: { width: page.w, height: page.h }, deviceScaleFactor: scale });
await tab.goto(pathToFileURL(file).href);
await tab.evaluate(() => document.fonts.ready);
await tab.waitForTimeout(200);
const png = join(work, `${name}@${scale}.png`);
await tab.screenshot({ path: png, clip: { x: 0, y: 0, width: page.w, height: page.h } });
await tab.close();
if (scale === 3 && args.preview) { mkdirSync(resolve(args.preview), { recursive: true }); writeFileSync(join(resolve(args.preview), `${name}.png`), readFileSync(png)); }
}
}
await browser.close();
// 24-bit BMP, the format NSIS wants: render at 3x, shrink with a good filter
const py = `import sys\nfrom PIL import Image\nfor name in ('sidebar','header'):\n im = Image.open(sys.argv[1] + '/' + name + '@3.png').convert('RGB')\n w, h = im.width // 3, im.height // 3\n im.resize((w, h), Image.LANCZOS).save(sys.argv[2] + '/' + name + '.bmp', 'BMP')\n`;
const done = spawnSync('python', ['-c', py, work, out], { encoding: 'utf8' });
if (done.status !== 0) { console.error(done.stderr); process.exit(1); }
console.log('wrote', join(out, 'sidebar.bmp'), join(out, 'header.bmp'));
@@ -0,0 +1,85 @@
#!/usr/bin/env bash
# Write a tester's README with the checksums and sizes of the files it names, computed from the files themselves.
#
# apps/desktop/scripts/make-tester-readme.sh <template> <out.txt> --appimage <file name> --files-dir <dir> [--files-dir <dir> ...]
#
# Why: a README that carries a hand-pasted sha256 goes stale the moment the file is rebuilt (the Ubuntu 26.04 tester compared the file
# with the old build's hash). The template holds placeholders instead, and this script fills them from the real files:
# @APPIMAGE@ the AppImage's file name
# @APP_VERSION@ the app version, read from apps/desktop/src-tauri/Cargo.toml
# @SHA256:<name>@ sha256 of the file <name> found in one of the --files-dir folders
# @SIZE_GB:<name>@ its size in GB with one decimal
# @INCLUDE:<file>@ (alone on a line) the contents of <file> from the template's folder, with the same placeholders
# It refuses to write the README when a named file is missing or a placeholder is left over, so a stale or empty hash cannot ship.
# The same check is done by `--verify <readme> --files-dir <dir>`: every "<64 hex> <name>" or "sha256 <64 hex>" pair that names a file
# present in the folders must match that file.
set -euo pipefail
usage() { echo "usage: $0 <template> <out.txt> --appimage <name> --files-dir <dir> [--files-dir <dir> ...]" >&2; echo " $0 --verify <readme> --files-dir <dir> [...]" >&2; exit 2; }
VERIFY=""; TEMPLATE=""; OUT=""; APPIMAGE=""; DIRS=()
if [ "${1:-}" = "--verify" ]; then VERIFY="${2:?readme}"; shift 2; else TEMPLATE="${1:?template}"; OUT="${2:?out}"; shift 2 || usage; fi
while [ $# -gt 0 ]; do
case "$1" in
--appimage) APPIMAGE="$2"; shift 2 ;;
--files-dir) DIRS+=("$2"); shift 2 ;;
*) usage ;;
esac
done
[ ${#DIRS[@]} -gt 0 ] || usage
find_file() { local d; for d in "${DIRS[@]}"; do [ -f "$d/$1" ] && { echo "$d/$1"; return 0; }; done; return 1; }
sha_of() { sha256sum "$1" | awk '{print $1}'; }
size_gb() { awk -v b="$(stat -c %s "$1" 2>/dev/null || stat -f %z "$1")" 'BEGIN{printf "%.1f", b/1000000000}'; }
if [ -n "$VERIFY" ]; then
# every checksum in the README must be the checksum of a file that is here, and a line that names a file and carries a checksum must carry that file's checksum
hashes=""; bad=0; seen=0
for d in "${DIRS[@]}"; do for f in "$d"/*; do [ -f "$f" ] && grep -qF "$(basename "$f")" "$VERIFY" && hashes="$hashes $(sha_of "$f")"; done; done # only files the README names are hashed (a folder may hold big unrelated files)
readme_hashes="$(grep -Eo '[0-9a-f]{64}' "$VERIFY" | sort -u || true)"
for h in $readme_hashes; do
seen=$((seen + 1))
case " $hashes " in *" $h "*) ;; *) echo "STALE: the README lists $h, which is the checksum of no file in: ${DIRS[*]}" >&2; bad=$((bad + 1)) ;; esac
done
for d in "${DIRS[@]}"; do for f in "$d"/*; do
[ -f "$f" ] || continue
grep -qF "$(basename "$f")" "$VERIFY" || continue
# a line that names this file AND carries a checksum must carry this file's checksum (a sentence that merely mentions the file is fine)
line_hash="$(grep -F "$(basename "$f")" "$VERIFY" | grep -Eo '[0-9a-f]{64}' | head -1 || true)"
if [ -n "$line_hash" ] && [ "$line_hash" != "$(sha_of "$f")" ]; then
echo "STALE: $(basename "$f") is $(sha_of "$f"), the README says $line_hash" >&2; bad=$((bad + 1))
fi
done; done
echo "$seen checksum(s) in the README checked, $bad stale"
[ "$bad" = 0 ] && [ "$seen" -gt 0 ]
exit
fi
[ -n "$APPIMAGE" ] || usage
TDIR="$(cd "$(dirname "$TEMPLATE")" && pwd)"
# the app version has one source (the same file Tauri and the shell use)
APP_VERSION="$(grep -m1 '^version = ' "$TDIR/../src-tauri/Cargo.toml" | cut -d'"' -f2)"
[ -n "$APP_VERSION" ] || { echo "make-tester-readme: no version in $TDIR/../src-tauri/Cargo.toml" >&2; exit 1; }
TMP="$(mktemp)"; trap 'rm -f "$TMP" "$TMP.2"' EXIT
# includes first (one level), then the placeholders
while IFS= read -r line || [ -n "$line" ]; do
if [[ "$line" =~ ^@INCLUDE:([^@]+)@$ ]]; then tr -d '\r' < "$TDIR/${BASH_REMATCH[1]}"; else printf '%s\n' "$line"; fi
done < <(tr -d '\r' < "$TEMPLATE") | sed "s|@APPIMAGE@|$APPIMAGE|g; s|@APP_VERSION@|$APP_VERSION|g" > "$TMP"
# @SHA256:name@ and @SIZE_GB:name@
while IFS= read -r token; do
kind="${token#@}"; kind="${kind%%:*}"; name="${token#@*:}"; name="${name%@}"
f="$(find_file "$name")" || { echo "make-tester-readme: no file named $name in: ${DIRS[*]} (a README must not carry a hash it did not compute)" >&2; exit 1; }
case "$kind" in
SHA256) value="$(sha_of "$f")" ;;
SIZE_GB) value="$(size_gb "$f")" ;;
*) echo "make-tester-readme: unknown placeholder $token" >&2; exit 1 ;;
esac
# replace every occurrence of this exact token (plain text, no pattern characters)
text="$(cat "$TMP"; printf x)"; text="${text%x}"
printf '%s' "${text//"$token"/$value}" > "$TMP"
done < <(grep -Eo '@(SHA256|SIZE_GB):[^@ ]+@' "$TMP" | sort -u)
if grep -Eq '@[A-Z0-9_]+[:@]' "$TMP"; then echo "make-tester-readme: placeholders left over:" >&2; grep -En '@[A-Z0-9_]+[:@]' "$TMP" >&2; exit 1; fi
cp "$TMP" "$OUT"
echo "wrote $OUT"
+200
View File
@@ -0,0 +1,200 @@
#!/usr/bin/env node
/**
* Attached pictures reaching a model, end to end on a test box, through the installed app layout (packaged host + bundled Control + an engine
* component that carries strata-vision):
* 1. the model catalog marks the picture-capable models (images: true) when the picture reader is installed;
* 2. API models: a llama-server with a small picture model (--mmproj) is saved as a Custom model API; the host works out that it reads pictures,
* and a real picture goes out as image_url and is answered. A server without a picture reader, a model set "not to read pictures", and an API
* of another style each get a plain refusal (409) that says how to fix it; ticking "This model reads pictures" lets the picture through;
* 3. a local model (unless --skip-local): Download and prepare with Read pictures ON, load, and a picture through the app's own chat.
*
* node pics-e2e.mjs --install <node/ host/ control/> --components <engine components> --work <scratch> --image shapes.png \
* Also: --silent <url> (a server whose model list says nothing about pictures: ON by default), --agent (the same picture through the agent; needs a Harness)
* --vlm http://127.0.0.1:18090/v1 --vlm-model smolvlm --text-only http://127.0.0.1:18091/v1 [--skip-local --family qwen --model IQ3_XXS --data <folder> --device cpu]
*/
import { spawn } from 'node:child_process';
import { existsSync, mkdirSync, readFileSync, readdirSync } from 'node:fs';
import { createServer } from 'node:net';
import { join, resolve } from 'node:path';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]?.startsWith('--') || all[i + 1] === undefined ? true : all[i + 1]]] : acc), []));
for (const need of ['install', 'components', 'work', 'image']) if (!args[need]) { console.error('missing --' + need); process.exit(2); }
const install = resolve(args.install), work = resolve(args.work), components = resolve(args.components);
const want = String(args.match ?? args.model ?? 'IQ3_XXS').toLowerCase(), contextSize = Number(args.context ?? 32768), skipLocal = args['skip-local'] === true, device = String(args.device ?? 'cpu'), family = String(args.family ?? 'qwen'), modelId = String(args.model ?? 'IQ3_XXS');
const node = join(install, 'node', 'node'), hostEntry = join(install, 'host', 'server', 'index.mjs');
const dirs = { models: join(work, 'models'), profiles: join(work, 'profiles'), runtime: join(work, 'runtime'), home: join(work, 'home'), xdg: join(work, 'xdg'), appdata: join(work, 'appdata') };
for (const d of [dirs.models, dirs.home, dirs.xdg, dirs.appdata]) mkdirSync(d, { recursive: true });
const delay = ms => new Promise(r => setTimeout(r, ms));
let failed = false;
const record = (name, ok, detail = '') => { if (!ok) failed = true; console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ' ' + String(detail).slice(0, 600) : ''}`); };
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
const env = { ...process.env, STRATA_COMPONENTS_DIR: components, STRATA_PROFILE_DIR: dirs.profiles, STRATA_RUNTIME_DIR: dirs.runtime, STRATA_INSTALL_DIR: dirs.models,
STRATA_HOME: dirs.home, STRATA_MODELS_DIR: join(work, 'models-unused'), APPDATA: dirs.appdata, XDG_CONFIG_HOME: dirs.xdg, PYTHONDONTWRITEBYTECODE: '1' };
delete env.STRATA_API_KEY;
const port = await freePort();
const base = `http://127.0.0.1:${port}`;
const host = spawn(node, [hostEntry], { cwd: join(install, 'host'), stdio: ['ignore', 'pipe', 'pipe'],
env: { ...env, STRATA_PORT: String(port), STRATA_DIST: join(install, 'host', 'dist'), NODE_ENV: 'production', STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1', STRATA_NO_ACQUIRE: '1',
STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()) } });
let hostLog = '';
host.stdout.on('data', c => { hostLog += c; }); host.stderr.on('data', c => { hostLog += c; });
const call = async (path, method = 'GET', body) => {
const response = await fetch(base + path, { method, headers: { 'content-type': 'application/json' }, body: body === undefined ? undefined : JSON.stringify(body) });
return { status: response.status, body: await response.json().catch(() => null) };
};
const png = 'data:image/png;base64,' + readFileSync(resolve(args.image)).toString('base64');
const QUESTION = 'Name each shape in this picture with its colour, and read out any text.';
/** One chat turn through the app's own endpoint: the plain-text answer, or the refusal (status + message). */
async function chat(extra, message = QUESTION) {
const response = await fetch(base + '/api/chat/stream', { method: 'POST', headers: { 'content-type': 'application/json' },
body: JSON.stringify({ message, agent: false, images: [{ url: png, name: 'shapes.png' }], ...extra }) });
if (response.status !== 200) { const b = await response.json().catch(() => ({})); return { status: response.status, message: b.error ?? b.message ?? '' }; }
let text = '', error = '';
for (const line of (await response.text()).split('\n')) {
if (!line.trim()) continue;
let f; try { f = JSON.parse(line); } catch { continue; }
if (f.type === 'error') error = f.message ?? 'error';
else if (typeof f.text === 'string' && f.type === 'delta') text += f.text;
}
return { status: 200, text, error };
}
const saveApi = async (name, baseUrl, models, protocol = 'openai-chat') => {
const existing = (await call('/api/model-providers')).body.connections.find(c => c.name === name);
return (await call('/api/model-providers', 'POST', { ...(existing ? { id: existing.id } : {}), kind: 'custom', provider: null, name, baseUrl, protocol, apiKey: '', models })).body;
};
async function readsAfter(connectionName, id, want, seconds = 20) {
for (let i = 0; i < seconds * 2; i++) {
const c = (await call('/api/model-providers')).body.connections.find(x => x.name === connectionName);
const m = c?.models.find(x => x.id === id);
if (m && m.reads === want) return { c, m };
await delay(500);
}
const c = (await call('/api/model-providers')).body.connections.find(x => x.name === connectionName);
return { c, m: c?.models.find(x => x.id === id) };
}
async function waitJob(id, seconds) {
let last = '';
for (let i = 0; i < seconds * 2; i++) {
const state = (await call('/api/native')).body;
const job = state.jobs.find(j => j.id === id);
if (job && job.summary !== last) { last = job.summary; console.log(' ... ' + String(last).slice(0, 120)); }
if (job && job.state !== 'running') return { job, state };
await delay(500);
}
throw new Error('operation did not finish');
}
const names = low => /(red|blue|green|circle|square|triangle|strata|42)/.test(low);
async function main() {
let up = false;
for (let i = 0; i < 200 && !up; i++) { try { up = (await fetch(base + '/api/health')).ok; } catch { /* not yet */ } if (!up) await delay(150); }
record('host healthy', up, up ? base : hostLog.slice(-500));
if (!up) return;
// ---- 1. the catalog
const packages = existsSync(join(components, 'engine')) ? readdirSync(join(components, 'engine')) : [];
const reader = packages.some(p => existsSync(join(components, 'engine', p, 'strata-vision')));
const native = (await call('/api/native?refresh=1')).body;
record('the engine component carries the picture reader and the host knows it (visionReader)', reader && native.visionReader === true, `component=${reader} host=${native.visionReader}`);
const rows = (native.catalog?.families ?? []).flatMap(f => (f.sizes ?? []).map(s => ({ family: f.title ?? f.family, size: s.id, images: s.images, capable: s.imagesCapable })));
console.log(' catalog (family / size / images / imagesCapable):');
for (const r of rows) console.log(` ${String(r.family).padEnd(26)} ${String(r.size).padEnd(30)} images=${r.images} capable=${r.capable}`);
record('some models in the catalog read pictures', rows.some(r => r.images === true), `${rows.filter(r => r.images).length} of ${rows.length}`);
record('every picture-capable model reads pictures when the reader is installed', rows.every(r => r.capable === r.images), '');
const flash = rows.filter(r => /flash-next/i.test(`${r.family} ${r.size}`) && !/gyro|qad/i.test(`${r.family} ${r.size}`));
record('the Qwen3.8-Flash-Next models are marked images: true', flash.length > 0 && flash.every(r => r.images === true), flash.map(r => r.size + '=' + r.images).join(' '));
// ---- 2. API models
if (args.vlm) {
const smol = args['vlm-model'] ?? 'smolvlm';
await saveApi('Small picture model', args.vlm, [{ id: smol, name: 'SmolVLM' }]);
const found = await readsAfter('Small picture model', smol, true);
record('the host works out that the llama-server reads pictures (no tick needed)', found.m?.reads === true, JSON.stringify(found.m));
const conn = found.c;
const sent = await chat({ remoteModel: { connectionId: conn?.id, modelId: smol } });
console.log(` answer from the API model: ${JSON.stringify((sent.text ?? sent.message ?? '').slice(0, 300))}`);
record('the picture goes to the API model and is answered', sent.status === 200 && !sent.error && (sent.text ?? '').trim().length > 0 && names((sent.text ?? '').toLowerCase()), `status=${sent.status} err=${sent.error ?? ''}`);
if (args['text-only']) {
await saveApi('Text only server', args['text-only'], [{ id: 'plain', name: 'Plain model' }]);
const t = await readsAfter('Text only server', 'plain', false);
record('a server without a picture reader is known not to read pictures', t.m?.reads === false, JSON.stringify(t.m));
const refused = await chat({ remoteModel: { connectionId: t.c.id, modelId: 'plain' } });
console.log(` refusal: ${JSON.stringify(refused.message)}`);
record('a picture for it is refused in plain words that say how to fix it', refused.status === 409 && /Read pictures with this model/.test(refused.message ?? '') && /Settings/.test(refused.message ?? ''), `${refused.status}`);
const ticked = await saveApi('Text only server', args['text-only'], [{ id: 'plain', name: 'Plain model', vision: true }]);
const after = (await call('/api/model-providers')).body.connections.find(x => x.name === 'Text only server');
record('ticking "Read pictures with this model" is saved and makes the model read pictures', after?.models[0]?.vision === true && after?.models[0]?.reads === true, JSON.stringify(after?.models[0]));
const through = await chat({ remoteModel: { connectionId: after.id, modelId: 'plain' } });
record('with the tick the picture is sent on (the server itself decides what to do with it)', through.status === 200 && (through.error ?? '') !== '' ? /turn off/.test(through.error) : through.status === 200, `${through.status} ${through.message ?? through.error ?? ''}`);
}
await saveApi('Small picture model', args.vlm, [{ id: smol, name: 'SmolVLM', vision: false }]);
const off = await readsAfter('Small picture model', smol, false);
const notRead = await chat({ remoteModel: { connectionId: off.c.id, modelId: smol } });
record('a model switched OFF refuses pictures and says where to turn them on again', notRead.status === 409 && /not to read pictures/.test(notRead.message ?? '') && /Settings/.test(notRead.message ?? ''), `${notRead.status} ${notRead.message ?? ''}`);
await saveApi('Small picture model', args.vlm, [{ id: smol, name: 'SmolVLM', vision: null }]);
// unknown is ON by default: a server whose list and /props say nothing is sent the picture
if (args.silent) {
await saveApi('Silent server', args.silent, [{ id: 'quiet', name: 'Quiet model' }]);
const q = await readsAfter('Silent server', 'quiet', true, 8);
record('a server that says nothing about pictures is ON by default', q.m?.reads === true && q.m?.vision === null, JSON.stringify(q.m));
const sentQuiet = await chat({ remoteModel: { connectionId: q.c.id, modelId: 'quiet' } });
record('and the picture is sent to it (not refused)', sentQuiet.status === 200, `${sentQuiet.status} ${sentQuiet.message ?? sentQuiet.error ?? ''}`);
}
await saveApi('Other style', 'https://api.anthropic.com/v1', [{ id: 'claude-x', name: 'Claude' }], 'anthropic');
const other = (await call('/api/model-providers')).body.connections.find(x => x.name === 'Other style');
const style = await chat({ remoteModel: { connectionId: other.id, modelId: 'claude-x' } });
console.log(` other style: ${JSON.stringify(style.message)}`);
record('an API of another style says plainly why pictures stay in the preview', style.status === 409 && /OpenAI/.test(style.message ?? ''), `${style.status}`);
}
if (skipLocal) return;
// ---- 3. a local model
const values = { family, model: modelId, context: contextSize, vision: 'yes', loadVision: device, ...(args.data ? { data_dir: resolve(args.data) } : {}) };
const started = await call('/api/native/operations', 'POST', { kind: 'install', args: values });
record('the install operation starts', started.status === 200 || started.status === 202, JSON.stringify(started.body).slice(0, 200));
const done = await waitJob(started.body.id, 3600);
record('Download and prepare finishes', done.job.state === 'done', `${done.job.state} ${done.job.error ?? done.job.summary}`);
const refreshed = (await call('/api/native?refresh=1')).body;
const model = refreshed.installed?.models?.find(m => [m.id, m.model, m.tag].some(x => String(x ?? '').toLowerCase() === want));
record('the model is prepared with its picture reader', !!model?.visionPrepared, JSON.stringify({ ready: model?.ready, visionPrepared: model?.visionPrepared }));
const load = await call('/api/native/operations', 'POST', { kind: 'load', args: { model: model?.model ?? want } });
const loaded = await waitJob(load.body.id, 900);
const instance = loaded.state.instances[0];
record('the model loads with the picture reader', loaded.job.state === 'done' && instance?.status === 'ready' && instance?.vision === true, `${loaded.job.state} ${loaded.job.error ?? ''} vision=${instance?.vision}`);
record('the model loaded with pictures ON without being asked per load (default)', instance?.vision === true, '');
const t0 = Date.now();
let local = { status: 0 };
for (let i = 0; i < 2 && !(local.text ?? '').trim(); i++) local = await chat({ nativeModelId: instance?.id, maxTokens: 4096 });
console.log(` answer from the local model (${Math.round((Date.now() - t0) / 1000)} s): ${JSON.stringify((local.text ?? local.message ?? '').slice(0, 300))}`);
const low = (local.text ?? '').toLowerCase();
record('the local model answers about the picture (names the shapes)', local.status === 200 && /red|circle/.test(low) && /blue|square/.test(low) && /green|triangle/.test(low), `status=${local.status} err=${local.error ?? ''}`);
record('the local model reads STRATA 42', low.includes('42'));
// the same picture through the agent (the Harness route must declare image input: the "does not declare image input" error of the tester)
if (args.agent) { // needs a Harness beside the host (the installed layout has one; a bare test box does not)
const t1 = Date.now(); let a = { status: 0 };
for (let i = 0; i < 2 && !/red|circle|square|triangle/.test((a.text ?? '').toLowerCase()); i++) a = await chat({ nativeModelId: instance?.id, agent: true, sessionId: 'pics-agent-' + i, mode: 'plan', maxTokens: 4096 });
console.log(` answer through the agent (${Math.round((Date.now() - t1) / 1000)} s): ${JSON.stringify((a.text ?? a.message ?? '').slice(0, 300))}`);
record('the agent route reads the picture too', a.status === 200 && /red|circle|square|triangle/.test((a.text ?? '').toLowerCase()) && !/does not declare image input/.test(a.text ?? ''), `status=${a.status} err=${a.error ?? ''}`);
}
// OFF: loading with Read pictures off refuses a picture in plain words and says what to do
await call(`/api/native/instances/${instance?.id}/unload`, 'POST', {});
const loadOff = await call('/api/native/operations', 'POST', { kind: 'load', args: { model: model?.model ?? want, vision: 'no' } });
const loadedOff = await waitJob(loadOff.body.id, 900);
const off2 = loadedOff.state.instances[0];
record('loaded with Read pictures OFF', loadedOff.job.state === 'done' && off2?.vision === false, `${loadedOff.job.state} vision=${off2?.vision}`);
const refusedLocal = await chat({ nativeModelId: off2?.id, maxTokens: 64 });
console.log(` refusal: ${JSON.stringify(refusedLocal.message)}`);
record('a picture for the OFF model is refused with a next step', refusedLocal.status === 409 && /Read pictures/.test(refusedLocal.message ?? '') && /Models/.test(refusedLocal.message ?? ''), `${refusedLocal.status}`);
const unloaded = await call(`/api/native/instances/${off2?.id}/unload`, 'POST', {});
record('unload', unloaded.status === 200, unloaded.status);
}
try { await main(); } catch (e) { record('run', false, e.stack ?? e); }
host.kill();
await delay(1500);
console.log(failed ? 'PICS-E2E FAIL' : 'PICS-E2E PASS');
process.exit(failed ? 1 : 0);
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env bash
# Put the bundled payloads back to the bytes they were staged with, inside a finished AppImage.
#
# apps/desktop/scripts/repack-appimage.sh <in.AppImage> <out.AppImage> [--resources <src-tauri/resources>] [--check]
#
# Why: Tauri hands the AppDir to linuxdeploy, which treats every ELF file under usr/lib as a library to fix up: it rewrites the RPATH of
# the engine (strata, libstrata_kq.so, libcudart), of the bundled Python and Node and of the Harness's native modules. The files then no
# longer match the SHA-256 and size their manifests list (engine-component.json), so the host refuses the bundled engine, and the
# installer's own integrity promise ("what we staged is what runs") is broken. This script unpacks the AppImage, copies the staged
# folders (control, harness, node, engine-bundle) over the AppDir byte for byte, and packs it again with the same runtime.
#
# --check does not write anything: it lists every engine-bundle file whose bytes differ from the manifest in the AppImage.
set -euo pipefail
IN="$(realpath "${1:?in.AppImage}")"; OUT="${2:-}"
RES="$(cd "$(dirname "${BASH_SOURCE[0]}")/../src-tauri/resources" && pwd)"
CHECK=0
shift 2 || true
while [ $# -gt 0 ]; do
case "$1" in
--resources) RES="$(cd "$2" && pwd)"; shift 2 ;;
--check) CHECK=1; shift ;;
*) echo "unknown option $1" >&2; exit 2 ;;
esac
done
OFFSET="$("$IN" --appimage-offset)"
WORK="$(mktemp -d -t strata-repack.XXXXXX)"
trap 'rm -rf "$WORK"' EXIT
unsquashfs -no-progress -o "$OFFSET" -d "$WORK/AppDir" "$IN" >/dev/null
# the product folder under usr/lib (Tauri names it after productName)
LIB="$(find "$WORK/AppDir/usr/lib" -maxdepth 2 -type d -name engine-bundle -printf '%h\n' | head -1)"
[ -n "$LIB" ] || LIB="$(find "$WORK/AppDir/usr/lib" -maxdepth 2 -type d -name control -printf '%h\n' | head -1)"
[ -n "$LIB" ] || { echo "no Strata resource folder in the AppImage" >&2; exit 1; }
if [ "$CHECK" = 1 ]; then
bad=0
while IFS= read -r f; do
rel="${f#$RES/}"
if ! cmp -s "$f" "$LIB/$rel"; then echo "differs: $rel"; bad=$((bad + 1)); fi
done < <(find "$RES/engine-bundle" "$RES/node" -type f ! -name .gitignore)
echo "$bad file(s) differ from what was staged"
[ "$bad" = 0 ]
exit
fi
[ -n "$OUT" ] || { echo "give out.AppImage" >&2; exit 2; }
for d in control harness node engine-bundle; do
[ -d "$RES/$d" ] || continue
rsync -aL --checksum --delete --exclude .gitignore "$RES/$d/" "$LIB/$d/"
done
# The launcher records the environment the AppImage was started from, before its own hooks add the AppImage's libraries and GTK/GIO
# settings (LD_LIBRARY_PATH, GTK_*, XDG_DATA_DIRS ...). The host gives it back to every program that is not Strata's own (zenity,
# kdialog, xdg-open, the assistant's shell): apps/host/server/external-env.mjs. Without it a system program loads the AppImage's libssl
# and dies ("OPENSSL_3.2.0 not found"). The line goes right after `this_dir=...`, ahead of every hook.
if ! grep -q STRATA_ORIGINAL_ENV "$WORK/AppDir/AppRun"; then
sed -i '/^this_dir=/a # Strata: the environment this AppImage was started from (given back to programs that are not Strata'"'"'s own)\nexport STRATA_ORIGINAL_ENV="$(env -0 | base64 -w0)"' "$WORK/AppDir/AppRun"
grep -q STRATA_ORIGINAL_ENV "$WORK/AppDir/AppRun" || { echo "could not add the environment record to AppRun" >&2; exit 1; }
fi
mksquashfs "$WORK/AppDir" "$WORK/app.squashfs" -root-owned -noappend -comp zstd -b 131072 -no-progress >/dev/null
head -c "$OFFSET" "$IN" > "$OUT"
cat "$WORK/app.squashfs" >> "$OUT"
chmod +x "$OUT"
# The checksum is computed here, from the finished file, every time: a README or a note must take it from this file
# (apps/desktop/scripts/make-tester-readme.sh) and never from an older build.
( cd "$(dirname "$OUT")" && sha256sum "$(basename "$OUT")" > "$(basename "$OUT").sha256" )
echo "repacked: $OUT ($(stat -c %s "$OUT") bytes)"
cat "$OUT.sha256"
+202
View File
@@ -0,0 +1,202 @@
#!/usr/bin/env python3
"""Stage the Control component the installer ships: src-tauri/resources/control.
python apps/desktop/scripts/stage-control.py # build the terminal bundle, stage it
python apps/desktop/scripts/stage-control.py --archive X.zip # stage an archive you already built
python apps/desktop/scripts/stage-control.py --wheelhouse DIR # build with no network (wheels from DIR)
python apps/desktop/scripts/stage-control.py --check # verify what is staged
What lands in `control/`: the pinned CPython 3.12.15 (python-build-standalone, digest-checked), the hash-locked wheels
(`packaging/locks/runtime-<target>-cp312.txt`, installed with --require-hashes at build time), strata_control,
strata_cli, serve, tools/strata_tokenizer.py + gguf_reader.py, and BUNDLE.json (version, interpreter and lock digests,
the digest of the archive it came from). It is the same bundle as the Terminal archive (packaging/package_terminal.py):
one Python runtime-core for both products. The app then never runs pip, npm, git or a compiler to start or serve.
This script is a BUILD step. It may download the interpreter and the wheels (verified); the app never does.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import shutil
import subprocess
import sys
import tempfile
import zipfile
import tarfile
from pathlib import Path
HERE = Path(__file__).resolve().parent
ROOT = HERE.parents[2]
OUT = HERE.parent / "src-tauri" / "resources" / "control"
sys.path.insert(0, str(ROOT / "packaging"))
import package_terminal as pt # noqa: E402
def sha256(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1 << 20), b""):
digest.update(block)
return digest.hexdigest()
def build_archive(work: Path, wheelhouse: str | None) -> Path:
command = [sys.executable, str(ROOT / "packaging" / "package_terminal.py"), "--out", str(work)]
if wheelhouse:
command += ["--wheelhouse", wheelhouse]
subprocess.run(command, check=True, cwd=str(ROOT))
archives = sorted(work.glob("strata-terminal-*.zip")) + sorted(work.glob("strata-terminal-*.tar.gz"))
if len(archives) != 1:
raise SystemExit(f"expected one archive in {work}, found {archives}")
return archives[0]
def extract(archive: Path, destination: Path) -> Path:
"""Unpack the archive; returns the folder that holds python/ and strata.py."""
scratch = destination.parent / (destination.name + ".incoming")
shutil.rmtree(scratch, ignore_errors=True)
scratch.mkdir(parents=True)
if archive.name.endswith(".zip"):
with zipfile.ZipFile(archive) as zf:
for member in zf.namelist():
target = (scratch / member).resolve()
if scratch.resolve() not in target.parents and target != scratch.resolve():
raise SystemExit(f"unsafe member in {archive.name}: {member}")
zf.extractall(scratch)
else:
with tarfile.open(archive) as tf:
tf.extractall(scratch, filter="data")
(inner,) = [p for p in scratch.iterdir() if p.is_dir()]
return inner
#: What the app's installer needs on top of the Terminal bundle: the model catalog, GPU probe and install job. The list and the copy
#: live in packaging/package_terminal.py (the Terminal archive carries the same files, for `strata launch`).
INSTALLER_FILES, INSTALLER_TOOLS, INSTALLER_DATA = pt.INSTALLER_FILES, pt.INSTALLER_TOOLS, pt.INSTALLER_DATA
local_imports = pt.local_imports
def add_installer(folder: Path) -> dict:
"""Copy the installer's files from the source tree into the staged Control (after the Terminal bundle is in place)."""
return pt.add_installer(folder, ROOT)
#: What the other names of the interpreter become: Tauri copies a symlink as a full copy of its target, so `python`, `python3` and
#: `python3.12` in a bundle would be three real files (68 MB each on Linux). Only `python3.12` stays the real one.
#: The launcher runs inside the AppImage's environment (LD_LIBRARY_PATH holds the bundled libraries), where an external program such as
#: `dirname` can fail to start (Ubuntu 26.04's coreutils: "libsystemd.so.0: version LIBSYSTEMD_254 not found"), so it uses shell built-ins
#: only. The first version called `dirname`; that left the folder empty and "Control stopped" on such a computer.
LAUNCHER = """#!/bin/sh
# Strata: this name only starts the real interpreter beside it (the bundle keeps one copy, not one per name). Built-ins only: no dirname.
case "$0" in */*) dir=${0%%/*} ;; *) dir=. ;; esac
here=$(cd "$dir" && pwd)
exec "$here/%s" "$@"
"""
def dedupe_python(folder: Path) -> list[str]:
"""Make every other name of the interpreter in `python/bin` a small launcher of the real binary (Linux and macOS; Windows has one
python.exe). A symlink would not survive the bundling, and a plain copy is what costs the space. Returns the names changed."""
bindir = folder / "python" / "bin"
if not bindir.is_dir():
return []
real = bindir / f"python{pt.PYTHON_VERSION.rsplit('.', 1)[0]}"
if not real.exists():
return []
if real.is_symlink(): # the real file is a link itself: make the target the real one
target = real.resolve()
real.unlink()
shutil.move(str(target), str(real))
changed = []
for name in ("python", "python3"):
path = bindir / name
if not (path.is_symlink() or (path.is_file() and path.stat().st_size == real.stat().st_size)):
continue
path.unlink()
path.write_bytes((LAUNCHER % real.name).encode("utf-8"))
path.chmod(0o755)
changed.append(name)
return changed
def write_bundle_record(folder: Path, archive: Path) -> dict:
manifest_path = archive.with_name(archive.name.rsplit(".", 2 if archive.name.endswith(".tar.gz") else 1)[0] + ".manifest.json")
manifest = json.loads(manifest_path.read_text(encoding="utf-8")) if manifest_path.exists() else {}
target = manifest.get("target") or pt.host_target()
record = {
"schema_version": 1, "component": "strata-control",
"version": (folder / "VERSION").read_text(encoding="utf-8").strip(),
"target": target,
"python": {"version": pt.PYTHON_VERSION, "build": pt.PBS_RELEASE, "sha256": pt.PYTHON_PINS[target]["sha256"]},
"lock": manifest.get("lock"),
"installed": manifest.get("installed"),
"archive": {"name": archive.name, "sha256": sha256(archive), "bytes": archive.stat().st_size},
"acquires_at_runtime": False,
"installer": add_installer(folder),
}
(folder / "BUNDLE.json").write_text(json.dumps(record, indent=1) + "\n", encoding="utf-8")
return record
def check(folder: Path) -> int:
target = pt.host_target()
python = folder / pt.PYTHON_PINS[target]["python"]
problems = []
for need in (python, folder / "strata.py", folder / "strata_control", folder / "serve" / "server.py", folder / "BUNDLE.json",
folder / "setup.py", folder / "tools" / "strata_mcp.py", folder / "tools" / "iq_pack.py", folder / "data" / "draft_vocab.bin"):
if not need.exists():
problems.append(f"missing {need}")
if problems:
print("\n".join(problems), file=sys.stderr)
return 1
env = {"PYTHONNOUSERSITE": "1", "PYTHONDONTWRITEBYTECODE": "1", "STRATA_NO_ACQUIRE": "1", "SYSTEMROOT": __import__("os").environ.get("SYSTEMROOT", "")}
done = subprocess.run([str(python), str(folder / "strata.py"), "--version"], capture_output=True, text=True, env=env,
cwd=str(folder), timeout=60)
print(done.stdout.strip() or done.stderr.strip())
if done.returncode:
return done.returncode
# the installer: its catalog, a GPU probe and its imports work on the bundled Python alone (nothing is installed or fetched)
probe = ("import sys; sys.path.insert(0, %r); import importlib.util as u; s = u.spec_from_file_location('s', %r); m = u.module_from_spec(s); "
"s.loader.exec_module(m); import iq_pack, strata_pack, pack_index, mtp_fetch; "
"print('installer ok: %%d models, %%d families' %% (len(m.MODELS), len(m.FAMILIES)))") % (str(folder / "tools"), str(folder / "setup.py"))
env["STRATA_CONTROL_INSTALL"] = "1"
again = subprocess.run([str(python), "-c", probe], capture_output=True, text=True, env=env, cwd=str(folder), timeout=120)
print(again.stdout.strip() or again.stderr.strip()[-600:])
return again.returncode
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--archive", type=Path, help="stage this terminal archive instead of building one")
ap.add_argument("--wheelhouse", help="build with the wheels in this folder (no network)")
ap.add_argument("--out", type=Path, default=OUT)
ap.add_argument("--check", action="store_true", help="verify the staged component and exit")
a = ap.parse_args()
if a.check:
return check(a.out)
with tempfile.TemporaryDirectory(prefix="strata-stage-control-") as work:
archive = a.archive.resolve() if a.archive else build_archive(Path(work), a.wheelhouse)
keep = [p for p in a.out.iterdir() if p.name == ".gitignore"] if a.out.exists() else []
if a.out.exists():
for child in a.out.iterdir():
if child.name != ".gitignore":
shutil.rmtree(child, ignore_errors=True) if child.is_dir() else child.unlink()
a.out.mkdir(parents=True, exist_ok=True)
inner = extract(archive, a.out)
record = write_bundle_record(inner, archive)
for child in inner.iterdir():
shutil.move(str(child), str(a.out / child.name))
shutil.rmtree(inner.parent, ignore_errors=True)
launched = dedupe_python(a.out)
if launched:
print(f"python: {', '.join(launched)} are launchers of the one real interpreter")
print(f"staged {record['component']} {record['version']} ({record['target']}) into {a.out}")
return check(a.out)
if __name__ == "__main__":
raise SystemExit(main())
+176
View File
@@ -0,0 +1,176 @@
#!/usr/bin/env node
/**
* Stage the DeepSeek Harness (pinned, MIT) as the installer's `harness/` folder: a pruned PRODUCTION install, built at
* build time, so nothing is fetched or compiled on the person's PC.
*
* node apps/desktop/scripts/stage-harness.mjs --src <built Harness checkout> [--out apps/desktop/src-tauri/resources/harness]
*
* <built Harness checkout> is a clone of the pinned commit after `pnpm install --frozen-lockfile` and `pnpm run build:artifacts`
* (see phaseA-tests/notes-split-s4a.md, step 1). The script never touches it: it runs `pnpm deploy --legacy --prod` into
* the output folder, removes what a running Harness does not read (sources, tests, maps, docs, other operating systems'
* prebuilt binaries, the LibreOffice document converter and the speech engine, which the app does not use), keeps the
* MIT licence and third-party notices, writes BUNDLE.json, and finally boots the staged copy once as proof.
*
* The host finds the result at `<install>/harness` (server/agent/harness-runtime.mjs), or in the component folder.
*/
import { spawnSync } from 'node:child_process';
import { cpSync, existsSync, mkdirSync, readFileSync, readdirSync, realpathSync, rmSync, statSync, writeFileSync } from 'node:fs';
import { join, relative, resolve, sep } from 'node:path';
import { createHash } from 'node:crypto';
import { flattenPnpm } from './flatten-pnpm.mjs';
import { isOtherOsBinaryFolder, missingHarnessImports } from './harness-prune.mjs';
const args = process.argv.slice(2);
const opt = (name, fallback) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : fallback; };
const src = resolve(opt('src', process.env.STRATA_HARNESS_SRC ?? ''));
const out = resolve(opt('out', join(import.meta.dirname, '..', 'src-tauri', 'resources', 'harness')));
if (!opt('src', process.env.STRATA_HARNESS_SRC) || !existsSync(join(src, 'apps', 'cli', 'lib', 'bin.js'))) {
console.error('Give --src <a built Harness checkout> (apps/cli/lib/bin.js must exist).');
process.exit(2);
}
const platform = process.platform;
const run = (cmd, argv, cwd, extra = {}) => {
const r = spawnSync(cmd, argv, { cwd, stdio: 'inherit', shell: false, ...extra });
if (r.status !== 0) { console.error(`${cmd} ${argv.join(' ')} failed (${r.status ?? r.error})`); process.exit(1); }
};
// The output folder in the repo carries a .gitignore that keeps the staged files out of git: keep it across the clean-up.
const keepIgnore = existsSync(join(out, '.gitignore')) ? readFileSync(join(out, '.gitignore')) : null;
rmSync(out, { recursive: true, force: true });
mkdirSync(out, { recursive: true });
const pnpm = platform === 'win32' ? ['corepack.cmd', ['pnpm']] : ['corepack', ['pnpm']];
run(pnpm[0], [...pnpm[1], 'deploy', '--legacy', '--prod', '--filter', '@deepseek-ai/dsh', out], src, { shell: platform === 'win32' });
// --- peers -------------------------------------------------------------------------------------------------------------
// `pnpm deploy --prod` leaves out peer dependencies, but the Harness loads some of them by name at run time (for example the
// plugin-group helper that app-boot imports). Walk the installed packages; every dependency or peer that cannot be
// resolved from the output is copied from the source checkout's own node_modules, and its own requirements after it.
function packageJsonsUnder(dir, out2 = []) {
for (const e of readdirSync(dir, { withFileTypes: true })) {
const p = join(dir, e.name);
if (e.isSymbolicLink()) continue;
if (e.isDirectory()) packageJsonsUnder(p, out2);
else if (e.name === 'package.json' && p.includes(`${sep}node_modules${sep}`)) out2.push(p);
}
return out2;
}
function requirementsOf(file) {
try { const j = JSON.parse(readFileSync(file, 'utf8')); return [...Object.keys(j.dependencies ?? {}), ...Object.keys(j.peerDependencies ?? {}), ...Object.keys(j.optionalDependencies ?? {})]; } catch { return []; }
}
/** The checkout's own workspace packages (packages/**, vendor/*, apps/*) by name: where a peer that is one of ours comes from. */
const workspace = new Map();
function indexWorkspace(dir, depth = 0) {
if (depth > 4) return;
for (const e of readdirSync(dir, { withFileTypes: true })) {
if (!e.isDirectory() || e.isSymbolicLink() || e.name === 'node_modules' || e.name.startsWith('.')) continue;
const p = join(dir, e.name);
const manifest = join(p, 'package.json');
if (existsSync(manifest)) { try { workspace.set(JSON.parse(readFileSync(manifest, 'utf8')).name, p); } catch { /* not a package */ } }
indexWorkspace(p, depth + 1);
}
}
for (const top of ['packages', 'vendor', 'apps']) if (existsSync(join(src, top))) indexWorkspace(join(src, top));
const resolvable = name => existsSync(join(out, 'node_modules', ...name.split('/')));
const queue = [...new Set(packageJsonsUnder(join(out, 'node_modules')).flatMap(requirementsOf))].filter(n => !resolvable(n));
const added = [];
while (queue.length) {
const name = queue.shift();
if (resolvable(name)) continue;
const from = workspace.get(name) ?? join(src, 'node_modules', ...name.split('/'));
if (!existsSync(from)) continue; // an optional package for another platform, or not installed in the source: nothing to copy
const to = join(out, 'node_modules', ...name.split('/'));
mkdirSync(join(to, '..'), { recursive: true });
const realFrom = realpathSync(from);
cpSync(realFrom, to, { recursive: true, dereference: true, filter: f => !relative(realFrom, f).split(sep).includes('node_modules') });
added.push(name);
queue.push(...requirementsOf(join(to, 'package.json')).filter(n => !resolvable(n)));
}
if (added.length) console.log(`peer packages added: ${added.join(', ')}`);
// --- the browser tool server ---------------------------------------------------------------------------------------------
// Settings -> Tools can switch on a browser the assistant drives (Playwright's MCP server, Apache-2.0) on the person's own Chrome or Edge,
// so no browser is downloaded. The server and the two Playwright libraries it needs are not part of the Harness's production closure;
// they are copied from the source checkout's package store (~20 MB) and the host starts it with Node (server/agent/browser-probe.mjs).
const pnpmStore = join(src, 'node_modules', '.pnpm');
function storePackage(name, version) {
if (!existsSync(pnpmStore)) return null;
const prefix = `${name.replace('/', '+')}@`;
const hit = readdirSync(pnpmStore).filter(n => n.startsWith(prefix) && (!version || n === `${prefix}${version}` || n.startsWith(`${prefix}${version}_`))).sort().at(-1);
const dir = hit ? join(pnpmStore, hit, 'node_modules', ...name.split('/')) : null;
return dir && existsSync(dir) ? dir : null;
}
const browserDir = storePackage('@playwright/mcp');
if (browserDir) {
const wanted = JSON.parse(readFileSync(join(browserDir, 'package.json'), 'utf8')).dependencies ?? {};
const parts = [['@playwright/mcp', browserDir], ...['playwright', 'playwright-core'].map(n => [n, storePackage(n, wanted[n])])];
if (parts.every(([, dir]) => dir)) {
for (const [name, dir] of parts) {
const to = join(out, 'node_modules', ...name.split('/'));
mkdirSync(join(to, '..'), { recursive: true });
cpSync(realpathSync(dir), to, { recursive: true, dereference: true, filter: f => !relative(dir, f).split(sep).includes('node_modules') });
}
console.log(`browser tool server added: ${parts.map(([n]) => n).join(', ')}`);
} else console.log('browser tool server NOT added: Playwright libraries missing from the package store');
} else console.log('browser tool server NOT added: @playwright/mcp is not in the source checkout');
// --- one real folder per package, no links -------------------------------------------------------------------------------
// The installer copies files; pnpm's layout is links. See flatten-pnpm.mjs. The two packages the app does not use are never copied.
const flat = flattenPnpm(out, { skip: /libreoffice-kit|sherpa-onnx/i });
console.log(`flattened: ${flat.packages} placed (${flat.nested} nested for a second version)`);
// --- prune ------------------------------------------------------------------------------------------------------------
/** Whole packages the app never loads: office-document conversion (hundreds of MB) and speech input. */
const DROP_PACKAGES = [/libreoffice-kit/i, /sherpa-onnx/i];
/** Only when sitting directly in a package's root (a library may have a runtime folder called `doc`, e.g. yaml/dist/doc). */
const DROP_DIRS = new Set(['test', 'tests', '__tests__', 'docs', 'website', 'benchmarks', 'examples', '.github']);
const DROP_FILES = /\.(map|md|markdown|pdb|tsbuildinfo)$/i;
/** Licences and notices travel with the files they cover. */
const KEEP_FILES = /^(LICENSE|LICENCE|NOTICE|THIRD_PARTY)/i;
let removed = 0;
function prune(dir) {
for (const e of readdirSync(dir, { withFileTypes: true })) {
const p = join(dir, e.name);
if (e.isSymbolicLink()) continue;
if (e.isDirectory()) {
const inLib = p.split(sep).includes('lib');
if ((DROP_DIRS.has(e.name) && !inLib && existsSync(join(dir, 'package.json'))) || DROP_PACKAGES.some(r => r.test(e.name)) || isOtherOsBinaryFolder(platform, e.name, p)) {
rmSync(p, { recursive: true, force: true }); removed++; continue;
}
prune(p);
} else if (DROP_FILES.test(e.name) && !KEEP_FILES.test(e.name)) {
rmSync(p, { force: true }); removed++;
}
}
}
prune(out);
// --- licence: the MIT text and the third-party notices travel with the files ---------------------------------------------
for (const name of ['LICENSE', 'THIRD_PARTY_NOTICES.md']) {
if (existsSync(join(src, name))) cpSync(join(src, name), join(out, name));
}
const cli = JSON.parse(readFileSync(join(out, 'package.json'), 'utf8'));
let commit = process.env.STRATA_HARNESS_COMMIT ?? null;
if (!commit) { const r = spawnSync('git', ['rev-parse', 'HEAD'], { cwd: src, encoding: 'utf8' }); if (r.status === 0) commit = r.stdout.trim(); }
function size(dir) { let t = 0, n = 0; for (const e of readdirSync(dir, { withFileTypes: true })) { const p = join(dir, e.name); if (e.isDirectory()) { const s = size(p); t += s.bytes; n += s.files; } else if (e.isFile()) { t += statSync(p).size; n++; } } return { bytes: t, files: n }; }
const total = size(out);
writeFileSync(join(out, 'BUNDLE.json'), JSON.stringify({
component: 'deepseek-harness', version: cli.version, commit, license: 'MIT', platform, node: process.version,
bytes: total.bytes, files: total.files, entry: 'apps/cli/lib/bin.js', builtAt: new Date().toISOString(),
note: 'Pruned production install (pnpm deploy --prod). No package manager runs on the user\'s PC.',
}, null, 2));
if (keepIgnore) writeFileSync(join(out, '.gitignore'), keepIgnore);
// --- proof 1: nothing the Harness imports by name was pruned ------------------------------------------------------------
// (2026-10-17: the other-OS rule once removed @deepseek-ai/dsh-win32-process, which every platform imports; Linux/Mac agent chats died.)
const lost = missingHarnessImports(out);
if (lost.length) { console.error(['staged Harness is missing imported packages:', ...lost.map(l => ` ${l.from} -> ${l.missing}`)].join(String.fromCharCode(10))); process.exit(1); }
console.log('import check: every @deepseek-ai package the Harness imports is staged');
// --- proof 2: the staged copy answers --version and boots the web profile once ----------------------------------------
const bin = join(out, 'apps', 'cli', 'lib', 'bin.js');
const entry = existsSync(bin) ? bin : join(out, 'lib', 'bin.js');
const v = spawnSync(process.execPath, [entry, '--version'], { encoding: 'utf8' });
if (v.status !== 0) { console.error('staged Harness does not start:', v.stderr); process.exit(1); }
console.log(`staged ${cli.version} (${commit?.slice(0, 8) ?? 'no commit'}) in ${out}: ${(total.bytes / 1048576).toFixed(0)} MiB, ${total.files} files, ${removed} pruned items; --version -> ${v.stdout.trim()}`);
console.log(`sha256(BUNDLE.json)=${createHash('sha256').update(readFileSync(join(out, 'BUNDLE.json'))).digest('hex')}`);
+721
View File
@@ -0,0 +1,721 @@
#!/usr/bin/env node
/**
* Headless UI flows of the installed app against a packaged host with a FAKE installer and a FAKE engine (nothing is
* downloaded, no real engine runs; the real Control starts the fake engine through the owner runtime):
*
* node apps/desktop/scripts/ui-flows.mjs [--control <staged control>] [--node <node.exe>] [--shots <dir>] [--only download,models,tune,fail,theme]
*
* download the onboarding download: ONE combined bar over both files, an ETA in minutes left of Stop, Finish later closes the
* pop-up with the download still running, the Models screen shows the same progress, Stop stops it
* models an installed model is the one Models selects and loads, without a download pop-up
* tune the tuning step offers no picture reading, tunes, writes a RAM budget for "leave 20%", opens chat; reading size
* fail an engine that stops while loading: the error is in words and the pop-up does not scroll
* theme the app follows the system light/dark setting
* Per-user folders are all under a scratch folder. Exit 1 if a check fails.
*/
import { spawn, spawnSync } from 'node:child_process';
import { createHash } from 'node:crypto';
import { copyFileSync, cpSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, rmdirSync, symlinkSync, writeFileSync } from 'node:fs';
import { createServer } from 'node:net';
import { tmpdir } from 'node:os';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]?.startsWith('--') ? true : all[i + 1]]] : acc), []));
const here = dirname(fileURLToPath(import.meta.url));
const repoHost = resolve(here, '../../host');
const staged = resolve(here, '../src-tauri/resources');
const stagedControl = resolve(args.control ?? join(staged, 'control'));
const nodeExe = resolve(args.node ?? join(staged, 'node', process.platform === 'win32' ? 'node.exe' : 'node'));
const work = mkdtempSync(join(tmpdir(), 'strata-ui-flows-'));
const shots = resolve(args.shots ?? join(work, 'shots'));
mkdirSync(shots, { recursive: true });
const only = new Set(String(args.only ?? 'download,models,profile,ui2,ui2d,tune,fail,delete,theme,restart,perf,polish,stop,latest').split(','));
const isWin = process.platform === 'win32';
const { chromium } = await import(pathToFileURL(join(repoHost, 'node_modules/playwright-core/index.mjs')).href).catch(() => import('playwright-core'));
const CHROME = process.env.CHROME ?? ['C:/Program Files/Google/Chrome/Application/chrome.exe', '/usr/bin/google-chrome'].find(existsSync);
const delay = ms => new Promise(r => setTimeout(r, ms));
const sha = buf => createHash('sha256').update(buf).digest('hex');
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
let failed = false;
const check = (ok, label, detail = '') => { console.log(`${ok ? 'ok ' : 'FAIL'} ${label}${detail ? ' ' + detail : ''}`); if (!ok) failed = true; };
// ------------------------------------------------------------------ the app folder: the real Control with a fake installer
const control = join(work, 'control');
mkdirSync(control, { recursive: true });
for (const name of ['python', 'strata_control', 'strata_cli', 'serve']) symlinkSync(join(stagedControl, name), join(control, name), 'junction');
for (const name of ['strata.py', 'VERSION', 'BUNDLE.json', 'requirements.txt']) copyFileSync(join(stagedControl, name), join(control, name));
cpSync(join(stagedControl, 'tools'), join(control, 'tools'), { recursive: true });
const python = isWin ? join(control, 'python', 'python.exe') : join(control, 'python', 'bin', 'python3');
const FAKE_INSTALLER = String.raw`
import json, os, sys, time
from pathlib import Path
ROOT = Path(sys.argv[sys.argv.index('--root') + 1]) if '--root' in sys.argv else Path('.')
STATE = Path(os.environ['FAKE_STATE']); STATE.mkdir(parents=True, exist_ok=True)
RATE = float(os.environ.get('FAKE_RATE_GB_S', '0.5'))
def size(model, gb, ram, mid):
return {'model': model, 'about': model, 'download_gb': gb, 'ram_needed_gb': ram, 'experts_gb': ram - 8, 'on_this_pc': 'fits', 'installed': False, 'images': True, 'experimental': False, 'id': mid}
CATALOG = {'summary': 'fake', 'families': [{'family': 'qwen', 'title': 'Qwen3.8-Flash-Next', 'about': 'x', 'experimental': False, 'images': True, 'sizes': [
size('Q2_0', 66.4, 48, 'q2_0'), size('IQ3_XXS', 75.8, 60, 'iq3_xxs'), size('IQ3_S', 83.6, 62, 'iq3_s')]}],
'contexts': [8192, 32768, 65536], 'recommendation': {'family': 'qwen', 'model': 'IQ3_XXS', 'title': 'Qwen3.8-Flash-Next IQ3_XXS', 'context': 32768}, 'context_note': ''}
HARDWARE = {'ram_gb': 63.1, 'cpu': {'name': 'cpu', 'cores': 12}, 'gpus': [{'index': 0, 'name': 'NVIDIA GeForce RTX 5070', 'vram_gb': 11.9, 'usable': True}]}
TOTAL = 83.6
def job():
f = STATE / 'job.json'
if not f.exists(): return None
j = json.loads(f.read_text())
if j.get('cancelled'):
return {'running': False, 'exit_code': 1, 'error': 'stopped', 'result': 'failed'}
done = min(TOTAL, RATE * (time.time() - j['start']))
return {'running': True, 'step': '5/7: downloading Qwen3.8-Flash-Next IQ3_S', 'model': 'IQ3_S',
'download': {'file': 'model files', 'done_gb': round(done, 3), 'total_gb': TOTAL, 'remaining_gb': round(TOTAL - done, 3), 'percent': int(100 * done / TOTAL)}}
def call(name, a):
if name == 'strata_models': return CATALOG
if name == 'strata_remove':
cfg = ROOT / ('strata-%s.json' % a['model'])
if a.get('confirm') and cfg.exists(): cfg.unlink()
return {'summary': 'Deleted', 'plan': {'gb': 83.6}, 'failed': []}
if name == 'strata_status': return {'summary': 'fake', 'installed': {'models': [{'model': 'iq3_s', 'disk_gb': 83.6, 'disk_bytes': 83600000000}]}, 'hardware': HARDWARE, 'recommendation': CATALOG['recommendation'], 'install': job(), 'running': []}
if name == 'strata_install':
if a.get('cancel'):
f = STATE / 'job.json'
if f.exists(): j = json.loads(f.read_text()); j['cancelled'] = True; f.write_text(json.dumps(j))
return {'summary': 'stopped', 'install': job()}
if a.get('confirm'):
(STATE / 'job.json').write_text(json.dumps({'start': time.time()}))
return {'summary': 'started', 'plan': {'download_gb': TOTAL}, 'install': job()}
if not a: return {'summary': 'progress', 'install': job()}
return {'summary': 'Plan', 'plan': {'title': 'Qwen3.8-Flash-Next IQ3_S', 'download_gb': TOTAL, 'disk_short': None}}
raise Exception('unknown tool ' + name)
for line in sys.stdin:
line = line.strip()
if not line: continue
m = json.loads(line)
if 'id' not in m: continue
method = m.get('method')
if method == 'initialize': res = {'protocolVersion': '2024-11-05'}
elif method == 'tools/list': res = {'tools': [{'name': n} for n in ('strata_models', 'strata_status', 'strata_install', 'strata_remove')]}
else:
p = m['params']
res = {'structuredContent': call(p['name'], p.get('arguments') or {})}
sys.stdout.write(json.dumps({'jsonrpc': '2.0', 'id': m['id'], 'result': res}) + '\n'); sys.stdout.flush()
`;
writeFileSync(join(control, 'tools', 'strata_mcp.py'), FAKE_INSTALLER);
const FAKE_ENGINE = `import os, sys, json, time
from pathlib import Path
if os.environ.get('FAKE_ENGINE_FAIL'):
print('memory guard: 5.1 GiB of RAM free is under the 6 GiB limit (limit 63.1 GiB)', flush=True)
sys.exit(3)
ctx = '4096'
if '--max-context' in sys.argv: ctx = sys.argv[sys.argv.index('--max-context') + 1]
print('READY %s stop' % ctx, flush=True)
for line in sys.stdin:
if line.strip() == 'QUIT': break
if line.startswith(('GEN ', 'BGEN ')):
slot = int(line.split()[1]) if line.startswith('BGEN ') else None
t0 = time.time(); n = 0
for ch in os.environ.get('FAKE_REPLY', 'a'):
print('T %d' % ord(ch), flush=True); n += 1
if os.environ.get('FAKE_TOKEN_DELAY'): time.sleep(float(os.environ['FAKE_TOKEN_DELAY']))
print('DONE %d 12 300.0 %.1f length 0 0 0' % (n, max(1.0, (time.time() - t0) * 1000)), flush=True)
if slot is not None: print(f'BADM {slot} 0', flush=True)
`;
// one scenario = one fresh host with its own per-user folders
async function scenario(name, { installed = false, env = {} } = {}, body) {
if (!only.has(name)) return;
console.log(`--- ${name}`);
const dir = join(work, name);
const dirs = { home: join(dir, 'home'), appdata: join(dir, 'appdata'), local: join(dir, 'local'), components: join(dir, 'local', 'Strata', 'components'), install: join(dir, 'local', 'Strata', 'install'), state: join(dir, 'fake-state'), profiles: join(dir, 'profiles'), runtime: join(dir, 'runtime') };
for (const d of [dirs.home, dirs.appdata, dirs.local, dirs.components, dirs.install, dirs.state, join(dir, 'models')]) mkdirSync(d, { recursive: true });
const baseEnv = { ...process.env, STRATA_COMPONENTS_DIR: dirs.components, STRATA_PROFILE_DIR: dirs.profiles, STRATA_RUNTIME_DIR: dirs.runtime, STRATA_HOME: dirs.home, STRATA_MODELS_DIR: join(dir, 'models'), APPDATA: dirs.appdata, LOCALAPPDATA: dirs.local, USERPROFILE: dirs.home, XDG_CONFIG_HOME: join(dir, 'xdg'), XDG_DATA_HOME: join(dir, 'xdgdata'), PYTHONDONTWRITEBYTECODE: '1', STRATA_ENGINE_BACKEND: 'cuda', FAKE_STATE: dirs.state, ...env };
delete baseEnv.STRATA_API_KEY; delete baseEnv.STRATA_INSTALL_DIR;
if (installed) {
const platform = isWin ? 'windows-x86_64' : 'linux-x86_64';
const folder = join(dirs.components, 'engine', `0.1.45-${platform}-cuda`);
mkdirSync(folder, { recursive: true });
writeFileSync(join(folder, 'engine_main.py'), FAKE_ENGINE);
const entry = isWin ? 'engine.cmd' : 'engine';
writeFileSync(join(folder, entry), isWin ? `@echo off\r\n"${python}" "%~dp0engine_main.py" %*\r\n` : `#!/bin/sh\nexec "${python}" "$(dirname "$0")/engine_main.py" "$@"\n`);
const files = [entry, 'engine_main.py'].map(p => { const b = readFileSync(join(folder, p)); return { path: p, sha256: sha(b), bytes: b.length }; });
writeFileSync(join(folder, 'engine-component.json'), JSON.stringify({ schema_version: 1, id: 'strata-engine', version: '0.1.45', platform, backend: 'cuda', entrypoint: entry, files }));
const tok = join(dirs.install, 'tokenizer');
mkdirSync(tok, { recursive: true });
const script = `import json, sys\nfrom tools.strata_tokenizer import BYTE_TO_UNICODE\ntok = sys.argv[1]\ntokens = [BYTE_TO_UNICODE[i] for i in range(256)] + ['ab', '<|im_start|>', '<|im_end|>', '<|endoftext|>']\njson.dump({t: i for i, t in enumerate(tokens)}, open(tok + '/vocab.json', 'w'))\nopen(tok + '/merges.txt', 'w').write('a b')\njson.dump([0] * 257 + [3, 3, 3], open(tok + '/token_type.json', 'w'))\n`;
const made = spawnSync(python, ['-c', script, tok], { cwd: control, env: { ...baseEnv, PYTHONNOUSERSITE: '1' }, encoding: 'utf8' });
if (made.status !== 0) throw new Error('tokenizer fixture: ' + made.stderr);
writeFileSync(join(dirs.install, 'strata-iq3_s.json'), JSON.stringify({ exe: 'component:strata-engine', backend: 'cuda', tokenizer: tok, args: ['--max-context', '32768', '--pool-workers', '12'], cwd: dir, model_name: 'stub', host: '127.0.0.1', port: 0, kv: 'int8' }));
}
const port = env.FIXED_PORT ? Number(env.FIXED_PORT) : await freePort(), base = `http://127.0.0.1:${port}`;
const host = spawn(nodeExe, [join(repoHost, 'server', 'index.mjs')], { cwd: repoHost, stdio: ['ignore', 'pipe', 'pipe'], windowsHide: true,
env: { ...baseEnv, STRATA_PORT: String(port), STRATA_DIST: join(repoHost, 'dist'), STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1', STRATA_NO_ACQUIRE: '1', STRATA_CONTROL_DIR: control, STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()) } });
let log = ''; host.stdout.on('data', c => { log += c; }); host.stderr.on('data', c => { log += c; });
const api = async (path, init) => { const r = await fetch(base + path, init); return { status: r.status, body: await r.json().catch(() => null) }; };
const browser = await chromium.launch({ executablePath: CHROME, headless: true });
try {
let up = false;
for (let i = 0; i < 200 && !up; i++) { try { up = (await api('/api/health')).status === 200; } catch { await delay(150); } }
check(up, `${name}: the packaged host starts`);
await api('/api/native?refresh=1');
await body({ base, api, browser, dirs, shot: (page, n) => page.screenshot({ path: join(shots, `${name}-${n}.png`) }), log: () => log });
} catch (error) {
check(false, `${name}: the flow`, String(error?.stack ?? error).slice(0, 700) + '\n' + log.slice(-600));
} finally {
await browser.close().catch(() => {});
host.kill(); await new Promise(r => host.once('exit', r));
}
}
const newPage = async (browser, base, options = {}) => {
const context = await browser.newContext({ viewport: { width: 1400, height: 950 }, ...(process.env.SCHEME ? { colorScheme: process.env.SCHEME } : {}), ...options });
const page = await context.newPage();
const problems = [];
page.on('pageerror', e => problems.push(e.message));
page.problems = problems;
await page.goto(base, { waitUntil: 'domcontentloaded' });
return page;
};
const clickText = async (page, re, timeout = 30000) => { const b = page.getByRole('button', { name: re }).first(); await b.waitFor({ timeout }); await b.click(); };
const noScroll = async page => page.evaluate(() => {
const root = document.querySelector('.guided-onboarding');
const bad = [];
for (let el = document.querySelector('.onboard-tune-stage, .onboard-prepare-stage, .onboard-model-stage') ?? root; el && el !== document.body; el = el.parentElement) {
const style = getComputedStyle(el);
if (/(auto|scroll)/.test(style.overflowY) && el.scrollHeight > el.clientHeight + 1) bad.push(`${el.className || el.tagName}: ${el.scrollHeight} > ${el.clientHeight}`);
}
return bad;
});
// ------------------------------------------------------------------------------------------------ download
await scenario('download', {}, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /get started/i);
await page.getByText(/Choose a version/i).waitFor({ timeout: 30000 });
await page.locator('[role=radio]').filter({ hasText: 'IQ3_S' }).first().click();
await clickText(page, /^Continue/);
await clickText(page, /Download model/i, 60000);
await page.locator('.onboard-progress progress').waitFor({ timeout: 30000 });
// ONE bar: sample the percent while it runs, it only goes up
const samples = [];
let eta = '';
for (let i = 0; i < 40 && !eta; i++) {
const state = (await api('/api/native')).body;
const d = state.jobs.at(-1)?.result?.install?.download;
if (d) samples.push(d.done_gb);
eta = (await page.locator('.onboard-progress-actions span').first().textContent({ timeout: 1000 }).catch(() => '')) ?? '';
await delay(1000);
}
await shot(page, '1-wizard-progress');
check(/About \d+ (min left|h)/.test(eta), 'the wizard shows about how many minutes are left', eta.trim());
const order = await page.evaluate(() => { const a = document.querySelector('.onboard-progress-actions'); const kids = a ? [...a.children].map(k => k.tagName + ':' + Math.round(k.getBoundingClientRect().left)) : []; return kids; });
check(order.length === 2 && order[0].startsWith('SPAN') && order[1].startsWith('BUTTON') && Number(order[0].split(':')[1]) < Number(order[1].split(':')[1]), 'the time left sits left of Stop', order.join(' '));
check(samples.length >= 3 && samples.every((v, i) => i === 0 || v >= samples[i - 1]), 'one combined count that only goes up', samples.map(v => v.toFixed(1)).join(', '));
const meta = await page.locator('.onboard-progress-meta span').first().textContent();
check(/GB/.test(meta ?? '') && /83\.6/.test(meta ?? ''), 'the bar says done / total over the whole model', meta);
// close the pop-up while it downloads
await clickText(page, /Finish later/i);
await page.waitForURL(/#\/models/, { timeout: 15000 }).catch(() => {});
await page.getByText(/Preparing your model/i).waitFor({ timeout: 15000 }).catch(() => {});
await delay(1500);
await shot(page, '2-models-progress');
await page.locator('.native-progress').first().screenshot({ path: join(shots, 'download-3-models-progress-card.png') }).catch(() => {});
check(await page.locator('.guided-onboarding').count() === 0, 'the setup pop-up closed');
check(await page.getByText(/Preparing your model/i).count() > 0, 'the Models screen shows the running download');
const still = (await api('/api/native')).body.jobs.at(-1);
check(still?.state === 'running', 'the download keeps running in the background', still?.state);
const modelsEta = await page.locator('.native-progress .onboard-progress-actions span').first().textContent({ timeout: 15000 }).catch(() => '');
check(/About \d+ (min left|h)/.test(modelsEta ?? ''), 'the Models screen shows the time left too', (modelsEta ?? '').trim());
await clickText(page, /^Stop$/);
await delay(2500);
check((await api('/api/native')).body.jobs.at(-1)?.state !== 'running', 'Stop stops it');
check(page.problems.length === 0, 'no page error', page.problems.join('; '));
});
// ------------------------------------------------------------------------------------------------ models
await scenario('models', { installed: true }, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /Go straight to the app/i);
await page.waitForURL(/#\/models/, { timeout: 15000 }).catch(() => {});
await page.getByText(/Conversation context/i).first().waitFor({ timeout: 30000 });
await delay(1500);
await shot(page, '1-models');
await page.locator('.version-trigger').first().waitFor({ timeout: 20000 }).catch(async () => { await shot(page, '0-models-missing-summary'); console.log('info page text:', JSON.stringify((await page.locator('body').innerText()).slice(0, 500))); });
const summary = await page.locator('.version-trigger').first().textContent({ timeout: 5000 }).catch(() => '');
// the Models tab fits the first view: nothing to scroll while no list is open and Expert mode is closed
if (await page.getByRole('button', { name: /^Close expert mode$/ }).count()) { await page.getByRole('button', { name: /^Close expert mode$/ }).click(); await delay(500); }
await shot(page, '1b-models-one-view');
const overflow = await page.evaluate(() => { const el = document.querySelector('.main__body'); return el ? el.scrollHeight - el.clientHeight : 0; });
check(overflow <= 2, 'the Models tab fits the first view without scrolling', String(overflow) + ' px over');
check(await page.locator('.family-switch').count() === 1, 'the model family switch is in the navbar');
check(await page.locator('h1').filter({ hasText: /^Model$/ }).count() === 0, 'and the navbar does not say "Model"');
check(/IQ3_S|High quality/.test(summary ?? ''), 'Models selects the model that is on this PC, not the recommendation', (summary ?? '').replace(/\s+/g, ' ').trim());
check(!/IQ3_XXS|Balanced/.test(summary ?? ''), 'the recommendation (IQ3_XXS) is not what is selected');
const load = page.getByRole('button', { name: /^Load model$/ }).first();
await load.click();
await delay(1200);
check(await page.getByRole('dialog').filter({ hasText: /Download and prepare/i }).count() === 0, 'loading an installed model opens no download pop-up');
let ready = false;
for (let i = 0; i < 90 && !ready; i++) { const s = (await api('/api/native')).body; ready = s.instances.some(x => x.status === 'ready'); const err = s.instances.find(x => x.status === 'error'); if (err) throw new Error('load failed: ' + err.error); await delay(1000); }
if (!ready) { const st = (await api('/api/native')).body; console.log('info jobs', JSON.stringify(st.jobs.map(j => [j.kind, j.state, j.error ?? j.summary])), 'instances', JSON.stringify(st.instances.map(i => [i.status, i.error]))); await shot(page, '2-models-after-click'); }
check(ready, 'the model loads (a fake engine under the real Control)');
await page.reload({ waitUntil: 'domcontentloaded' });
await page.getByRole('button', { name: /^Unload$/ }).first().waitFor({ timeout: 30000 }).catch(() => {});
await shot(page, '3-models-loaded');
check(await page.getByRole('button', { name: /^Unload$/ }).count() > 0, 'after a reload the Models screen shows the model as loaded, with Unload');
check(await page.getByRole('button', { name: /^Load model$/ }).count() === 0, 'and no longer offers Load model');
await page.getByRole('button', { name: /^Unload$/ }).first().click();
let gone = false;
for (let i = 0; i < 40 && !gone; i++) { gone = (await api('/api/native')).body.instances.length === 0; await delay(500); }
check(gone, 'Unload stops the engine');
await page.getByRole('button', { name: /^Load model$/ }).first().waitFor({ timeout: 15000 }).catch(() => {});
check(await page.getByRole('button', { name: /^Load model$/ }).count() > 0, 'and Load model is back');
check(page.problems.length === 0, 'no page error', page.problems.join('; '));
});
// ------------------------------------------------------------------------------------------------ the profile used, the popover, the pop-up
await scenario('profile', { installed: true }, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /Go straight to the app/i);
await page.getByText(/Conversation context/i).first().waitFor({ timeout: 30000 });
await delay(1500);
const work = () => page.getByRole('button', { name: /Leave 20% for my work/ }).first();
const full = () => page.getByRole('button', { name: /Full performance/ }).first();
await work().click();
check(await work().getAttribute('aria-pressed') === 'true', 'Leave 20% is chosen before loading');
await page.getByRole('button', { name: /^Load model$/ }).first().click();
let ready = false;
for (let i = 0; i < 90 && !ready; i++) { const s = (await api('/api/native')).body; ready = s.instances.some(x => x.status === 'ready'); await delay(500); }
check(ready, 'the model loads');
const inst = (await api('/api/native')).body.instances.find(x => x.status === 'ready');
check(inst?.resourceProfile === 'work', 'the running model reports the profile it was started with', String(inst?.resourceProfile));
await delay(1500);
check(await work().getAttribute('aria-pressed') === 'true' && await full().getAttribute('aria-pressed') === 'false', 'after the load the page still shows Leave 20%, not Full performance');
await page.reload({ waitUntil: 'domcontentloaded' });
await page.getByRole('button', { name: /^Unload$/ }).first().waitFor({ timeout: 30000 });
await delay(1000);
check(await work().getAttribute('aria-pressed') === 'true', 'and after a reload it is still the profile in use');
await shot(page, '1-profile-used');
// the reasoning popover names the loaded model
await page.evaluate(() => { location.hash = '#/chat'; });
const effort = page.getByRole('button', { name: /^Reasoning effort/ }).first();
await effort.waitFor({ timeout: 30000 });
await effort.click();
const header = await page.locator('.reasoning-menu-model strong').first().textContent({ timeout: 10000 });
check(header && !/No model loaded/i.test(header), 'the Reasoning popover names the loaded model, not "No model loaded"', String(header));
await shot(page, '2-reasoning-popover');
await page.keyboard.press('Escape');
// an installed model asked for with other settings: the pop-up never speaks of downloading
await page.evaluate(() => { location.hash = '#/models'; });
await page.getByRole('button', { name: /^Unload$/ }).first().waitFor({ timeout: 30000 });
await page.getByRole('button', { name: /^Unload$/ }).first().click();
await page.getByRole('button', { name: /^Load model$/ }).first().waitFor({ timeout: 30000 });
await page.getByRole('combobox', { name: 'Conversation context' }).first().click();
const options = page.getByRole('option');
const pick = (await options.allTextContents()).findIndex(t => /^8K$/.test(t.trim()));
if (pick >= 0) await options.nth(pick).click();
await page.getByRole('button', { name: /^Load model$|^Checking/ }).first().click();
const dialog = page.getByRole('dialog').first();
await dialog.waitFor({ timeout: 15000 }).catch(() => {});
const text = (await dialog.textContent().catch(() => '')) ?? '';
await shot(page, '3-apply-dialog');
if (text) {
check(!/Download and prepare|Download . load|approximately|[0-9] GB/i.test(text), 'for a downloaded model the pop-up has no download wording or size', text.replace(/\s+/g, ' ').slice(0, 600));
check(/Apply/.test(text), 'it says what happens: apply settings and load');
} else console.log('info the settings were the saved ones, so loading needed no pop-up at all');
check(page.problems.length === 0, 'no page error', page.problems.join('; '));
});
// ------------------------------------------------------------------------------------------------ version list, This computer, Tune, a task
for (const scheme of ['light', 'dark']) await scenario(scheme === 'light' ? 'ui2' : 'ui2d', { installed: true, env: { FAKE_TOKEN_DELAY: '0.02', FAKE_REPLY: 'I am thinking about it. </think> ' + 'Here is the answer, word by word. '.repeat(12), STRATA_TUNE_NO_QUIET_CHECK: '1' } }, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base, { colorScheme: scheme });
await clickText(page, /Go straight to the app/i);
await page.getByText(/Conversation context/i).first().waitFor({ timeout: 30000 });
await delay(1500);
// the version list: Recommended (the fixed four), then folded groups, one line per model
console.log('info triggers', await page.locator('.version-trigger').count(), await page.getByRole('button', { name: 'Model version' }).count());
await page.locator('.version-trigger').first().click();
await delay(300);
console.log('info expanded after click', await page.locator('.version-trigger').first().getAttribute('aria-expanded'));
await delay(800); await shot(page, `0-after-click-${scheme}`);
console.log('info menu', await page.evaluate(() => { const m = document.querySelector('.version-menu'); const t = document.querySelector('.version-trigger'); return JSON.stringify({ menu: m ? m.getBoundingClientRect() : null, parent: m ? [m.parentElement.className, getComputedStyle(m.parentElement).display, getComputedStyle(m.parentElement).visibility, m.parentElement.style.cssText.slice(0, 120)] : null, expanded: t?.getAttribute('aria-expanded'), roles: [...document.querySelectorAll('[role=menu]')].length }); }));
await page.locator('.version-menu').waitFor({ timeout: 10000 });
const heads = await page.locator('.version-head span:first-child, .version-group__head').allTextContents();
const names = await page.locator('.version-menu > .version-row .version-row__name strong').allTextContents();
console.log('info version list', JSON.stringify(heads.map(t => t.replace(/\s+/g, ' ').trim())), JSON.stringify(names));
check(heads[0]?.trim() === 'Recommended', `${scheme}: the list starts with Recommended`);
check(names.length >= 1 && names.length <= 4, `${scheme}: Recommended shows only the picks this catalog has (at most the fixed four)`, names.join(' | '));
check(heads.some(t => /Swift models|GSQ-RCO models|Gyro models|NVFP4 models|Unsloth models/.test(t)), `${scheme}: the other models are in folded groups`);
check(heads.at(-1)?.includes('Use your own GGUF'), `${scheme}: the last row is Use your own GGUF`);
const tall = await page.locator('.version-row').evaluateAll(els => els.filter(e => e.getBoundingClientRect().height > 44).length);
check(tall === 0, `${scheme}: every model is one line`, String(tall));
await shot(page, `1-versions-${scheme}`);
await page.keyboard.press('Escape');
// This computer
const pc = page.locator('.pc-card');
await pc.waitFor({ timeout: 10000 });
const pcText = ((await pc.textContent()) ?? '').replace(/\s+/g, ' ');
check(/This computer/.test(pcText) && /(Fits fully in memory|read from the SSD)/.test(pcText), `${scheme}: This computer says in one line whether it fits`, pcText.slice(0, 200));
check(await pc.locator('.pc-meter').count() >= 1, `${scheme}: it shows the memory bars`);
check(await pc.locator('.chip, .st-pill, [class*="badge"]').count() === 0, `${scheme}: no chips or badges in it`);
await shot(page, `2-computer-${scheme}`);
// Tune: five steps, a live speed, a result card
await page.getByRole('button', { name: /^Tune model$/ }).first().click({ timeout: 10000 });
await delay(1500); await shot(page, `2b-tune-dialog-${scheme}`);
// The fake engine has no other setting worth trying, so its tune passes through "trying" too fast to be seen. The page is shown the progress a
// real tune reports (Control's status file: stage, speeds) for the running view; the start, the result and every other call are real.
let fakePhase = 'run';
await page.route('**/api/native*', async route => {
const response = await route.fetch(); let json = null;
try { json = await response.json(); } catch { return route.fulfill({ response }); }
for (const job of json?.jobs ?? []) if (job.kind === 'autotune' && fakePhase === 'done') { job.state = 'done'; job.progress = null; job.result = { outcome: 'improved', gain_pct: 18, base: { decode_tok_s: 41.2 }, chosen: { decode_tok_s: 48.6 }, preset: { id: 'fast', name: 'Fast · 64K' }, answers: { class: 'exact', fields: [] } }; } else if (job.kind === 'autotune' && job.state === 'running') job.progress = { stage: 'rounds', text: null, minutesLeft: 9, runsDone: 3, runsPlanned: 6, elapsedS: 300, step: 'Trying: more experts kept in memory (round 1 of 2)', speeds: [{ label: 'base', tok_s: 41.2 }, { label: 'Fewer helper threads', tok_s: 40.1 }, { label: 'More experts kept in memory', tok_s: 46.8 }] };
return route.fulfill({ response, json });
});
await page.getByRole('button', { name: /Start tuning|Tune again/ }).first().click({ timeout: 15000 });
await page.locator('.tune-step').first().waitFor({ timeout: 60000 });
check(await page.locator('.tune-step').count() === 5, `${scheme}: the tuner shows its five steps`);
let sawRun = false, sawSpeed = false;
for (let i = 0; i < 400; i++) {
const done = await page.locator('.tune-result, .tune-progress').count();
if (i % 4 === 0) { const jobs = (await api('/api/native')).body?.jobs ?? []; const j = jobs.find(x => x.kind === 'autotune'); if (j) console.log('info tune job', j.state, JSON.stringify(j.progress ?? null).slice(0, 300)); }
if (await page.locator('.tune-step[data-s=run]').count()) sawRun = true;
const big = await page.locator('.tune-live__big').first().textContent().catch(() => '');
if (/[0-9]/.test(big ?? '')) sawSpeed = true;
if (sawRun && sawSpeed) { await delay(700); await shot(page, `3-tune-running-${scheme}`); break; }
if (done) break;
await delay(600);
}
check(sawRun, `${scheme}: a step is shown as running`);
check(sawSpeed, `${scheme}: the live speed shows a number while it runs`);
if (sawSpeed) { fakePhase = 'done'; await page.locator('.tune-outcome').waitFor({ timeout: 15000 }).catch(() => {}); check(/18% faster/.test((await page.locator('.tune-outcome__body h3').textContent().catch(() => '')) ?? ''), `${scheme}: the result card shows the gain as its headline`); check(/Fast/.test((await page.locator('.tune-outcome__facts').textContent().catch(() => '')) ?? ''), `${scheme}: and what it was saved as`); }
await shot(page, `4-tune-result-${scheme}`);
await page.getByRole('button', { name: /^Close$/ }).first().click({ timeout: 10000 }).catch(() => {});
// a task: the opened task is skimmable, with pause / delete / Edit / Run now
const made = await api('/api/tasks', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ title: 'Open Downloads folder', prompt: 'Open my Downloads folder and tell me what is new.', frequency: 'weekdays', at: '08:00', weekday: 1, timeZone: 'Europe/Ljubljana', mode: 'manual', enabled: true }) });
const id = made.body?.id ?? made.body?.task?.id;
check(!!id, `${scheme}: a task is created`, JSON.stringify(made.body).slice(0, 120));
await page.evaluate(i => { location.hash = '#/tasks/' + i; }, id);
await page.getByRole('button', { name: /^Edit$/ }).first().waitFor({ timeout: 15000 });
console.log('info header buttons', await page.getByRole('button', { name: /^Pause / }).count(), await page.getByRole('button', { name: /^Delete / }).count(), await page.getByRole('button', { name: /Run now/ }).count(), JSON.stringify(await page.locator('.task-header-actions button').evaluateAll(b => b.map(x => (x.getAttribute('aria-label') ?? x.textContent ?? '').trim()))));
check(await page.getByRole('button', { name: /^Pause / }).count() === 1 && await page.getByRole('button', { name: /^Delete / }).count() >= 1 && await page.getByRole('button', { name: /Run now/ }).count() >= 1, `${scheme}: pause, delete, Edit and Run now are in the header`);
check(await page.locator('.main__back').count() === 0 && await page.locator('.main__title-line .task-state').count() === 0, `${scheme}: nothing sits on the left of an opened task's navbar`);
const factsText = ((await page.locator('.task-facts').textContent()) ?? '').replace(/\s+/g, ' ');
check(/Weekdays/.test(factsText) && /Schedule/i.test(factsText) && /Last result/i.test(factsText), `${scheme}: schedule, what it may do and the last result are at the top`, factsText.slice(0, 160));
await shot(page, `5-task-${scheme}`);
await page.getByRole('button', { name: /^Pause / }).first().click();
await page.getByRole('button', { name: /^Resume / }).first().waitFor({ timeout: 10000 });
check(true, `${scheme}: Pause becomes Resume`);
check(page.problems.length === 0, `${scheme}: no page error`, page.problems.join('; '));
});
// ------------------------------------------------------------------------------------------------ tune + reading size
await scenario('tune', { installed: true, env: { FAKE_TOKEN_DELAY: '0.02', FAKE_REPLY: ['I am thinking about it.', '</think>', '', 'Here is a list:', '', '- one', '- two', '', '## Heading', '', 'Some `code` and **bold** text.', '', '| a | b |', '|---|---|', '| 1 | 2 |', '', '```', 'code line', '```', ''].join(String.fromCharCode(10)) } }, async ({ base, api, browser, dirs, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /get started/i);
await page.getByText(/Choose a version/i).waitFor({ timeout: 30000 });
await page.locator('[role=radio]').filter({ hasText: 'IQ3_S' }).first().click();
await clickText(page, /^Continue/);
await page.getByText(/Leave 20% for my work/i).waitFor({ timeout: 30000 });
await delay(800);
await shot(page, '1-tune-step');
check(await page.getByText(/Read pictures when this model loads/i).count() === 0, 'the tuning step offers no picture reading for an app install');
await clickText(page, /Tune & start chatting/i);
let done = false, job = null;
for (let i = 0; i < 120 && !done; i++) { job = (await api('/api/native')).body.jobs.findLast?.(j => j.kind === 'tune') ?? null; done = !!job && job.state !== 'running'; await delay(1000); }
check(job?.state === 'done', 'tuning finishes without a vision error', job?.error ?? job?.state);
const cfg = JSON.parse(readFileSync(join(dirs.install, 'strata-iq3_s.json'), 'utf8'));
check(cfg.args.includes('--resident-budget-gib'), '"leave 20%" wrote a RAM budget for the experts', cfg.args.join(' '));
const gib = Number(cfg.args[cfg.args.indexOf('--resident-budget-gib') + 1]);
check(gib + 10 <= 63.1 * 0.8, 'the budget keeps the model within 80% of the PC RAM', `${gib} GiB`);
// chat + reading size
await page.getByRole('textbox', { name: /Message Strata/i }).waitFor({ timeout: 60000 });
await page.getByRole('textbox', { name: /Message Strata/i }).fill('hello');
await page.keyboard.press('Enter');
await page.locator('.st-msg--assistant .st-bubble').first().waitFor({ timeout: 60000 });
await page.locator('.st-msg--assistant .st-bubble p').first().waitFor({ timeout: 60000 }).catch(async () => console.log('info reply so far:', JSON.stringify(await page.locator('.st-msg--assistant').first().innerText()).slice(0, 300)));
await page.waitForFunction(() => !document.querySelector('.st-msg--assistant .caret'), null, { timeout: 90000 }).catch(() => {});
// the chat's telemetry chips: speed and context
const chips = page.locator('.telemetry__chip');
const chipTexts = await chips.allInnerTexts();
console.log('info chips', JSON.stringify(chipTexts));
let speedText = '', contextText = '';
for (let i = 0; i < await chips.count(); i++) {
await chips.nth(i).click();
await delay(400);
const pop = await page.locator('.telepop').last().innerText().catch(() => '');
if (/tok\/s|speed/i.test(pop) && !speedText) speedText = pop;
if (/context/i.test(pop) && !contextText) contextText = pop;
await page.keyboard.press('Escape');
}
const tps = Number(speedText.match(/([\d.]+)\s*tok(?:ens)?\/s/)?.[1] ?? NaN);
check(tps > 15 && tps < 120, 'the speed shown is the decode speed (the engine paced at ~50 tokens a second)', speedText.replace(/\s+/g, ' ').slice(0, 160));
check(/32\.8K|32K|32,768|32768/.test(contextText) && !/128K/.test(contextText), 'the context shown is the one the model runs with (32K), not 128K', contextText.replace(/\s+/g, ' ').slice(0, 160));
const sizes = [];
for (const px of [12, 14, 16, 22]) {
await page.evaluate(v => { const k = 'strata.prefs.v2'; const cur = JSON.parse(localStorage.getItem(k) || '{}'); localStorage.setItem(k, JSON.stringify({ ...cur, contentFontSize: v })); }, px);
await page.reload({ waitUntil: 'domcontentloaded' });
await page.locator('.st-msg--assistant .st-bubble').first().waitFor({ timeout: 30000 });
await page.screenshot({ path: join(shots, `tune-3-chat-${px}px.png`) });
sizes.push(await page.evaluate(() => { const b = document.querySelector('.st-msg--assistant .st-bubble'); const t = b.querySelector('p, .md-li, span') ?? b; const u = document.querySelector('.st-msg--user .st-bubble'); const all = {}; for (const el of b.querySelectorAll('*')) { if (!el.childNodes.length || ![...el.childNodes].some(n => n.nodeType === 3 && n.textContent.trim())) continue; all[el.tagName.toLowerCase() + (el.className ? '.' + String(el.className).split(' ')[0] : '')] = parseFloat(getComputedStyle(el).fontSize); } const extra = {}; for (const [k, q] of Object.entries({ reasoning: '.st-msg--assistant .st-collapse__body', codeblock: '.st-msg--assistant .st-code pre', codehead: '.st-msg--assistant .st-code__head' })) { const e = document.querySelector(q); if (e) extra[k] = parseFloat(getComputedStyle(e).fontSize); } Object.assign(all, extra); const chain = []; for (let e = b.querySelector('p'); e && e !== b; e = e.parentElement) chain.push(e.tagName + '.' + String(e.className).split(' ')[0] + '=' + getComputedStyle(e).fontSize); all.chain = chain; globalThis.__all = all; return [parseFloat(getComputedStyle(b).fontSize), parseFloat(getComputedStyle(t).fontSize), parseFloat(getComputedStyle(u).fontSize), all]; }));
}
console.log('info chain', JSON.stringify(await page.evaluate(() => { const el = document.querySelector('.st-msg--assistant .st-bubble p'); const out = []; for (let e = el; e && !e.classList.contains('st-msg'); e = e.parentElement) out.push(e.tagName + '.' + String(e.className).split(' ')[0] + '=' + getComputedStyle(e).fontSize); return out; })));
// which visible text does NOT follow the reading size? (same elements at 12 and 22)
const survey = async px => { await page.evaluate(v => { const k = 'strata.prefs.v2'; const cur = JSON.parse(localStorage.getItem(k) || '{}'); localStorage.setItem(k, JSON.stringify({ ...cur, contentFontSize: v })); }, px); await page.reload({ waitUntil: 'domcontentloaded' }); await page.locator('.st-msg--assistant .st-bubble').first().waitFor({ timeout: 30000 }); await delay(600);
return page.evaluate(() => { const out = {}; for (const el of document.querySelectorAll('main *, .chat *, textarea')) { if (![...el.childNodes].some(n => n.nodeType === 3 && n.textContent.trim()) && el.tagName !== 'TEXTAREA') continue; const r = el.getBoundingClientRect(); if (!r.width || !r.height) continue; const key = el.tagName.toLowerCase() + '.' + String(el.className).split(' ')[0] + ' <' + (el.parentElement?.className ?? '').toString().split(' ')[0] + '>'; out[key] = parseFloat(getComputedStyle(el).fontSize); } return out; }); };
const a = await survey(12), b = await survey(22);
// a fresh app (nothing saved) uses 14 px
await page.evaluate(() => localStorage.removeItem('strata.prefs.v2'));
await page.reload({ waitUntil: 'domcontentloaded' });
await page.locator('.st-msg--assistant .st-bubble').first().waitFor({ timeout: 30000 });
check(await page.evaluate(() => getComputedStyle(document.querySelector('.st-msg--assistant .st-bubble')).fontSize) === '14px', 'the default reading size is 14 px');
const fixed = Object.keys(a).filter(k => k in b && a[k] === b[k]);
console.log('info text that does not follow the reading size:', JSON.stringify(fixed.map(k => k + '=' + a[k])));
await shot(page, '2-chat-22px');
console.log('info sizes', JSON.stringify(sizes), JSON.stringify(await page.locator('.st-msg--assistant .st-bubble').first().innerHTML()).slice(0, 400));
check(sizes[0][1] === 12 && sizes[1][1] === 14 && sizes[2][1] === 16 && sizes[3][1] === 22 && sizes[0][2] === 12 && sizes[3][2] === 22, 'the reading size changes the text size of the conversation', JSON.stringify(sizes));
// the slider itself
await page.evaluate(() => { location.hash = '#/chat?settings=1'; });
const slider = page.getByLabel('Reading text size');
if (await slider.count()) {
await slider.first().focus();
for (let i = 0; i < 4; i++) await page.keyboard.press('ArrowRight');
await delay(500);
const now = await page.evaluate(() => getComputedStyle(document.querySelector('.st-msg--assistant .st-bubble') ?? document.body).fontSize);
check(parseFloat(now) === 18, 'moving the slider changes the conversation text at once', now);
const box = await slider.first().boundingBox();
await page.mouse.click(box.x + box.width * 0.9, box.y + box.height / 2);
await delay(400);
const dragged = await page.evaluate(() => [getComputedStyle(document.querySelector('.st-msg--assistant .st-bubble') ?? document.body).fontSize, getComputedStyle(document.documentElement).getPropertyValue('--st-content-fs'), JSON.parse(localStorage.getItem('strata.prefs.v2') || '{}').contentFontSize]);
check(parseFloat(dragged[0]) >= 20 && parseFloat(dragged[2]) >= 20, 'clicking the slider with the mouse changes it too, and it is saved', dragged.join(' '));
} else console.log('info the settings slider was not reachable by hash here; the stored size was checked above');
check(page.problems.length === 0, 'no page error', page.problems.join('; '));
});
// ------------------------------------------------------------------------------------------------ delete
await scenario('delete', { installed: true }, async ({ base, api, browser, dirs, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /Go straight to the app/i);
await page.waitForURL(/#\/models/, { timeout: 15000 }).catch(() => {});
await page.getByRole('button', { name: /^Load model$/ }).first().click();
let ready = false;
for (let i = 0; i < 90 && !ready; i++) { ready = (await api('/api/native')).body.instances.some(x => x.status === 'ready'); await delay(1000); }
check(ready, 'a model is loaded');
await clickText(page, /Manage models/i);
await page.getByRole('button', { name: /^Delete$/ }).first().click();
await delay(600);
await shot(page, '1-delete-confirm');
const dialog = await page.getByRole('dialog').last().innerText();
check(/Delete this version\?/.test(dialog) && /Remove 83\.6 GB of downloaded files\. You can download this version again\./.test(dialog), 'the confirm dialog uses the existing words and the real size', dialog.replace(/\s+/g, ' ').slice(0, 600));
check(/running model will be unloaded/i.test(dialog), 'and says the loaded model is unloaded first');
await page.getByRole('button', { name: /^Delete version$/ }).click();
let state = null;
for (let i = 0; i < 40; i++) { state = (await api('/api/native')).body; if (state.instances.length === 0 && !state.installed?.models?.some(m => m.model === 'iq3_s')) break; await delay(500); }
check(state.instances.length === 0, 'the loaded model was unloaded');
check(!existsSync(join(dirs.install, 'strata-iq3_s.json')), 'the installer removed the run config of the model');
check(!state.installed?.models?.some(m => m.model === 'iq3_s'), 'the model is no longer listed as downloaded');
await shot(page, '2-after-delete');
});
// ------------------------------------------------------------------------------------------------ perf
await scenario('perf', { installed: true }, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /Go straight to the app/i);
await page.waitForURL(/#\/models/, { timeout: 15000 }).catch(() => {});
await page.getByRole('button', { name: /^Load model$/ }).first().click();
let ready = false;
for (let i = 0; i < 90 && !ready; i++) { ready = (await api('/api/native')).body.instances.some(x => x.status === 'ready'); await delay(1000); }
check(ready, 'a model is loaded');
await delay(2500);
const requests = [];
const started = new Map();
page.on('request', r => started.set(r, Date.now()));
page.on('requestfinished', r => requests.push([r.url().replace(base, '').slice(0, 60), Date.now() - (started.get(r) ?? Date.now())]));
await page.evaluate(() => { window.__long = []; new PerformanceObserver(l => { for (const e of l.getEntries()) window.__long.push(Math.round(e.duration)); }).observe({ entryTypes: ['longtask'] }); });
const t0 = Date.now();
await page.evaluate(() => { location.hash = '#/chat'; });
await page.getByRole('textbox', { name: /Message Strata/i }).waitFor({ timeout: 30000 });
const open = Date.now() - t0;
await delay(1500);
const long = await page.evaluate(() => window.__long);
console.log('info chat opened in', open, 'ms; long tasks', JSON.stringify(long), 'requests', JSON.stringify(requests.filter(r => r[1] > 100)), 'all', requests.length);
check(open < 1500 && Math.max(0, ...long) < 300, 'Models to Chat opens at once, with no long block', open + ' ms, longest task ' + Math.max(0, ...long) + ' ms');
});
// ------------------------------------------------------------------------------------------------ polish
await scenario('polish', { installed: true }, async ({ base, api, browser, dirs, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /Go straight to the app/i);
await page.waitForURL(/#\/models/, { timeout: 15000 }).catch(() => {});
await page.getByText(/Conversation context/i).first().waitFor({ timeout: 30000 });
check(await page.getByRole('button', { name: /^Tune model$/ }).count() === 1, 'the Models tab has the Tune model button (the real tuner, not the old short benchmark)');
await clickText(page, /Manage models/i);
await delay(500);
const rows = await page.locator('.native-manage-row').count();
check(rows === 1, 'Manage models lists only the downloaded model, not the catalog', rows + ' row(s)');
await shot(page, '1-manage-models');
await page.getByRole('button', { name: /^Done$/ }).click();
await page.getByRole('button', { name: /Other local models/i }).click().catch(async () => { await clickText(page, /Manage models/i); await clickText(page, /Use a local GGUF/i); });
await page.getByText(/Folder with the GGUF files/i).first().waitFor({ timeout: 15000 }).catch(() => {});
await shot(page, '2-local-gguf');
check(await page.getByText(/Folder with the GGUF files/i).count() > 0, 'the local GGUF page opens in the same design as Models');
await page.getByPlaceholder(/Full path to the folder/i).fill(join(dirs.install, 'does-not-exist'));
await page.getByRole('button', { name: /^Check files$/ }).click();
await delay(3000);
check(await page.locator('.error-notice').count() > 0, 'a folder that cannot be used says so in words');
await shot(page, '2b-local-gguf-problem');
await page.evaluate(() => { location.hash = '#/chat?settings=1'; });
await delay(1500);
const text = await page.locator('body').innerText();
check(!/Contrast/.test(text), 'Settings has no Contrast option');
check(!/View request log/.test(text), 'and no "View request log" button');
await page.getByText(/^Engine$/).first().click().catch(() => {});
await delay(1500);
await shot(page, '3-settings-engine');
const engine = await page.locator('body').innerText();
check(/Update component/.test(engine) && /Diagnostics/.test(engine), 'Settings > Engine has Update component and Diagnostics');
check(!/Performance|Model settings|Calibrate|Engine installation/.test(engine), 'and nothing else');
await page.getByRole('button', { name: /^View logs$/ }).click(); // the logs open in a pop-up that holds Save logs
await page.getByRole('button', { name: /^Save logs$/ }).click();
await page.getByText(/Saved to/).first().waitFor({ timeout: 15000 }).catch(() => {});
const savedText = await page.getByText(/Saved to/).first().innerText().catch(() => '');
const savedPath = savedText.replace(/^Saved to\s*/, '').trim();
check(!!savedPath && existsSync(savedPath), 'Save logs writes a file to Downloads and says where', savedPath);
await shot(page, '4-logs-saved');
await page.evaluate(() => { location.hash = '#/monitor'; });
await delay(6000);
await shot(page, '5-monitor');
const mon = await page.locator('body').innerText();
check(/°C/.test(mon), 'the Monitor shows GPU data with no model loaded', (mon.match(/.{0,30}°C.{0,20}/)?.[0] ?? 'no temperature text').replace(/\s+/g, ' '));
});
// ------------------------------------------------------------------------------------------------ stop
await scenario('stop', { installed: true, env: { FAKE_TOKEN_DELAY: '0.1', FAKE_REPLY: 'thinking '.repeat(30) } }, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /Go straight to the app/i);
await page.waitForURL(/#\/models/, { timeout: 15000 }).catch(() => {});
await page.getByRole('button', { name: /^Load model$/ }).first().click();
let ready = false;
for (let i = 0; i < 90 && !ready; i++) { ready = (await api('/api/native')).body.instances.some(x => x.status === 'ready'); await delay(1000); }
if (!ready) { const st = (await api('/api/native')).body; console.log('info jobs', JSON.stringify(st.jobs.map(j => [j.kind, j.state, j.error ?? j.summary])), JSON.stringify(st.instances.map(i => [i.status, i.error]))); await shot(page, '0-stop-notloaded'); }
check(ready, 'a model is loaded');
await page.evaluate(() => { location.hash = '#/chat'; });
const box = page.getByRole('textbox', { name: /Message Strata/i });
await box.waitFor({ timeout: 30000 });
await box.fill('hello');
await page.keyboard.press('Enter');
await page.locator('.typing').first().waitFor({ timeout: 30000 });
await delay(1500);
await page.getByRole('button', { name: /^Stop/i }).first().click().catch(async () => { await page.keyboard.press('Escape'); });
await delay(4000);
const dots = await page.locator('.typing').count();
await shot(page, '1-after-stop');
check(dots === 0, 'after Stop in the middle of thinking the three dots are gone', dots + ' left');
});
// ------------------------------------------------------------------------------------------------ latest
await scenario('latest', {}, async ({ base, browser, shot }) => {
const page = await newPage(browser, base);
const now = Date.now();
await page.evaluate(now => {
const messages = [];
for (let i = 0; i < 40; i++) { messages.push({ id: 'u' + i, role: 'user', content: 'question ' + i, at: now }, { id: 'a' + i, role: 'assistant', content: 'answer number ' + i + ' with some text to fill the line and more of it.', at: now, tokens: 20, inputTokens: 5 }); }
localStorage.setItem('strata.sessions.v1', JSON.stringify([{ id: 's_latest', title: 'long', updatedAt: now, createdAt: now, messages }]));
localStorage.setItem('strata.active-session.v1', 's_latest');
}, now);
await page.reload({ waitUntil: 'domcontentloaded' });
await delay(1500);
if (await page.getByRole('button', { name: /Go straight to the app/i }).count()) { await clickText(page, /Go straight to the app/i); await delay(1500); }
await page.goto(base + '/#/chat', { waitUntil: 'domcontentloaded' });
await page.getByRole('textbox', { name: /Message Strata/i }).waitFor({ timeout: 30000 });
await delay(800);
await page.locator('.main__body.chat').first().waitFor({ timeout: 15000 }).catch(async () => { await shot(page, '0-latest-missing'); console.log('info body', JSON.stringify((await page.locator('body').innerText()).slice(0, 300))); });
await page.locator('.main__body.chat').first().evaluate(el => { el.scrollTop = 3000; });
await delay(600);
await page.mouse.wheel(0, -400);
await delay(600);
await page.locator('.main__body.chat').first().evaluate(el => { el.scrollTop = 0; });
await delay(800);
const gaps = await page.evaluate(() => { const b = document.querySelector('.jumplatest'), c = document.querySelector('.main__body.chat'); if (!b || !c) return null; const r = b.getBoundingClientRect(), cr = c.getBoundingClientRect(); return { top: Math.round(r.top - cr.top), bottom: Math.round(cr.bottom - r.bottom) }; });
console.log('info chat dom', JSON.stringify(await page.evaluate(() => { const c = document.querySelector('.main__body.chat'); return { msgs: document.querySelectorAll('.st-msg').length, scrollH: c?.scrollHeight, clientH: c?.clientHeight, top: c?.scrollTop, jump: document.querySelectorAll('.jumplatest').length }; })));
await shot(page, '1-latest-button');
check(!!gaps && gaps.top >= 12 && gaps.bottom >= 12, 'the Latest button keeps a gap from the top navigation and from the bottom', JSON.stringify(gaps));
});
// ------------------------------------------------------------------------------------------------ fail
await scenario('fail', { installed: true, env: { FAKE_ENGINE_FAIL: '1' } }, async ({ base, api, browser, shot }) => {
const page = await newPage(browser, base);
await clickText(page, /get started/i);
await page.getByText(/Choose a version/i).waitFor({ timeout: 30000 });
await page.locator('[role=radio]').filter({ hasText: 'IQ3_S' }).first().click();
await clickText(page, /^Continue/);
await clickText(page, /Tune & start chatting/i, 30000);
await page.getByText(/ran low on memory/i).first().waitFor({ timeout: 120000 });
await delay(1000);
await shot(page, '1-tune-error');
const text = await page.locator('.guided-onboarding').innerText();
check(/ran low on memory/.test(text), 'the error says in words that the PC ran low on memory');
check(/Leave 20% for my work/.test(text) && /low-RAM/.test(text), 'and what to try');
check(!/server_exited|memory guard/.test(text), 'no code or log line in the main text');
check(!/vision files/i.test(text), 'no picture-reading error');
const bad = await noScroll(page);
check(bad.length === 0, 'the tuning pop-up does not scroll when an error shows', bad.join('; '));
await page.locator('.error-notice summary').first().click();
await delay(500);
await shot(page, '2-tune-error-details');
const details = await page.locator('.error-notice pre').first().innerText();
check(/server_exited/.test(details) && /exited before/.test(details), 'Details holds the code and the reason the server logged', details.replace(/\s+/g, ' ').slice(0, 120));
const badOpen = await noScroll(page);
check(badOpen.length === 0, 'and the pop-up still does not scroll with Details open', badOpen.join('; '));
await page.setViewportSize({ width: 1280, height: 700 });
await delay(600);
const small = await noScroll(page);
check(small.length === 0, 'nor in a small window (1280x700), Details open', small.join('; '));
await shot(page, '3-tune-error-small-window');
void api;
});
// ------------------------------------------------------------------------------------------------ restart
// The app keeps its theme, text size, language and chat list in the browser's storage, which belongs to the window's origin
// (127.0.0.1:<port>). The shell now uses the same port every launch: stop the host, start it again on that port, reopen the
// window with the same browser profile, and everything is still there.
if (only.has('restart')) {
console.log('--- restart');
const fixed = await freePort();
const profile = join(work, 'browser-profile');
const run = async (body) => scenario('restart', { env: { FIXED_PORT: String(fixed) } }, async ctx => {
const context = await chromium.launchPersistentContext(profile, { executablePath: CHROME, headless: true, viewport: { width: 1200, height: 800 } });
try { await body(context, ctx); } finally { await context.close(); }
});
only.add('restart');
await run(async (context, { base }) => {
const page = context.pages()[0] ?? await context.newPage();
await page.goto(base, { waitUntil: 'domcontentloaded' });
await page.evaluate(() => { localStorage.setItem('strata.prefs.v2', JSON.stringify({ theme: 'dark', contentFontSize: 19, language: 'de' })); localStorage.setItem('strata.sessions.v1', JSON.stringify([{ id: 'chat-1', title: 'My chat' }])); });
});
await run(async (context, { base }) => {
const page = context.pages()[0] ?? await context.newPage();
await page.goto(base, { waitUntil: 'domcontentloaded' });
const saved = await page.evaluate(() => ({ prefs: JSON.parse(localStorage.getItem('strata.prefs.v2') || '{}'), chats: localStorage.getItem('strata.sessions.v1'), theme: document.documentElement.dataset.theme }));
check(saved.prefs.contentFontSize === 19 && saved.prefs.language === 'de', 'text size and language survive a restart on the same port', JSON.stringify(saved.prefs));
check(saved.theme === 'dark', 'the theme survives too', saved.theme);
check(/My chat/.test(saved.chats ?? ''), 'and the chat list', saved.chats);
});
}
// ------------------------------------------------------------------------------------------------ theme
await scenario('theme', {}, async ({ base, browser, shot }) => {
for (const scheme of ['dark', 'light']) {
const page = await newPage(browser, base, { colorScheme: scheme });
await delay(1500);
const theme = await page.evaluate(() => document.documentElement.dataset.theme);
check(theme === scheme, `a fresh app follows the system (${scheme})`, theme);
await shot(page, `1-${scheme}`);
await page.context().close();
}
});
// the junctions into the staged Control go first (rmdir on a junction removes only the link)
for (const name of ['python', 'strata_control', 'strata_cli', 'serve']) { try { rmdirSync(join(control, name)); } catch { /* already gone */ } }
try { rmSync(work, { recursive: true, force: true, maxRetries: 3 }); } catch (e) { console.log('info scratch folder left behind:', work); }
console.log(`shots in ${shots}`);
process.exit(failed ? 1 : 0);
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env node
/**
* Picture reading in the APP, end to end on a test box, through the installed app layout (packaged host + bundled Control + a real engine component
* that carries strata-vision):
* Download and prepare with "Read pictures" ON (the picture-reader file is fetched, the vision settings are written), load the model with the picture
* reader, send a real picture through the host's own /v1 endpoint and check that the answer names what is in it, unload, shut down.
*
* node vision-e2e.mjs --install <folder with node/ host/ control/> --components <folder holding engine/<package with strata-vision>> \
* --work <scratch> --family qwen --model IQ3_XXS --data <data folder that has (or will get) the model files> --image shapes.png [--device cpu|gpu]
*
* The picture has a red circle, a blue square, a green triangle and the text "STRATA 42". The run exits 1 if any step fails.
*/
import { spawn } from 'node:child_process';
import { existsSync, mkdirSync, readFileSync, readdirSync, statSync } from 'node:fs';
import { createServer } from 'node:net';
import { join, resolve } from 'node:path';
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => (a.startsWith('--') ? [...acc, [a.slice(2), all[i + 1]?.startsWith('--') || all[i + 1] === undefined ? true : all[i + 1]]] : acc), []));
for (const need of ['install', 'components', 'work', 'model', 'image']) if (!args[need]) { console.error('missing --' + need); process.exit(2); }
const install = resolve(args.install), work = resolve(args.work), components = resolve(args.components), family = String(args.family ?? 'qwen'), modelId = String(args.model);
const device = String(args.device ?? 'cpu');
const node = join(install, 'node', 'node'), hostEntry = join(install, 'host', 'server', 'index.mjs'), controlDir = join(install, 'control');
const dirs = { models: join(work, 'models'), profiles: join(work, 'profiles'), runtime: join(work, 'runtime'), home: join(work, 'home'), xdg: join(work, 'xdg'), appdata: join(work, 'appdata') };
for (const d of [dirs.models, dirs.home, dirs.xdg, dirs.appdata]) mkdirSync(d, { recursive: true });
const delay = ms => new Promise(r => setTimeout(r, ms));
let failed = false;
const record = (name, ok, detail = '') => { if (!ok) failed = true; console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ' ' + String(detail).slice(0, 600) : ''}`); };
const freePort = async () => { const s = createServer(); await new Promise(r => s.listen(0, '127.0.0.1', r)); const p = s.address().port; await new Promise(r => s.close(r)); return p; };
const env = { ...process.env, STRATA_COMPONENTS_DIR: components, STRATA_PROFILE_DIR: dirs.profiles, STRATA_RUNTIME_DIR: dirs.runtime, STRATA_INSTALL_DIR: dirs.models,
STRATA_HOME: dirs.home, STRATA_MODELS_DIR: join(work, 'models-unused'), APPDATA: dirs.appdata, XDG_CONFIG_HOME: dirs.xdg, PYTHONDONTWRITEBYTECODE: '1' };
delete env.STRATA_API_KEY;
const port = await freePort();
const base = `http://127.0.0.1:${port}`;
const host = spawn(node, [hostEntry], { cwd: join(install, 'host'), stdio: ['ignore', 'pipe', 'pipe'],
env: { ...env, STRATA_PORT: String(port), STRATA_DIST: join(install, 'host', 'dist'), NODE_ENV: 'production', STRATA_PACKAGED: '1', STRATA_REAL_ENGINE: '1', STRATA_NO_ACQUIRE: '1',
STRATA_SHARE_PORT: String(await freePort()), STRATA_TUNNEL_PORT: String(await freePort()) } });
let hostLog = '';
host.stdout.on('data', c => { hostLog += c; }); host.stderr.on('data', c => { hostLog += c; });
const call = async (path, method = 'GET', body) => {
const response = await fetch(base + path, { method, headers: { 'content-type': 'application/json' }, body: body === undefined ? undefined : JSON.stringify(body) });
return { status: response.status, body: await response.json().catch(() => null) };
};
async function waitJob(id, seconds) {
let last = '';
for (let i = 0; i < seconds * 2; i++) {
const state = (await call('/api/native')).body;
const job = state.jobs.find(j => j.id === id);
if (job && job.summary !== last) { last = job.summary; console.log(' ... ' + String(last).slice(0, 120)); }
if (job && job.state !== 'running') return { job, state };
await delay(500);
}
throw new Error('operation did not finish');
}
async function main() {
let up = false;
for (let i = 0; i < 200 && !up; i++) { try { up = (await fetch(base + '/api/health')).ok; } catch { /* not yet */ } if (!up) await delay(150); }
record('host healthy', up, up ? base : hostLog.slice(-500));
if (!up) return;
const packages = existsSync(join(components, 'engine')) ? readdirSync(join(components, 'engine')) : [];
const reader = packages.some(p => existsSync(join(components, 'engine', p, 'strata-vision')));
record('the engine component carries the picture reader (strata-vision)', reader, packages.join(', '));
const status = (await call('/api/control/status')).body;
record('Control: engine usable', status.engine?.usable === true, `engine=${status.engine?.engine?.version}`);
// ---- Download and prepare, "Read pictures" on (the app sends vision "yes" for the download)
const values = { family, model: modelId, context: 32768, vision: 'yes', loadVision: device, ...(args.data ? { data_dir: resolve(args.data) } : {}) };
const plan = await call('/api/native/plan', 'POST', values);
record('the plan for the download is made', plan.status === 200, JSON.stringify(plan.body).slice(0, 300));
const started = await call('/api/native/operations', 'POST', { kind: 'install', args: values });
record('the install operation starts', started.status === 200 || started.status === 202, JSON.stringify(started.body).slice(0, 200));
const done = await waitJob(started.body.id, 3600);
record('Download and prepare finishes', done.job.state === 'done', `${done.job.state} ${done.job.error ?? done.job.summary}`);
const native = (await call('/api/native?refresh=1')).body;
console.log(' installed models: ' + JSON.stringify((native.installed?.models ?? []).map(m => ({ id: m.id, model: m.model, ready: m.ready, visionPrepared: m.visionPrepared, images: m.images, config: m.config }))));
const want = modelId.toLowerCase();
const model = native.installed?.models?.find(m => [m.id, m.model, m.tag].some(x => String(x ?? '').toLowerCase() === want));
record('the model is installed with the picture reader prepared (visionPrepared)', !!model?.visionPrepared, JSON.stringify({ ready: model?.ready, visionPrepared: model?.visionPrepared, images: model?.images }));
const configFile = model?.configPath ?? (model?.config ? join(dirs.models, model.config) : null);
let cfg = {};
try { cfg = JSON.parse(readFileSync(configFile, 'utf8')); } catch { /* reported below */ }
const mm = cfg.vision?.mmproj;
record('the run settings carry the vision entry and --vision, and the mmproj file is on disk',
!!cfg.vision && (cfg.args ?? []).includes('--vision') && !!mm && existsSync(mm) && statSync(mm).size > 800e6,
`vision=${cfg.vision ? 'yes' : 'no'} --vision=${(cfg.args ?? []).includes('--vision')} mmproj=${mm ? Math.round(statSync(mm).size / 1048576) + ' MB' : 'none'}`);
// ---- load with the picture reader
const load = await call('/api/native/operations', 'POST', { kind: 'load', args: { model: model?.model ?? modelId.toLowerCase(), vision: device } });
const loaded = await waitJob(load.body.id, 900);
const instance = loaded.state.instances[0];
record('load starts the engine with the picture reader', loaded.job.state === 'done' && instance?.status === 'ready', `${loaded.job.state} ${loaded.job.error ?? ''}; instance=${instance?.status}; vision=${instance?.vision}`);
record('the host says the loaded model reads pictures', !!instance?.vision, `vision=${instance?.vision}`);
// ---- a real picture through the host's own /v1 endpoint
const png = readFileSync(resolve(args.image)).toString('base64');
const t = Date.now();
let answer = '', finish = '';
for (const budget of [1536, 4096]) {
const chat = await call('/v1/chat/completions', 'POST', { model: instance?.apiModelId ?? modelId, max_tokens: budget, stream: false, messages: [{ role: 'user', content: [
{ type: 'text', text: 'Describe this picture: name each shape with its colour and read out any text.' },
{ type: 'image_url', image_url: { url: 'data:image/png;base64,' + png } }] }] });
const choice = chat.body?.choices?.[0] ?? {};
answer = choice.message?.content ?? '';
finish = choice.finish_reason ?? String(chat.status);
if (answer.trim() || finish !== 'length') break;
}
console.log(` answer (${Math.round((Date.now() - t) / 1000)} s, finish=${finish}): ${JSON.stringify(answer.slice(0, 300))}`);
const low = answer.toLowerCase();
record('the answer names the red circle', /red|circle/.test(low));
record('the answer names the blue square', /blue|square/.test(low));
record('the answer names the green triangle', /green|triangle/.test(low));
record('the answer reads STRATA 42', low.includes('42'));
// ---- unload
const unloaded = await call(`/api/native/instances/${instance?.id}/unload`, 'POST', {});
record('unload', unloaded.status === 200, unloaded.status);
}
try { await main(); } catch (e) { record('run', false, e.stack ?? e); }
host.kill();
await delay(1500);
console.log(failed ? 'VISION-E2E FAIL' : 'VISION-E2E PASS');
process.exit(failed ? 1 : 0);
+2
View File
@@ -0,0 +1,2 @@
# Rust build output. Regenerated by `cargo tauri build`; never committed.
/target/
+5458
View File
File diff suppressed because it is too large. Load diff
+30
View File
@@ -0,0 +1,30 @@
[package]
name = "strata-desktop"
version = "0.2.1"
description = "Strata Desktop — local AI, tuned to one model."
edition = "2021"
rust-version = "1.77"
license = "MIT"
# Linux: the drop of a file on the window is read straight from the GTK drag signals (see hook_linux_drops in main.rs); the versions are the ones Tauri itself uses.
[target.'cfg(target_os = "linux")'.dependencies]
gtk = "0.18"
webkit2gtk = "2.0"
[build-dependencies]
tauri-build = { version = "2", features = [] }
[dependencies]
tauri = { version = "2", features = [] }
tauri-plugin-shell = "2"
tauri-plugin-updater = "2"
tauri-plugin-single-instance = "2"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
[profile.release]
codegen-units = 1
lto = true
opt-level = "s"
panic = "abort"
strip = true
+3
View File
@@ -0,0 +1,3 @@
fn main() {
tauri_build::build()
}
+15
View File
@@ -0,0 +1,15 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<!-- Tauri uses the system WKWebView, so this bundle needs no JIT entitlement of its own;
the web content process gets what it needs from the OS. The App Host runs as an ordinary
child process and needs no special rights. Kept minimal on purpose. -->
<key>com.apple.security.files.user-selected.read-write</key>
<true/>
<key>com.apple.security.network.client</key>
<true/>
<key>com.apple.security.network.server</key>
<true/>
</dict>
</plist>
@@ -0,0 +1 @@
{"core":{"default_permission":{"identifier":"default","description":"Default core plugins set.","permissions":["core:path:default","core:event:default","core:window:default","core:webview:default","core:app:default","core:image:default","core:resources:default","core:menu:default","core:tray:default"]},"permissions":{},"permission_sets":{},"global_scope_schema":null},"core:app":{"default_permission":{"identifier":"default","description":"Default permissions for the plugin.","permissions":["allow-version","allow-name","allow-tauri-version","allow-identifier","allow-bundle-type","allow-register-listener","allow-remove-listener","allow-supports-multiple-windows"]},"permissions":{"allow-app-hide":{"identifier":"allow-app-hide","description":"Enables the app_hide command without any pre-configured scope.","commands":{"allow":["app_hide"],"deny":[]}},"allow-app-show":{"identifier":"allow-app-show","description":"Enables the app_show command without any pre-configured scope.","commands":{"allow":["app_show"],"deny":[]}},"allow-bundle-type":{"identifier":"allow-bundle-type","description":"Enables the bundle_type command without any pre-configured scope.","commands":{"allow":["bundle_type"],"deny":[]}},"allow-default-window-icon":{"identifier":"allow-default-window-icon","description":"Enables the default_window_icon command without any pre-configured scope.","commands":{"allow":["default_window_icon"],"deny":[]}},"allow-exit":{"identifier":"allow-exit","description":"Enables the exit command without any pre-configured scope.","commands":{"allow":["exit"],"deny":[]}},"allow-fetch-data-store-identifiers":{"identifier":"allow-fetch-data-store-identifiers","description":"Enables the fetch_data_store_identifiers command without any pre-configured scope.","commands":{"allow":["fetch_data_store_identifiers"],"deny":[]}},"allow-identifier":{"identifier":"allow-identifier","description":"Enables the identifier command without any pre-configured scope.","commands":{"allow":["identifier"],"deny":[]}},"allow-name":{"identifier":"allow-name","description":"Enables the name command without any pre-configured scope.","commands":{"allow":["name"],"deny":[]}},"allow-register-listener":{"identifier":"allow-register-listener","description":"Enables the register_listener command without any pre-configured scope.","commands":{"allow":["register_listener"],"deny":[]}},"allow-remove-data-store":{"identifier":"allow-remove-data-store","description":"Enables the remove_data_store command without any pre-configured scope.","commands":{"allow":["remove_data_store"],"deny":[]}},"allow-remove-listener":{"identifier":"allow-remove-listener","description":"Enables the remove_listener command without any pre-configured scope.","commands":{"allow":["remove_listener"],"deny":[]}},"allow-set-app-theme":{"identifier":"allow-set-app-theme","description":"Enables the set_app_theme command without any pre-configured scope.","commands":{"allow":["set_app_theme"],"deny":[]}},"allow-set-dock-visibility":{"identifier":"allow-set-dock-visibility","description":"Enables the set_dock_visibility command without any pre-configured scope.","commands":{"allow":["set_dock_visibility"],"deny":[]}},"allow-supports-multiple-windows":{"identifier":"allow-supports-multiple-windows","description":"Enables the supports_multiple_windows command without any pre-configured scope.","commands":{"allow":["supports_multiple_windows"],"deny":[]}},"allow-tauri-version":{"identifier":"allow-tauri-version","description":"Enables the tauri_version command without any pre-configured scope.","commands":{"allow":["tauri_version"],"deny":[]}},"allow-version":{"identifier":"allow-version","description":"Enables the version command without any pre-configured scope.","commands":{"allow":["version"],"deny":[]}},"deny-app-hide":{"identifier":"deny-app-hide","description":"Denies the app_hide command without any pre-configured scope.","commands":{"allow":[],"deny":["app_hide"]}},"deny-app-show":{"identifier":"deny-app-show","description":"Denies the app_show command without any pre-configured scope.","commands":{"allow":[],"deny":["app_show"]}},"deny-bundle-type":{"identifier":"deny-bundle-type","description":"Denies the bundle_type command without any pre-configured scope.","commands":{"allow":[],"deny":["bundle_type"]}},"deny-default-window-icon":{"identifier":"deny-default-window-icon","description":"Denies the default_window_icon command without any pre-configured scope.","commands":{"allow":[],"deny":["default_window_icon"]}},"deny-exit":{"identifier":"deny-exit","description":"Denies the exit command without any pre-configured scope.","commands":{"allow":[],"deny":["exit"]}},"deny-fetch-data-store-identifiers":{"identifier":"deny-fetch-data-store-identifiers","description":"Denies the fetch_data_store_identifiers command without any pre-configured scope.","commands":{"allow":[],"deny":["fetch_data_store_identifiers"]}},"deny-identifier":{"identifier":"deny-identifier","description":"DenLine truncated
@@ -0,0 +1 @@
{}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 433 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1004 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 771 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 KiB

@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
<foreground android:drawable="@mipmap/ic_launcher_foreground"/>
<background android:drawable="@color/ic_launcher_background"/>
</adaptive-icon>
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 949 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.6 KiB

@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<color name="ic_launcher_background">#FFFFFF</color>
</resources>
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 504 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1019 B

Loaded 100 of 3933 files, more files were not shown because too many files have changed in this diff. Show more