Niko1221/Strata
1
0 Fork 0
mirror of https://github.com/Niko1221/Strata.git synced 2026-11-07 06:15:58 -10:00
Code Issues Packages Projects Releases 1 Wiki Activity

1 Release 73 Tags

RSS Feed
  • v0.2.12 146df3c468
    Compare

    Strata 0.2.12 Stable

    yindo released this 2026-11-07 05:59:26 -10:00 | 0 commits to main since this release

    📅 Originally published on GitHub: Sat, 07 Nov 2026 08:54:40 GMT
    🏷️ Git tag created: Sat, 07 Nov 2026 08:24:51 GMT

    Strata 0.2.12: a tuned DFlash2 drafter, faster prompts on 12 GB cards, Strix Halo and Mac, and tool-call fixes

    The app and the Terminal are 0.2.12. Control and every engine package are 0.1.58. The update feed is sequence 122.
    The DGX Spark (linux-aarch64) gets 0.2.12 too.

    Highlights

    • 12 GB cards (RTX 3060 class): UD-IQ4_XS-v31 prompts read about 3 times faster, 87 to 264 tokens per second at 2K.
    • Strix Halo (Windows): prompts x1.24 to x2.14 on UD-IQ4_XS-v31 and UD-Q4_K_XL.
    • Mac (M5 Max): output +10.3% to +12.7% and prompts +11.7% to +13.9% on UD-Q4_K_XL and UD-IQ4_XS-v31.
    • Tuned DFlash2 drafter: Strata's own copy of the helper model that guesses the next words; decode +2.4% to +5.0% on the RTX PRO 5000 and the DGX Spark.
    • Radeon R9700: 32K prompts +8.6% to +11.6%.
    • Tool calls: Claude Desktop's model name is accepted (#5312), plus fixes for #5409, #5276, #5281, #4622 and PRs #4828 and #4832.

    Speed against 0.2.11

    Tokens per second, 0.2.11 to 0.2.12. "Output" is the geometric mean over the kinds we tried where we have one; the worst kind is shown when it is a loss.

    Machine, model Prompt Output
    Mac M5 Max, UD-Q4_K_XL 4K 1,406 to 1,605; 32K 1,350 to 1,524 +12.7% (every kind up)
    Mac M5 Max, UD-IQ4_XS-v31 4K 1,440 to 1,649; 32K 1,373 to 1,543 +10.3% (agent -0.9%)
    Strix Halo, UD-IQ4_XS-v31 4K 690 to 1,220; 32K 1,107 to 1,377 code +10%, chat +3%; tool call -9.5%
    Strix Halo, UD-Q4_K_XL 4K 597 to 1,267; 32K 1,034 to 1,370 code +24%, chat +6%; tool call -5.8%, agent -2.5%
    RTX 3060 12 GB, UD-IQ4_XS-v31 2K 87 to 264; 32K 361 to 981 code +37%, agent +34%; JSON -18.9%
    RTX PRO 5000, IQ3_S 8K level (x1.002) +2.4%; agent -1.5%, chat -1.0%
    DGX Spark, IQ3_S see Known problems +3.1%; chat -3.3%
    DGX Spark, UD-IQ4_XS-v31 see Known problems +5.0%; chat -0.4%
    Radeon R9700, IQ3_S (Auto now MTP) 32K 3,076 to 3,304 code -10.5%, tool call -9.3%, JSON -2.3%
    Radeon R9700, UD-IQ4_XS-v31 (Auto now MTP) 32K 2,817 to 3,015 code -8.5%, JSON -4.8%, tool call -3.2%
    2 x RTX A6000, IQ3_S level split engine -0.40%, layer split +0.38%

    How it was measured: each machine ran 0.2.11 and the 0.2.12 candidate side by side, 2 to 3 rounds, a fresh engine start per run, the arms swapped in order each round, greedy answers of 300 tokens. Each version used the settings it writes for that machine, so the 0.2.12 column includes the new drafter choices. Speeds were not measured again on the final 0.1.58 packages; the engine source after the check changed only in the version number and two build fixes for old NVIDIA cards.

    What changed for you

    • Tuned DFlash2 drafter (0.34 GB, 0.52 GB on Intel Arc). Auto now uses it on: Mac (UD-Q4_K_XL, UD-IQ4_XS-v31; DFlash2 is now the Mac default for these, confidence 0.7); NVIDIA cards with every expert on the card and the DGX Spark (IQ3_S, UD-IQ4_XS-v31); Strix Halo (IQ3_S, UD-Q4_K_XL, UD-IQ4_XS, UD-IQ4_XS-v31); Intel Arc A-series (IQ3_XXS); 12 GB NVIDIA cards (UD-IQ4_XS-v31). Back: strata launch --drafting mtp.
    • Radeon gfx1201 (R9700, RX 9070): Auto starts MTP, not DFlash2, because 11 of 50 DFlash2 engines on 0.2.11 stopped on the prompt. Line shown: "DFlash2 is off by default on this graphics card for now; choose it in Drafting to use it." Back: --drafting dflash2.
    • 12 GB cards (Linux): the second start-up check no longer counts the engine's own memory as free, so it stops switching to the slower file cache. When it still switches it says: strata: expert reads now go through the file cache (X GiB of RAM free for them, Y GiB limit). On the RTX 3060, 0.2.11 switched in 3 of 3 starts and 0.2.12 in 0 of 3. Auto now starts MTP for IQ3_S and the tuned DFlash2 for UD-IQ4_XS-v31 on these cards. Back: STRATA_FILE_TIER_HONEST=0.
    • A download with the wrong checksum is kept as <name>.bad, not deleted: <name> does not match the pinned SHA-256 (it is <got>, Strata expects <want>); the file was kept as <name>.bad. Setup checks it again on the next run.
    • Strix Halo (Windows): the fused prompt kernels for UD-Q4_K_XL and the Q5_0 layers of UD-IQ4_XS-v31, and the grouped decode kernels, are on. Back: STRATA_GFX1151_DEFAULTS=0.
    • Mac: the prompt path overlaps host work with the GPU (same bits). Back: STRATA_METAL_PROMPT_HOSTPIPE=0, STRATA_PROMPT_MERGEH=0, STRATA_PROMPT_V6=0. Mac line: "On your Mac, DFlash2 was faster than MTP on this model in our tests."
    • Radeon gfx12, prompts of 30,000 tokens or more use the chunked GDN scan (32K +6.1%, 64K +6.7% on an R9700): strata: RDNA4 (gfx12): prompt chunks of 30,000+ tokens use the chunked GDN scan (STRATA_GDN_CHUNKED=0 turns it off).

    Fixed

    • Tool calls:
      • #5312: Claude Desktop's claude-5-5-sonnet on /v1/messages now uses the one loaded model.
      • #5409: <function=tools NAME> and a bare <function=tools> are read as the declared tool.
      • #5276: <toolcall> and <tool-call> openers are read when a declared tool follows.
      • #5281: an MCP tool written by its last segment is matched when exactly one tool ends that way.
      • #4828 (gdevenyi): forced tool_choice with several tools; 24 of 24 runs now, 0 of 24 before.
      • #4832 (gdevenyi): Anthropic tool results follow tool_use_id.
      • #4622: new setting undeclared_tool_calls (text by default, call is opt-in).
    • Engine fixes from pull requests:
      • Copy ordering and lost adaptive swaps (#5096, #4844).
      • Row-table over-read that could fault long CUDA prompts (#5108).
      • A first-request hang with --pipeline-windows 2 on two AMD cards (#5149); not run on hardware.
      • Speed and time figures in the server reports (#5318, #5307, #4006); a short request admitted into a slot now reads 125, not 215, tokens per second because admission time is counted.
      • Hyper-connection rows copied per stage: 414 MiB less per card on two A6000 (#3559).
      • A window of slots kept in every expert pool, so 12, 20 and 30 GiB RAM splits start (#5158).
      • File tier defaults stay off when every expert is on the cards (#4972).
    • Mac install (#5390, cwf75): a fresh install of UD-Q4_K_XL or UD-IQ4_XS-v31 from the Mac app or Terminal stopped at "Preparing your model" ("the desktop app's installer only runs its bundled preparation tools"); the installer now runs the bundled Mac tools in metal/tools. On an M5 Max a fresh Terminal install of UD-Q4_K_XL prepared the model, fetched the tuned drafter and answered plain, two-turn and tool chats.
    • Builds: the CUDA 12 and 13 engines build again for old NVIDIA cards (sm_60 to 75); MSVC and the Spark link failures; the Docker image build.

    Opt-in switches (all off unless set)

    • STRATA_PF_FUSED_Q50=1 (Radeon gfx12): Q5_0 layers on a fused kernel; prompts 2K +44%, 8K +40% on an R9700.
    • STRATA_IQ4_W4A8=1|2 (NVIDIA, cctan): int8 kernel for IQ4_XS and IQ4_NL experts, same output as MMQ.
    • STRATA_SELECT_TWO_PASS=1 (NVIDIA, bitaccesscomau): exact two-pass selection for long prompts; 128K +1.65%.
    • STRATA_ADAPT_FETCH=1|2 (CUDA, sergqwer): fetched misses go into the card's cache; decode +10.2% with =2 in one setup.
    • STRATA_ADAPT_OVERLAP=1 with --peer-device (bulhenvaster): the peer's copies land while the next window runs.
    • STRATA_PF_PEER_CPU=1 (bulhenvaster): the CPU helps beside a peer GPU; on two A6000, 12 follow-up turns took 14.28 s against 20.50 s.
    • STRATA_VNNI_TIER=1 (Michel Pollet): 512-bit expert rows on CPUs without AVX-512-VBMI.
    • undeclared_tool_calls: "call" (or STRATA_UNDECLARED_TOOL_CALLS=call): return a call of an undeclared tool as a tool call.

    Known problems

    • Peer mode (--peer-device, two cards) can fault on its first request, in about 4% of fresh starts (also in 0.2.11); the all-cards split engine is not affected. Cause not found.
    • Radeon gfx1201: DFlash2 chosen by hand can stall on the prompt (11 of 50 engines on 0.2.11). The engine stops itself and the next request restarts it.
    • DGX Spark, UD-IQ4_XS-v31: the first long prompt after a start can read slowly (590 to 670 tokens per second against 1,154 to 1,235 when the pages are cached); also in 0.2.11.
    • Losses: RTX 3060 UD-IQ4_XS-v31 JSON extraction -18.9%; Strix Halo tool calls -9.5% on UD-IQ4_XS-v31 (measured to come from the tuned drafter) and -5.8% on UD-Q4_K_XL (its answers differ from 0.2.11's, so that ratio is rougher).
    • Tool-call issues still open: #4923, #5226, #5282, #5110, #5039, #4949, #4981, #4704, #4077, and #5153.

    Updating

    • App: the in-app update offers 0.2.12, or download the installer below.
    • Terminal: run strata update. From a 0.2.11 install it updated Control and the engine from 0.1.57 to 0.1.58 on Windows (RTX 5070 and Strix Halo) and Linux (Intel Arc A750). strata components rollback returns to the 0.1.57 engine (Control stays 0.1.58, so strata --version still says 0.2.12), and a plain strata update brings 0.1.58 back.
    • The tuned drafter downloads once (0.34 GB) the first time Auto needs it. Offline, Strata uses the standard drafter or MTP. Nothing downloads if you use MTP.
    • Mac: the app, the Terminal and the DeepSeek engine are signed with Developer ID and notarized (Apple accepted all four files), and the app and the dmg pass Gatekeeper.
    • What we ran on the release files: fresh installs from the signed 0.2.12 feed, a model launch and plain, two-turn and tool chats on Windows (RTX 5070 and Strix Halo), Linux (RTX PRO 5000, Radeon R9700 with the Qwen and DeepSeek engines, Intel Arc A750), the DGX Spark (Qwen and DeepSeek) and the Linux AppImage. The tuned drafter came from the feed and passed its checks on the RTX PRO 5000, the DGX Spark, the Strix Halo and the Intel Arc (its own file). Not run on a matching card: CUDA 12 (Pascal), RX 7900, RX 6000 and Strix Halo on Linux; the Mac files beyond Apple's checks and a start of the app.

    Downloads

    42 files, 9.70 GiB in all. Names, sizes and SHA-256 are in SHA256SUMS.txt.

    • App installers for Windows, Linux (AppImage) and Mac (dmg), with .sig files.
    • Control 0.1.58 for Linux, Windows, Mac and the DGX Spark.
    • Qwen and DeepSeek engines 0.1.58 for Linux, Windows, Mac and the Spark.
    • Two drafter files (...strata1.q4_0-selq4.pack, 339,694,592 bytes; ...bf16sel.pack, 522,458,112 bytes).
    • The feeds, latest.json, install.sh, install.ps1, SHA256SUMS.txt.

    Thanks

    • Inco AI: incoai/Qwen3.8-Flash-Next-DFlash2 (Apache-2.0), the base of the tuned drafter; the licence and a notice of the change ship in data/dflash2-tuned/.
    • gdevenyi: #4828, #4832. bitaccesscomau: #4119. sergqwer: #2964. cctan: #4871. Phlimosx: #4972. bulhenvaster: #2720, #2721. midhatn: #5158. Michel Pollet: #5120.
    • AlanJager: #5096. uvlad23: #4844. quantforge-br: #5108. cloud4t0r: #5149. YusenZhang0601: #5318. shoucandanghehe: #5307. Vop3p: #4006. martifaj: #4910. havok2-htwo: #3559. bsorensen110: #5243.
    • John Fink, nazeshinjite and everyone named in 0.2.11: as in 0.2.11; their work is still in this release.

    Strata is free and open source. If it runs well on your PC, a coffee keeps the work on it going:

    Buy Me A Coffee

    Downloads