Engine 0.1.35: Windows AMD uses its bundled HIP runtime (#468 #461), the low-RAM resident mode on Windows 32 GB PCs (#467), fixes #460 #459 #446 #447 #457 #448 #444; setup requires it

README and docs/MODELS.md said only "Weaker outside coding". #438 measured Chinese answers wrong or looping on the
Coder where English was fine; both now say plainly that this includes Chinese and other CJK text and point to the
sizes that keep every expert (Q2_0, IQ2_XS, IQ3_S).

.gitattributes

@@ -4,3 +4,6 @@ bench/prompts/** -text
 # launchers keep their platform line endings wherever the repository is cloned
 *.sh text eol=lf
 *.bat text eol=crlf
+
+# copies of real packs' native_experts.txt, parsed byte for byte by tests/core/expert_layout_test.cpp
+tests/data/native_experts/** -text

.github/FUNDING.yml

@@ -0,0 +1 @@
+buy_me_a_coffee: strataengine

.gitignore

@@ -37,6 +37,7 @@ td.log
 /strata-*.json
 /strata-*.log
 /build-strata.bat
+/.strata-mcp/
 
 # image encoder (setup.py --vision): build output and the generated build script
 build-vision*/
@@ -49,3 +50,4 @@ build-vision.bat
 *.done
 /third_party/*.zip
 
+/.rocm-win/

AGENTS.md

@@ -0,0 +1,23 @@
+# AGENTS.md
+
+Strata runs the Qwen3.8-Flash-Next mixture-of-experts model (and its Coder, Swift 1.5 and Unsloth variants) on a
+normal PC: one NVIDIA or AMD graphics card plus system RAM, on Windows or Linux. It has a C++/CUDA/HIP engine
+(`src/`, `include/`), a Python server with an OpenAI- and Anthropic-compatible API and a web app (`serve/`), and a
+one-click installer (`setup.py`, started by `START-HERE.bat` / `setup.sh`).
+
+## Installing Strata for a user
+
+Follow **[docs/AI_SETUP.md](docs/AI_SETUP.md)**: check the PC, pick the model by RAM, run setup non-interactively,
+start and verify the server, and connect the user's apps. Never expose the server beyond `127.0.0.1` without
+`--api-key`. As an alternative to shell commands, Strata's MCP server ([docs/MCP_SERVER.md](docs/MCP_SERVER.md))
+offers the same steps as tools.
+
+## Working on the code
+
+- How the engine works, every measured number, the API and all settings: [docs/DETAILS.md](docs/DETAILS.md) and
+  the [paper](docs/paper/Strata-Paper.pdf).
+- AMD (HIP) build and validation: [docs/AMD_HIP.md](docs/AMD_HIP.md); multi-GPU: [docs/MULTI_GPU.md](docs/MULTI_GPU.md).
+- Setup's own tests run without a GPU or downloads: `python tools/test_setup_<name>.py` (for example
+  `tools/test_setup_amd.py`, `tools/test_setup_choices.py`).
+- Keep the docs' style: plain words, measured numbers with what they were measured on, no claims without a
+  measurement.

CMakeLists.txt

@@ -8,7 +8,7 @@
 # below refuses anything older: silently building for a pre-Ampere arch would produce a binary that runs
 # and is wrong about its own performance, which is the worst of both outcomes.
 cmake_minimum_required(VERSION 3.24)
-project(strata VERSION 0.1.30 LANGUAGES CXX)
+project(strata VERSION 0.1.35 LANGUAGES CXX)
 add_compile_definitions(STRATA_VERSION="${PROJECT_VERSION}")
 # issue #31: release builds keep a PDB beside the exe (not shipped in it), so a minidump from a stalled engine can
 # be read back to the exact source line
@@ -33,8 +33,13 @@ if(NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
 endif()
 
 option(STRATA_ENABLE_CUDA "Build the CUDA targets (Phase 2+)" OFF)
-option(STRATA_ENABLE_HIP "Build the HIP targets for wave32 AMD GPUs (gfx1100, gfx1201)" OFF)
+option(STRATA_ENABLE_HIP "Build the HIP targets for wave32 AMD GPUs (gfx1100, gfx1201; gfx1030 unvalidated)" OFF)
 option(STRATA_PREFILL_MMQ "Build the optional GGML MMQ prefill path for HIP" OFF)
+# CUDA: MMQ prompt kernels for Q4_K / Q5_K / Q5_1 (Unsloth's UD-Q4_K_XL experts).  Off by default: the engine loads
+# every kernel when it starts (CUDA_MODULE_LOADING=EAGER), so these took ~5 MB of VRAM - 1-3 expert slots - from every
+# model, and the UD-Q4_K_XL prompt is SSD-bound with or without them (docs/UNSLOTH_Q4.md)
+option(STRATA_MMQ_KQUANTS "Build the CUDA MMQ prompt kernels for Q4_K / Q5_K / Q5_1 experts" OFF)
+option(STRATA_ORCA_Q4KS_MMQ "Build the GGML Q5_0 MMQ instance for OrcaRouter Q4_K_S" OFF)
 if(STRATA_PREFILL_MMQ AND NOT STRATA_ENABLE_HIP)
   message(FATAL_ERROR "STRATA_PREFILL_MMQ requires STRATA_ENABLE_HIP=ON")
 endif()
@@ -68,6 +73,7 @@ if(STRATA_ENABLE_CUDA)
   # `enable_language(CUDA)` gives the .cu compile rules but NOT the imported targets, so `CUDA::cudart` has to
   # be brought in explicitly for the host-side tests that call cudaMalloc themselves.
   find_package(CUDAToolkit REQUIRED)
+  option(STRATA_EXPERIMENTAL_SM60 "community build for GPUs below compute capability 7.5: Pascal sm_60, Volta sm_70 (#236)" OFF)
   if(NOT DEFINED CMAKE_CUDA_ARCHITECTURES)
     set(CMAKE_CUDA_ARCHITECTURES 120 CACHE STRING "CUDA architectures" FORCE)
   endif()
@@ -79,16 +85,22 @@ if(STRATA_ENABLE_CUDA)
     if(_base STREQUAL "native" OR _base STREQUAL "all" OR _base STREQUAL "all-major")
       continue()
     endif()
-    # STRATA_EXPERIMENTAL_SM60: a Pascal (sm_60, GP100-class) build.  It sits below every architecture
-    # these kernels were written for, so two intrinsics are replaced for it - `__dp4a` (6.1+) and
-    # `__nanosleep` (7.0+) - in include/strata/kernels/dp4a.hpp, which is bit-exact for the first and
-    # merely a shorter pause for the second.  RTX 20/30/40/50 cards must use the release engine instead.
+    # STRATA_EXPERIMENTAL_SM60: a Pascal (sm_60, GP100-class) or Volta (sm_70, V100 / Titan V, #236) build.
+    # These sit below every architecture the kernels were written for, so two intrinsics are replaced below
+    # their arch - `__dp4a` (6.1+) and `__nanosleep` (7.0+) - in include/strata/kernels/dp4a.hpp, which is
+    # bit-exact for the first and merely a shorter pause for the second (sm_70 has both).  Community-tested,
+    # not in the ready-made engine; RTX 20/30/40/50 cards must use the release engine instead.  It needs a CUDA
+    # 12.x toolkit: CUDA 13 dropped sm_60 and sm_70 (nvcc fails CMake's compiler test with "compute_70").
     if(_base LESS 75 AND NOT STRATA_EXPERIMENTAL_SM60)
       message(FATAL_ERROR
         "Strata needs an NVIDIA GPU of compute capability 7.5 or newer (RTX 20 / 30 / 40 / 50 series), but "
-        "CMAKE_CUDA_ARCHITECTURES is '${CMAKE_CUDA_ARCHITECTURES}'.  (A Pascal build is "
+        "CMAKE_CUDA_ARCHITECTURES is '${CMAKE_CUDA_ARCHITECTURES}'.  (A Pascal sm_60 or Volta sm_70 build is "
         "-DSTRATA_EXPERIMENTAL_SM60=ON; upstream does not support it.)")
     endif()
+    if(_base LESS 60)
+      message(FATAL_ERROR "CMAKE_CUDA_ARCHITECTURES '${CMAKE_CUDA_ARCHITECTURES}': even the experimental build "
+                          "needs compute capability 6.0 (sm_60) or newer")
+    endif()
     # #220 / #224: RTX 50 engines built with CUDA 12.8 crashed in the prompt path on Linux (13.0 and newer did not)
     if(_base GREATER_EQUAL 120 AND CMAKE_CUDA_COMPILER_VERSION VERSION_LESS 13.0)
       message(WARNING "Building for sm_${_base} (RTX 50) with CUDA ${CMAKE_CUDA_COMPILER_VERSION}: engines built with "
@@ -148,6 +160,10 @@ if(STRATA_BUILD_TESTS)
   add_executable(gguf_reader_test src/artifact/gguf_reader_test.cpp)
   target_link_libraries(gguf_reader_test PRIVATE strata_artifact)
   add_test(NAME gguf_reader_test COMMAND gguf_reader_test)
+  # a split model's shards and their joint view: missing shard, split keys, duplicates, truncation (synthetic)
+  add_executable(gguf_split_test tests/core/gguf_split_test.cpp)
+  target_link_libraries(gguf_split_test PRIVATE strata_artifact)
+  add_test(NAME gguf_split_test COMMAND gguf_split_test)
 endif()
 
 # ---- strata_plan ----------------------------------------------------------
@@ -166,6 +182,10 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
   add_library(strata_core STATIC src/core/device.cu src/core/pinned.cu src/platform/memory.cpp src/core/graph.cpp src/core/weights.cpp
                                src/core/layout.cpp)
   target_include_directories(strata_core PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/include)
+  if(STRATA_ENABLE_CUDA AND STRATA_EXPERIMENTAL_SM60)
+    # the runtime device check (src/core/device.cu) admits the cards this build was made for (#236)
+    target_compile_definitions(strata_core PRIVATE STRATA_EXPERIMENTAL_SM60=1)
+  endif()
   # The selected runtime is PUBLIC because graph.cpp is a HOST translation unit that calls cudaGraphLaunch and
   # cudaEventQuery: the .cu sources get cudart through the CUDA language rules, a .cpp does not, and the
   # linker error ("unresolved external symbol cudaGraphInstantiate") names the function rather than the cause.
@@ -254,6 +274,15 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
     target_include_directories(qsa_prompt_attn_parity PRIVATE ${CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES})
     target_link_libraries(qsa_prompt_attn_parity PRIVATE strata_kernels CUDA::cudart)
   endif()
+  # the QSA selection (block scores on the warp kernel and the tensor-core one, top-k): a GPU bench with a
+  # scorer-vs-scorer comparison and an FP64 accuracy gate (synthetic, no model); the ctest runs a 32K context under a
+  # 262144-cell capacity (the top-k dispatch follows the capacity) and exits non-zero on a failed gate
+  if((STRATA_BUILD_TESTS OR STRATA_PARITY_PROMPT_ATTN) AND STRATA_ENABLE_HIP)
+    add_executable(qsa_select_bench src/kernels/qsa_select_bench.cpp)
+    set_source_files_properties(src/kernels/qsa_select_bench.cpp PROPERTIES LANGUAGE HIP)
+    target_link_libraries(qsa_select_bench PRIVATE strata_kernels strata_hip_runtime)
+    add_test(NAME qsa_select_bench COMMAND qsa_select_bench 32768 256 3 262144)
+  endif()
   # plan v0.3 P6: llama.cpp's block layouts and codebook grids for the i-quant formats (MIT, third_party/ggml)
   target_include_directories(strata_kernels PRIVATE third_party/ggml)
   # Match the pinned llama.cpp CUDA arithmetic contract for this adapter only.
@@ -321,6 +350,24 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
     add_test(NAME file_expert_source_test COMMAND file_expert_source_test)
     set_tests_properties(file_expert_source_test PROPERTIES TIMEOUT 60)
   endif()
+  if(STRATA_BUILD_TESTS AND STRATA_NATIVE_EXPERTS)
+    # native_experts.txt v2-v4 (the real packs' copies and the per-role shard column) and the arena filled from
+    # synthetic GGUF shards at the real expert dimensions
+    add_executable(expert_layout_test tests/core/expert_layout_test.cpp)
+    target_link_libraries(expert_layout_test PRIVATE strata_engine)
+    add_test(NAME expert_layout_test COMMAND expert_layout_test ${CMAKE_CURRENT_SOURCE_DIR}/tests/data/native_experts)
+    set_tests_properties(expert_layout_test PROPERTIES TIMEOUT 60)
+    # #326: a native pack's BF16 (--compat-bf16) PLE key stays in the arena; a quantized key row loads as before
+    add_executable(native_dense_ple_key_test tests/core/native_dense_ple_key_test.cpp)
+    target_link_libraries(native_dense_ple_key_test PRIVATE strata_engine)
+    add_test(NAME native_dense_ple_key_test COMMAND native_dense_ple_key_test)
+    set_tests_properties(native_dense_ple_key_test PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 60)
+    # #369: per-layer admission on a sized (native pack) cache starts each layer in its own range
+    add_executable(expert_cache_per_layer_test tests/core/expert_cache_per_layer_test.cpp)
+    target_link_libraries(expert_cache_per_layer_test PRIVATE strata_engine)
+    add_test(NAME expert_cache_per_layer_test COMMAND expert_cache_per_layer_test)
+    set_tests_properties(expert_cache_per_layer_test PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 60)
+  endif()
   if(STRATA_BUILD_CONVERSATION_TESTS)
     add_executable(conversation_snapshot_test src/core/conversation_snapshot_test.cpp)
     target_link_libraries(conversation_snapshot_test PRIVATE strata_engine)
@@ -379,6 +426,15 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
       add_test(NAME hip_handoff COMMAND hip_handoff)
       list(APPEND _strata_hip_tests hip_handoff)
     endif()
+    if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/tests/hip/mapped_alias.cpp")
+      # #325: reports what a hipHostGetDevicePointer alias is on this stack (on Windows: the host pointer, and a
+      # D2D copy into it does not land - hip_handoff's timeout there). A diagnosis: exits 0 whenever it ran.
+      add_executable(hip_mapped_alias tests/hip/mapped_alias.cpp)
+      target_link_libraries(hip_mapped_alias PRIVATE strata_kernels strata_hip_runtime)
+      add_test(NAME hip_mapped_alias COMMAND hip_mapped_alias)
+      set_tests_properties(hip_mapped_alias PROPERTIES SKIP_RETURN_CODE 77)
+      list(APPEND _strata_hip_tests hip_mapped_alias)
+    endif()
     if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/tests/hip/native_qsa_score.cpp")
       add_executable(hip_native_qsa_score tests/hip/native_qsa_score.cpp)
       target_link_libraries(hip_native_qsa_score PRIVATE strata_kernels strata_hip_runtime)
@@ -391,6 +447,17 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
       add_test(NAME hip_prefill_native_batch COMMAND hip_prefill_native_batch)
       list(APPEND _strata_hip_tests hip_prefill_native_batch)
     endif()
+    # S6: the AMD fast router against the portable kernel, bitwise
+    add_executable(hip_router_fast tests/hip/router_fast.cpp)
+    target_link_libraries(hip_router_fast PRIVATE strata_kernels strata_hip_runtime)
+    add_test(NAME hip_router_fast COMMAND hip_router_fast)
+    list(APPEND _strata_hip_tests hip_router_fast)
+    # S6: the RDNA4 matrix-core prompt attention (STRATA_HIP_WMMA) against the FP32 kernel and FP64; skips off gfx12
+    add_executable(hip_prompt_attn_wmma src/kernels/qsa_prompt_attn_parity.cpp)
+    target_link_libraries(hip_prompt_attn_wmma PRIVATE strata_kernels strata_hip_runtime)
+    add_test(NAME hip_prompt_attn_wmma COMMAND hip_prompt_attn_wmma 32768 2048 3)
+    set_tests_properties(hip_prompt_attn_wmma PROPERTIES TIMEOUT 600 SKIP_RETURN_CODE 77)
+    list(APPEND _strata_hip_tests hip_prompt_attn_wmma)
     if(STRATA_HIPBLASLT_AVAILABLE AND EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/tests/hip/prefill_hipblaslt_gemm.cpp")
       add_executable(hip_prefill_hipblaslt_gemm tests/hip/prefill_hipblaslt_gemm.cpp)
       target_include_directories(hip_prefill_hipblaslt_gemm PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/src/prefill")
@@ -426,6 +493,49 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
   add_executable(iq_parity src/kernels/iq_parity.cpp)
   target_include_directories(iq_parity PRIVATE ${_strata_gpu_include_directories})
   target_link_libraries(iq_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
+  # TODO 24: the i-quant fixtures are generated, not shipped.  tools/iq_fixture.py needs python3 + numpy
+  # and the repository's vendored gguf-py (no pip installs); it writes deterministic fixtures into the
+  # BUILD directory (the source tree is not touched).  The CTest fixture makes generation-to-test order
+  # explicit even under parallel execution; a fixture-generation failure fails iq_parity with it.
+  find_package(Python3 COMPONENTS Interpreter QUIET)
+  # the generator needs an interpreter WITH numpy: probe the configured one, the setup's .venv, and the
+  # system python3; -DIQ_FIXTURE_PYTHON=... overrides the search outright.
+  set(_iq_python "")
+  set(_iq_candidates "")
+  if(IQ_FIXTURE_PYTHON)
+    list(APPEND _iq_candidates ${IQ_FIXTURE_PYTHON})
+  else()
+    if(Python3_Interpreter_FOUND)
+      list(APPEND _iq_candidates ${Python3_EXECUTABLE})
+    endif()
+    list(APPEND _iq_candidates "${CMAKE_CURRENT_SOURCE_DIR}/.venv/bin/python"
+                                "${CMAKE_CURRENT_SOURCE_DIR}/.venv/Scripts/python.exe"
+                                /usr/bin/python3
+                                python3)
+  endif()
+  foreach(_cand IN LISTS _iq_candidates)
+    if(_cand STREQUAL "")
+      continue()
+    endif()
+    execute_process(COMMAND ${_cand} -c "import numpy" OUTPUT_QUIET ERROR_QUIET RESULT_VARIABLE _iq_probe)
+    if(_iq_probe EQUAL 0)
+      set(_iq_python "${_cand}")
+      break()
+    endif()
+  endforeach()
+  if(NOT _iq_python STREQUAL "")
+    add_test(NAME iq_parity_fixtures
+             COMMAND ${_iq_python} ${CMAKE_CURRENT_SOURCE_DIR}/tools/iq_fixture.py
+                     --out ${CMAKE_CURRENT_BINARY_DIR}/iq_fixture)
+    # exit 3: numpy or the vendored gguf-py is missing (a trimmed third_party) - a skip, not a failure
+    set_tests_properties(iq_parity_fixtures PROPERTIES FIXTURES_SETUP iq_fixture_data SKIP_RETURN_CODE 3)
+    add_test(NAME iq_parity COMMAND iq_parity ${CMAKE_CURRENT_BINARY_DIR}/iq_fixture)
+    set_tests_properties(iq_parity PROPERTIES FIXTURES_REQUIRED iq_fixture_data SKIP_RETURN_CODE 3)
+  else()
+    message(STATUS "STRATA tests: no python3 with numpy was found (the configured interpreter and .venv "
+                   "were probed) - the iq_parity fixtures and its test are not registered; the fixtures "
+                   "need python3 with numpy and the repository's vendored gguf-py")
+  endif()
   if(STRATA_ENABLE_CUDA)
     # native_mmvq's multi_exact contract, bitwise: a T-column call against T single-column calls on the same Q8_1
     # bytes, with the flag off as the negative control (synthetic, GPU, no model; the control assumes 32-lane warps)
@@ -434,6 +544,12 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
     target_link_libraries(mmvq_multi_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
     add_test(NAME mmvq_multi_parity COMMAND mmvq_multi_parity)
   endif()
+  # the IQ kernels that decode a weight part once for every column/entry, bitwise vs the per-column ones
+  # (GPU, synthetic, no model; --bench adds old/new timings)
+  add_executable(iq_multi_parity src/kernels/iq_multi_parity.cpp)
+  target_include_directories(iq_multi_parity PRIVATE ${_strata_gpu_include_directories})
+  target_link_libraries(iq_multi_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
+  add_test(NAME iq_multi_parity COMMAND iq_multi_parity)
   add_executable(shared_expert_parity src/kernels/shared_expert_parity.cpp)
   target_include_directories(shared_expert_parity PRIVATE ${_strata_gpu_include_directories})
   target_link_libraries(shared_expert_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
@@ -549,6 +665,13 @@ if(STRATA_ENABLE_CUDA OR STRATA_ENABLE_HIP)
   target_link_libraries(cvec_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
   add_test(NAME cvec_parity COMMAND cvec_parity)
 
+  # The rewritten per-hit and grouped expert kernels vs the previous ones, bitwise (GPU, synthetic, no model).
+  # `--bench` adds previous/new timings on engine-like shapes.
+  add_executable(s2_expert_grouped_parity src/kernels/s2_expert_grouped_parity.cpp)
+  target_include_directories(s2_expert_grouped_parity PRIVATE ${_strata_gpu_include_directories})
+  target_link_libraries(s2_expert_grouped_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
+  add_test(NAME s2_expert_grouped_parity COMMAND s2_expert_grouped_parity --selftest)
+
   # Plan v0.3 P3: native MMVQ for 1..8 tokens (bitwise selftest + --bench timing); synthetic data, no model.
   if(STRATA_BUILD_TESTS AND EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/bench/micro/native_mmvq_multi.cpp")
     add_executable(native_mmvq_multi bench/micro/native_mmvq_multi.cpp)
@@ -679,6 +802,9 @@ set(STRATA_GGML_DIR "" CACHE PATH "llama.cpp checkout to take ggml from (empty:
 if(STRATA_PREFILL_MMQ AND NOT STRATA_NATIVE_EXPERTS)
   message(FATAL_ERROR "STRATA_PREFILL_MMQ requires STRATA_NATIVE_EXPERTS=ON for GGML sources")
 endif()
+if(STRATA_ORCA_Q4KS_MMQ AND NOT STRATA_NATIVE_EXPERTS)
+  message(FATAL_ERROR "STRATA_ORCA_Q4KS_MMQ requires STRATA_NATIVE_EXPERTS=ON for GGML sources")
+endif()
 if(STRATA_NATIVE_EXPERTS)
   enable_language(C)
   set(BUILD_SHARED_LIBS OFF CACHE BOOL "" FORCE)
@@ -713,18 +839,25 @@ if(STRATA_NATIVE_EXPERTS)
   # part of strata_kernels_cpu (the pool calls these and they call its CPU probe: one library, so GNU ld's
   # one-pass static linking has no cycle to resolve)
   target_sources(strata_kernels_cpu PRIVATE src/kernels/cpu/native_expert.cpp src/kernels/cpu/iq_avx512.cpp
-                                           src/kernels/cpu/iq_avx2.cpp)
+                                           src/kernels/cpu/iq_avx2.cpp src/kernels/cpu/kq_avx2.cpp)
   target_include_directories(strata_kernels_cpu PRIVATE third_party/ggml)
   if(MSVC)
     set_source_files_properties(src/kernels/cpu/iq_avx512.cpp PROPERTIES COMPILE_OPTIONS "/arch:AVX512")
     set_source_files_properties(src/kernels/cpu/iq_avx2.cpp PROPERTIES COMPILE_OPTIONS "/arch:AVX2")
+    set_source_files_properties(src/kernels/cpu/kq_avx2.cpp PROPERTIES COMPILE_OPTIONS "/arch:AVX2")
   else()
     set_source_files_properties(src/kernels/cpu/iq_avx512.cpp PROPERTIES COMPILE_OPTIONS
                                 "-mavx512f;-mavx512bw;-mavx512vl;-mavx512dq;-mavx512vnni;-mavx512vbmi;-mf16c;-mfma")
     set_source_files_properties(src/kernels/cpu/iq_avx2.cpp PROPERTIES COMPILE_OPTIONS "-mavx2;-mfma;-mf16c")
+    set_source_files_properties(src/kernels/cpu/kq_avx2.cpp PROPERTIES COMPILE_OPTIONS "-mavx2;-mfma;-mf16c")
   endif()
   target_link_libraries(strata_kernels_cpu PUBLIC ggml-cpu ggml-base)
   target_compile_definitions(strata_kernels_cpu PUBLIC STRATA_NATIVE_EXPERTS=1)
+  set(_strata_mmq_types q2_0 iq2_xxs iq2_xs iq2_s iq3_xxs iq3_s iq4_nl iq4_xs q8_0)
+  # #296: OrcaRouter Q4_K_S's Q5_0 experts (Q4_K and Q5_1 come with STRATA_MMQ_KQUANTS on CUDA, S5's branch)
+  if(STRATA_ORCA_Q4KS_MMQ)
+    list(APPEND _strata_mmq_types q5_0)
+  endif()
   if(STRATA_ENABLE_CUDA)
     # prompt-speed plan step 2b: llama.cpp's MMQ kernels (ggml-cuda, MIT) for the prompt path's experts, compiled
     # from the same checkout as ggml above - only mmq.cuh's instances for the packs' types and the quantizer, with
@@ -736,7 +869,14 @@ if(STRATA_NATIVE_EXPERTS)
     endif()
     set(_strata_mmq ${_strata_gg}/src/ggml-cuda)
     add_library(strata_mmq STATIC src/prefill/moe_mmq.cu src/prefill/ggml_cuda_host.cu ${_strata_mmq}/quantize.cu)
-    foreach(_t q2_0 iq2_xxs iq2_xs iq2_s iq3_xxs iq3_s iq4_nl iq4_xs q8_0)   # q8_0: the drafter's matrices (E-9)
+    # q8_0: the drafter's matrices (E-9); STRATA_ORCA_Q4KS_MMQ adds q5_0 (#296, in _strata_mmq_types), and
+    # STRATA_MMQ_KQUANTS q4_k q5_k q5_1, the experts of Unsloth's UD-Q4_K_XL (docs/UNSLOTH_Q4.md; the dispatch
+    # follows eddoursul/Strata 7706c1b)
+    set(_strata_mmq_cuda_types ${_strata_mmq_types})
+    if(STRATA_MMQ_KQUANTS)
+      list(APPEND _strata_mmq_cuda_types q4_k q5_k q5_1)
+    endif()
+    foreach(_t IN LISTS _strata_mmq_cuda_types)
       target_sources(strata_mmq PRIVATE ${_strata_mmq}/template-instances/mmq-instance-${_t}.cu)
     endforeach()
     target_include_directories(strata_mmq PRIVATE ${_strata_gg}/include ${_strata_gg}/src ${_strata_mmq}
@@ -746,8 +886,21 @@ if(STRATA_NATIVE_EXPERTS)
       target_compile_options(strata_mmq PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:-Xcompiler=/bigobj>)
     endif()
     target_link_libraries(strata_mmq PUBLIC ggml-base ${_strata_gpu_runtime_target})
+    if(STRATA_MMQ_KQUANTS)
+      target_compile_definitions(strata_mmq PRIVATE STRATA_MMQ_KQUANTS=1)
+    endif()
+    if(STRATA_ORCA_Q4KS_MMQ)
+      target_compile_definitions(strata_mmq PRIVATE STRATA_ORCA_Q4KS_MMQ=1)
+    endif()
     target_link_libraries(strata_prefill PUBLIC strata_mmq)
     target_compile_definitions(strata_prefill PUBLIC STRATA_PREFILL_MMQ=1)
+    if(STRATA_BUILD_TESTS AND STRATA_MMQ_KQUANTS)
+      # the UD-Q4_K_XL expert formats through MMQ against ggml's dequantized weights (synthetic, a few seconds)
+      add_executable(prefill_mmq_kquant_test tests/cuda/prefill_mmq_kquant_test.cpp)
+      target_link_libraries(prefill_mmq_kquant_test PRIVATE strata_mmq ggml-base ${_strata_gpu_runtime_target})
+      add_test(NAME prefill_mmq_kquant_test COMMAND prefill_mmq_kquant_test)
+      set_tests_properties(prefill_mmq_kquant_test PROPERTIES TIMEOUT 120)
+    endif()
   elseif(STRATA_PREFILL_MMQ)
     # Compile only the pinned GGML MMQ and activation-quantizer sources used by prefill. This uses
     # common.cuh's native HIP path without enabling a second GGML HIP backend.
@@ -764,7 +917,7 @@ if(STRATA_NATIVE_EXPERTS)
       src/prefill/moe_mmq.cu
       src/prefill/ggml_cuda_host.cu
       "${_strata_mmq}/quantize.cu")
-    foreach(_t q2_0 iq2_xxs iq2_xs iq2_s iq3_xxs iq3_s iq4_nl iq4_xs q8_0)
+    foreach(_t IN LISTS _strata_mmq_types)
       list(APPEND _strata_mmq_sources "${_strata_mmq}/template-instances/mmq-instance-${_t}.cu")
     endforeach()
     set_source_files_properties(${_strata_mmq_sources} PROPERTIES LANGUAGE HIP)
@@ -773,6 +926,9 @@ if(STRATA_NATIVE_EXPERTS)
       PRIVATE "${_strata_gg}/include" "${_strata_gg}/src" "${_strata_mmq}"
       PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/include)
     target_compile_definitions(strata_mmq PRIVATE GGML_USE_HIP=1)
+    if(STRATA_ORCA_Q4KS_MMQ)
+      target_compile_definitions(strata_mmq PRIVATE STRATA_ORCA_Q4KS_MMQ=1)
+    endif()
     target_link_libraries(strata_mmq PUBLIC ggml-base hip::host roc::hipblas)
     target_link_libraries(strata_prefill PUBLIC strata_mmq)
     target_compile_definitions(strata_prefill PUBLIC STRATA_PREFILL_MMQ=1)
@@ -794,6 +950,32 @@ if(STRATA_NATIVE_EXPERTS)
     add_executable(native_expert_parity src/kernels/native_expert_parity.cpp)
     target_include_directories(native_expert_parity PRIVATE ${_strata_gpu_include_directories})
     target_link_libraries(native_expert_parity PRIVATE strata_kernels strata_kernels_cpu strata_artifact ${_strata_gpu_runtime_target})
+    if(STRATA_BUILD_TESTS AND STRATA_ENABLE_CUDA)
+      # Unsloth UD-Q4_K_XL's three expert pairs (and Q8_0 alone, its embedding) on synthetic ggml-quantized weights
+      # at 2560/640: GPU and CPU experts against the float reference, GPU dequantizers against to_float
+      foreach(_pair q4_K/q5_1 q4_K/q8_0 q5_K/q8_0 q8_0/q8_0)
+        string(REPLACE "/" "_" _name "${_pair}")
+        add_test(NAME native_expert_parity_${_name} COMMAND native_expert_parity --synthetic ${_pair})
+        set_tests_properties(native_expert_parity_${_name} PROPERTIES TIMEOUT 120)
+      endforeach()
+      add_test(NAME native_expert_parity_q5_1_min COMMAND native_expert_parity --q5_1-min)
+      # #290: the BF16 token embedding (--embd-gguf) gathered and dequantized bit for bit
+      add_test(NAME native_expert_parity_bf16_embd COMMAND native_expert_parity --bf16-embd)
+      # a pair without GPU kernels (Q6_K gate/up) is refused by native_expert_supported, not run
+      add_test(NAME native_expert_parity_refuses_q6_K COMMAND native_expert_parity --synthetic q6_K/q8_0)
+      set_tests_properties(native_expert_parity_refuses_q6_K PROPERTIES WILL_FAIL TRUE)
+    endif()
+  endif()
+  # #296's real-weight checks (OrcaRouter Q4_K_S files; docs/ORCA_Q4_K_S.md) - test programs, not part of a release
+  if(STRATA_ENABLE_CUDA AND STRATA_BUILD_TESTS)
+    add_executable(ple_q5_parity src/kernels/ple_q5_parity.cpp)
+    target_link_libraries(ple_q5_parity PRIVATE strata_kernels strata_artifact ggml-base ${_strata_gpu_runtime_target})
+    add_executable(q5_projection_parity src/kernels/q5_projection_parity.cpp)
+    target_include_directories(q5_projection_parity PRIVATE ${_strata_gpu_include_directories})
+    target_link_libraries(q5_projection_parity PRIVATE strata_kernels strata_artifact ggml-base ${_strata_gpu_runtime_target})
+    # #291: the FP8 PLE table (tools/ple_fp8_pack.py) through PleTable vs the checkpoint's own values
+    add_executable(ple_fp8_parity src/kernels/ple_fp8_parity.cpp)
+    target_link_libraries(ple_fp8_parity PRIVATE strata_kernels ${_strata_gpu_runtime_target})
   endif()
 endif()
 

Dockerfile

@@ -60,8 +60,7 @@ ARG BUILD_VISION=1
 
 RUN python3 -m venv .venv \
     && .venv/bin/pip install --no-cache-dir --upgrade pip \
-    && .venv/bin/pip install --no-cache-dir \
-        numpy jinja2 regex pyyaml tqdm requests cmake ninja pillow psutil \
+    && .venv/bin/pip install --no-cache-dir -r requirements.txt \
     && chmod +x setup.sh docker-entrypoint.sh
 
 # llama.cpp at the pinned commit, then the engine and the image encoder, built

README.md

@@ -1,255 +1,184 @@
 <h1 align="center">Strata</h1>
 
-<p align="center"><b>Run a 125-billion-parameter AI model on a normal gaming PC</b><br>
-one NVIDIA card (12-24 GB) + 64 GB of RAM · Windows or Linux · one click to install</p>
+<p align="center"><b>Run a 125-billion-parameter AI model on your own gaming PC</b><br>
+NVIDIA or AMD graphics card (12 GB or more) · Windows or Linux · free and open source</p>
 
 <p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br>
 <sub>A voxel pagoda garden, 1 shot prompt running on an RTX 5070 with Strata (IQ3_S, 128K context) ·
 <a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">full video (49 s)</a></sub></p>
 
 Strata runs **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** - a large, smart AI model that
-normally needs a server - on your own PC. It writes its answers at **60-95 tokens per second** (a token is about ¾
-of a word): faster than you can read.
-
-- **Free and open source.**
-
-> **Jump to:** [How fast?](#how-fast-is-it) · [Which model?](#which-model-should-i-pick) · [Install](#install) ·
-> [Using it](#using-it) · [Problems?](#something-went-wrong) · [How it works](#how-does-it-work) ·
-> [All the details](docs/DETAILS.md)
-
----
+normally needs a server - on a normal PC. It chats, writes code, reads pictures and works with your apps and coding
+agents, and nothing leaves your PC.
 
 ## How fast is it?
 
-Measured on an RTX 5070 (12 GB), a Ryzen 5 7600 and 64 GB of RAM:
+Measured on two ordinary gaming PCs. "Writes answers" is how fast the reply appears in a short chat; "reads your
+prompt" is how fast it takes in what you send (a 32K-token document, code or chat history). A token is about ¾ of a
+word, so 60 tokens per second is faster than you can read.
 
-| Size | Writes answers (short chat) | Writes answers (128K context) | Reads your prompt |
-| --- | ---: | ---: | ---: |
-| **Q2_0** | 93 tokens/s | 74 tokens/s | 2,170 tokens/s |
-| **IQ2_XS** | 79 tokens/s | 63 tokens/s | 2,090 tokens/s |
-| **IQ3_XXS** | 62 tokens/s | 49 tokens/s | 1,750 tokens/s |
-| **IQ3_S** | 53 tokens/s | 46 tokens/s | 1,620 tokens/s |
-| **Coder** (IQ1_M) | 55 tokens/s | 43 tokens/s | 2,180 tokens/s |
+<table>
+<tr><th>NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM</th><th>AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM</th></tr>
+<tr><td>
 
-- **Writes answers** = how fast the reply appears (tokens per second).
-- **Reads your prompt** = how fast it takes in what you send (long documents, code, chat history), measured on a
-  32K-token prompt; a 4K prompt reads at 910-1,580 tokens/s. A 32K prompt takes about 15 seconds with Q2_0.
+| Size | Writes answers | Reads your prompt |
+| --- | ---: | ---: |
+| **Q2_0** | 93 tokens/s | 2,170 tokens/s |
+| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |
+| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |
+| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |
+| **Coder** | 55 tokens/s | 2,180 tokens/s |
 
-A card with more VRAM is faster, because more of the model fits on the GPU: an RTX 3090 (24 GB) should do roughly
-100-140 tokens per second. All measurements, long-context numbers and estimates for other cards are in the
-[details](docs/DETAILS.md#speed-measured).
+</td><td>
 
-Every PC is different: `START-HERE.bat --calibrate` measures a few engine settings on yours and keeps the fastest
-(about 5-10 minutes; on the PC above it made the Coder 7% faster).
+| Size | Writes answers | Reads your prompt |
+| --- | ---: | ---: |
+| **Q2_0** | 60 tokens/s | 1,160 tokens/s |
+| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |
+| **Coder** | 44 tokens/s | 1,420 tokens/s |
 
-**Two or three NVIDIA cards?** Just run `START-HERE.bat`: it lists your cards, says which ones Strata can use, and
-asks whether to share the model across them (recommended when two can). An install made on one card asks once at
-its next start. Or choose yourself: `START-HERE.bat --gpus 0,2` (both, remembered), `--gpus all`, or `--gpu 0` (one
-card, this start only). Each card keeps the experts of its own layers, and prompts flow through the cards in a
-pipeline: on an RTX 5080 + RTX 3090 prompts were read 18-20% faster than on the 5080 alone, decoding on par.
-Every card must be an RTX 20 series or newer with 8 GB or more. See [docs/MULTI_GPU.md](docs/MULTI_GPU.md).
+</td></tr>
+</table>
 
-## Which model should I pick?
+A card with more VRAM is faster: an RTX 3090 (24 GB) should write roughly 100-140 tokens per second. Long chats,
+other cards: [speed of each model](docs/MODELS.md#how-fast-is-each-size), [community results](docs/COMMUNITY_BENCHMARKS.md).
 
-**The size** (the same model, compressed more or less):
+<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>
+<sub>Strata is free. If it runs well on your PC, a coffee keeps the work on it going.</sub></p>
 
-| Model | RAM+VRAM Requirements | Speed | Quality |
-| --- | ---: | --- | --- |
-| **Q2_0** | 37.6 GB | fastest | good |
-| **IQ2_XS** | 39.2 GB | fast | better (**recommended**) |
-| **IQ3_XXS** | 47.0 GB | slower | great |
-| **IQ3_S** | 54.8 GB | slowest | best: matches the full model on the published tests (original model only) |
+## What you need
 
-**Will it fit?** Shard 1 is the part of the model that gets loaded when it starts: its experts go into your **RAM**,
-the rest onto your graphics card (the second shard, a 29 GB lookup table, stays on the SSD). So it fits when your
-**RAM is at least shard 1 + about 10 GB** for Windows and your other programs. With 64 GB of RAM every size fits
-(IQ3_S with little else open); with 48 GB, Q2_0 and IQ2_XS. A bigger graphics card makes it faster, but it doesn't
-lower the RAM needed.
+| | |
+| --- | --- |
+| **Graphics card** | **NVIDIA** GeForce RTX 20, 30, 40 or 50 series, or **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 or RX 6800 / 6900 series - with **12 GB of VRAM or more** |
+| **RAM** | 32 GB or more - how much decides [which model](#which-model-should-i-pick) fits; 64 GB runs every size |
+| **Disk** | about 80 GB free, on an SSD if you can (the first start is much faster) |
+| **System** | Windows 10 / 11 or Linux, and a current graphics driver from NVIDIA or AMD |
 
-**The version:**
-
-- **Qwen3.8-Flash-Next** - the original.
-- **[Coder](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF)** - ISTA-DASLab's coding
-  version: half of the experts removed, keeping the ones that code, tool use and images need (91% of the full model's
-  SWE-bench Verified score, 99% of LiveCodeBench, by its authors). One size (IQ1_M: its experts stored like IQ3_S):
-  shard 1 is **29.6 GB**, so it fits a PC with **32 GB of RAM**, runs 262K context on 64 GB, and reads long prompts
-  the fastest of all. Weaker outside coding.
-- **[Swift 1.5](https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF)** - a fine-tune by UkisAI
-  that thinks much shorter before answering, so you get the answer sooner, with about the same quality. Same speed per
-  token, and about the same RAM as the same size of the original (no IQ3_S). Its own license applies (see its page).
-
-Not sure? Take **IQ2_XS** - or the **Coder** if you mainly write code, or have 32-48 GB of RAM. You can add another
-one later with `SETUP.bat` (the same as `START-HERE.bat --setup`; on Linux `./setup.sh --setup`).
-
-For **OrcaRouter's Flash-Next Uncensored IQ3_XXS**, see the [manual compatibility setup](docs/ORCA.md).
-It needs an explicit packing conversion and is not an installer menu option.
-
-An **AMD Radeon RX 7900 XT / XTX, RX 9070 / 9070 XT or Radeon AI PRO R9700 on Linux** works too (experimental):
-`./setup.sh --backend hip`, chosen by itself on a PC with no NVIDIA card Strata can use. It installs ROCm without sudo
-and compiles the engine (one GPU, no images yet). Details: [AMD HIP](docs/AMD_HIP.md).
+Everything else is installed for you. Two or three cards can share the model ([multi-GPU](docs/MULTI_GPU.md)).
+The full list: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).
 
 ## Install
 
-**You need:** an NVIDIA RTX 20, 30, 40 or 50 card with 12 GB of VRAM or more (RTX 20 since 0.1.27), enough RAM for the size you pick (above;
-a big GPU makes up for less RAM - the [low-RAM mode](docs/DETAILS.md)),
-~80 GB of free disk space (an SSD makes the first start much faster), and Windows 10/11 or Linux. The only thing you
-install yourself is a current **NVIDIA driver** ([nvidia.com/drivers](https://www.nvidia.com/drivers) or the NVIDIA
-App). Everything else - Python, the engine, the model - is set up for you.
+### Let your AI set it up
 
-**Windows**
+Use an AI coding assistant (Claude Code, Cursor, Codex, GitHub Copilot, ...)? Paste this into it:
 
-1. [Download this project](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) and unzip it (or `git clone` it).
-2. Double-click **`START-HERE.bat`**.
-3. Answer a few questions - or just press Enter each time for the recommended choice:
-   - **Which model and size?** The original or Swift 1.5, and Q2_0, IQ2_XS, IQ3_XXS or IQ3_S - see [above](#which-model-should-i-pick)
-   - **How much context?** How much text it can keep in mind at once (it suggests one for your card). 384K and
-     512K (experimental) extend the model past its trained 262K by rope scaling - the setup turns it on itself (yarn and a
-     covering factor; `--rope-scaling`/`--rope-scale` override) ([details](docs/DETAILS.md))
-   - **Images?** Whether it should also read pictures
-   - **Experimental speed projection?** Off unless you say yes - [read what it does](docs/DETAILS.md#experimental-speed-projection-experimental-off-by-default) first
+```text
+Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.
+```
 
-Then it downloads everything (the model is ~70 GB, so the first time takes a while - you can stop and it picks up
-where it left off) and **starts the model**. Your browser opens the Strata app at `http://127.0.0.1:8080`.
+It checks your graphics card, RAM and disk, picks the model that fits, installs it, starts it and tells you how to
+connect your apps. AI tools can also install, start and stop Strata themselves through its
+[MCP server](docs/MCP_SERVER.md).
+
+### Or do it yourself
+
+[Download Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) and unzip it (or `git clone` it).
+**Windows:** double-click **`START-HERE.bat`**. **Linux:** run **`./setup.sh`** in the Strata folder.
+
+The same steps for NVIDIA and AMD: the installer finds your card and sets up the right engine for it. It asks which
+model, which size, how much context (how much text it keeps in mind) and whether it should read pictures - press
+Enter each time for the recommended answer. Then it downloads the model (~70 GB; you can stop and it continues where
+it left off) and starts it. Your browser opens the Strata app at `http://127.0.0.1:8080`.
 
 > **While the model starts, your PC can be slow or stop responding for 1-3 minutes** (longest the first time): Strata
 > loads 35-55 GB into your RAM and locks part of it for the graphics card. That's normal - wait, and don't close the
 > window. The window tells you what it is doing.
 
-**Next time**, just double-click `START-HERE.bat` again: it starts right away, nothing is downloaded twice. Close its
-window to stop the model.
+**Next time**, run `START-HERE.bat` (or `./setup.sh`) again: it starts right away, nothing is downloaded twice. Close
+its window to stop the model. Updating, Docker, several cards, where the files go and every option:
+[docs/INSTALL.md](docs/INSTALL.md).
 
-**Updating:** download the new version and unzip it anywhere (or `git pull`), then run `START-HERE.bat` in it. The
-model files are kept in a `Strata-data` folder next to your Strata folder, so a new copy finds them and sets itself up
-the same way - nothing big is downloaded again.
+## Which model should I pick?
 
-**Linux:** run `./setup.sh` - same questions, same result.
+The installer recommends one for your RAM. The same model comes in sizes that are compressed more or less: smaller
+is faster, larger is a bit smarter.
 
-**Docker (Linux):** the same idea, in a container.
+| Your RAM | Take | Why |
+| --- | --- | --- |
+| **32 GB** | **Coder** | it fits 32 GB, and it is made for code (with a 24 GB card, Q2_0 and IQ2_XS run too) |
+| **48 GB** | **IQ2_XS** (or Q2_0, the fastest) | the larger sizes do not fit |
+| **64 GB** | **IQ2_XS** (recommended), or IQ3_XXS / IQ3_S | every size fits; IQ3_S is the best, and the slowest |
+| **96 GB or more** | **IQ3_S**, or Unsloth's 4-bit (experimental) | room for the largest sizes with everything else open |
 
-1. Host: Docker with the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
-   and a driver **580 or newer** (CUDA 13.0).
-2. Build (this compiles the engine into the image, so the container never compiles):
-   `docker build -t strata .`
-   `docker build -t strata --build-arg CUDA_ARCHITECTURES=89 .` builds for one card only (faster).
-   The default covers RTX 30 (86), RTX 40 (89), RTX 50 (120) and A-series (80); a card outside that
-   set needs a rebuild with its own arch. Add `--build-arg BUILD_VISION=0` to skip the image encoder.
-3. Run (the first start downloads the ~70 GB model, then starts; later starts go straight to serving):
-   `docker run --rm --gpus all -p 8080:8080 --ulimit memlock=-1 -v strata-data:/data strata`
+- **[Coder](docs/MODELS.md#coder)** - a coding version with half of the experts removed: 91% of the full model's
+  SWE-bench Verified score (by its authors), fits 32 GB of RAM. Weaker outside code, including Chinese and other
+  CJK text (#438): for those, take Q2_0, IQ2_XS or IQ3_S, which keep every expert.
+- **[Swift 1.5](docs/MODELS.md#swift-15)** - a fine-tune that thinks much shorter before it answers, so you get the
+  answer sooner, at about the same quality.
+- **[Unsloth UD-Q4_K_XL](docs/MODELS.md#unsloth-ud-q4_k_xl-experimental)** (experimental) - the closest to the full
+  model, but most of it is read from the SSD while it answers: 7-8.5 tokens/s on a 64 GB PC.
+- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** - a manual setup, not in the
+  installer's menu.
 
-   The setup choices are env vars: `-e MODEL=IQ2_XS -e FAMILY=qwen -e CONTEXT=32768 -e VISION=no`
-   (or `MODEL=Q2_0|IQ3_XXS|IQ3_S`, `FAMILY=swift|coder`; the defaults above are the recommended ones).
-   `-e VISION=cpu` keeps the image encoder on the CPU. `-e KV=int8|q4_0|k8v4` picks the KV cache
-   precision; `k8v4` is INT8 K with 4-bit V and keeps its KV in VRAM from 64K up.
-   Only the model files, the prepared pack, the MTP layer and the install config live in the
-   `strata-data` volume; the engine is part of the image. Switching between models already on the
-   volume needs no setup pass: `-e MODEL=Q2_0 -e FAMILY=coder` picks that model's config. Add
-   `-e REINSTALL=1` only to change settings for a model already set up (context, vision, KV, host,
-   api_key, LOW_RAM), since those are recorded in its config.
-   Strata loads 32-62 GB into RAM. `--gpus all` on a host with two usable cards takes both: the
-   layer split is setup's recommended default ([docs/MULTI_GPU.md](docs/MULTI_GPU.md)), and a volume
-   set up for one card switches to the pair on its first start there. Pin one card with `-e GPU=0`,
-   or name them with `-e GPUS=0,2` and where the later card's layers start with `-e LAYER_SPLIT=18`.
-   A memory limit needs `-e LOW_RAM=on`, which maps the model's experts from the pack instead of
-   keeping them in RAM: setup.py measures the host's RAM, not the container's limit, so it cannot
-   see a cap. LOW_RAM runs on one card.
-   The server listens on `0.0.0.0:8080` by default; set `-e API_KEY=<secret>` before exposing the port
-   to a network. The image has a `HEALTHCHECK` on `/health`, so `docker ps` shows the container
-   healthy once the model is loaded, and `GET /v1/status` says what it is running.
+Sizes, downloads and what fits where: [docs/MODELS.md](docs/MODELS.md). You can add another model later with
+`SETUP.bat` (Linux: `./setup.sh --setup`).
 
 ## Using it
 
 <p align="center"><img src="docs/media/runpagoda.png" width="900" alt="The Strata app's Monitor tab next to a coding agent"><br>
 <sub>The Strata app's <b>Monitor</b> (left) while a coding agent writes the pagoda garden from the video (right)</sub></p>
 
-- **In the browser:** `http://127.0.0.1:8080` - the Strata app (it opens by itself when the model starts): **Chat**, a
-  live **Monitor** of the model and your GPU/CPU/RAM, and **About** with the settings and addresses.
-- **Chat in the terminal:** `.venv\Scripts\python chat.py`
-- **Your apps and coding agents:** add it as an "OpenAI-compatible" provider with base URL
-  **`http://127.0.0.1:8080/v1`**, any API key and any model name. Apps that use Anthropic's API: `http://127.0.0.1:8080/v1/messages`.
-- **Thinking:** the model thinks before it answers. Choose **off, low, medium or high** - in the chat page menu, with
-  `/think low` in `chat.py`, or with your app's "reasoning effort" setting. Off is fastest; high is best for hard questions.
-- **Pictures:** in the chat page click **Picture**; in `chat.py` type `/image <path>`; in apps just attach them.
-- **From your phone or another PC:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>`, then open the
-  address the server window prints; see the [details](docs/DETAILS.md#using-it).
-- **Experimental speed projection (off by default):** an experimental control vector that setup can turn on; it
-  changes how the model answers - read [what it does](docs/DETAILS.md#experimental-speed-projection-experimental-off-by-default) first.
+- **In the browser:** `http://127.0.0.1:8080` - **Chat**, a live **Monitor** of the model and your GPU/CPU/RAM, and
+  **About** with the settings and addresses.
+- **Your apps and coding agents:** add an "OpenAI-compatible" provider with base URL **`http://127.0.0.1:8080/v1`**,
+  any API key and any model name. Apps that use Anthropic's API: `http://127.0.0.1:8080/v1/messages` (Claude Code:
+  `ANTHROPIC_BASE_URL=http://127.0.0.1:8080`).
+- **Thinking:** choose **off, low, medium or high** in the chat menu or your app's "reasoning effort". Off is
+  fastest; high is best for hard questions.
+- **Pictures:** say yes to "Images?" in setup, then click **Picture** in the chat, or attach them in your app
+  (AMD cards: on Linux through the processor, not on Windows yet).
+- **From your phone or another PC:** `START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>` - always with a key.
+- **Good to know:** it answers one request at a time. The first message of a chat is read in full (about 1 minute
+  per 30,000 tokens); follow-ups start in seconds.
 
-**Good to know:** it answers one request at a time. The first message of a chat is read in full (about 1 minute per
-30,000 tokens); after that it keeps the conversation and reads only what is new, so follow-ups start in seconds.
+More: [where your chats are stored](docs/INSTALL.md#where-things-are-stored), [the API](docs/DETAILS.md#using-it).
 
 ## Something went wrong?
 
-**My PC froze, or got very slow, the first time Strata started.**
-That's normal while it starts, most of all the first time. Strata loads 35-55 GB into your RAM, locks part of it for
-the graphics card, and works out how much of the model fits on your GPU. The mouse can freeze for a few minutes. **Wait, and don't close the
-window.** The next starts are much faster. Still frozen after 10 minutes? Restart the PC, close other programs
-(browsers use a lot of RAM) and try again. If it keeps happening, pick a smaller size (Q2_0 or IQ2_XS).
+- **My PC froze the first time Strata started.** Normal while it loads the model: wait, don't close the window.
+  Still frozen after 10 minutes? Restart the PC, close other programs and try again, or pick a smaller size.
+- **It stopped while downloading or installing.** Run `START-HERE.bat` (or `./setup.sh`) again: it continues where
+  it stopped.
+- **It's very slow and the disk light keeps blinking, or "the engine stopped unexpectedly".** Not enough free RAM:
+  close other programs (browsers use a lot), or pick a smaller size (Q2_0 or IQ2_XS).
+- **It says port 8080 is already in use.** Strata is already running - look for its window.
 
-**It stopped while downloading or installing.**
-Run `START-HERE.bat` again. It continues where it stopped.
-
-**It says the NVIDIA driver is too old.**
-Update it (NVIDIA App or [nvidia.com/drivers](https://www.nvidia.com/drivers)), restart the PC, and run
-`START-HERE.bat` again.
-
-**It says port 8080 is already in use.**
-Strata is already running. Look for its window.
-
-**It's very slow and the disk light keeps blinking.**
-Your PC is out of free RAM. Close other programs, or pick a smaller size (Q2_0 or IQ2_XS).
-
-**An answer stopped with "the engine stopped unexpectedly".**
-Usually not enough RAM (on Linux the system then stops the engine). Just send your message again: Strata starts the
-engine by itself. If it keeps happening, close other programs or pick a smaller size.
-
-**It says the prompt exceeds the context.**
-The conversation is longer than the context you chose. Start a new chat, or run `SETUP.bat` and pick more
-context.
-
-**Still stuck?** Look in the [full troubleshooting table](docs/DETAILS.md#troubleshooting), or open an issue and
-attach `strata-<model>.log` from the Strata folder.
+More problems and their fixes: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Still stuck? Open an
+[issue](https://github.com/Niko1221/Strata/issues) and attach `strata-<model>.log` from the Strata folder.
 
 ## How does it work?
 
 Models like this one normally run on servers with hundreds of gigabytes of graphics memory. Your graphics card has
-12-24 GB. Strata makes it fit by **sharing the work across your whole PC** - the same idea as a kitchen, where the
-things you use all the time stay on the counter and the rest waits in the pantry.
+12-24 GB. Strata makes it fit by **sharing the work across your whole PC** - like a kitchen, where the things you use
+all the time stay on the counter and the rest waits in the pantry.
 
 <p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
 
-- **The model is a team of 24,576 small specialists ("experts"),** and each word it writes needs only 10 of them.
-  So it doesn't have to have all of them on the graphics card at once.
-- **Your graphics card** does the part of the work needed for every word, and keeps the few thousand experts that
-  are asked most often. It keeps learning which ones those are while you use it.
-- **Your RAM** holds every expert. When a word needs one the card doesn't have, **your processor** works on it -
-  at the same time as the graphics card, so neither waits for the other.
-- **Your SSD** holds a big lookup table; the model only reads a few small rows of it per word.
+- **The model is a team of 24,576 small specialists ("experts"),** and each word needs only 10 of them.
+- **Your graphics card** keeps the few thousand experts that are asked most often; **your RAM** holds all of them,
+  and **your processor** works on the rest at the same time. **Your SSD** holds a big lookup table.
 
 <p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
 
-- **Guess, then check.** A small, fast helper built into the model guesses the next few words, and the big model
-  checks all the guesses in one go. It keeps the ones it agrees with and writes the next word itself - so one step
-  often produces several words. The helper only guesses - the big model decides every word - so you get the same
-  quality answer, 1.6-1.8x sooner.
-- **Long texts are read in big pieces** (up to 8,192 tokens - pieces of words - at a time), which is why a long
-  document or code base is read at over 1,000 tokens per second.
+- **Guess, then check:** a small helper guesses the next few words and the big model checks them all at once, so
+  you get the same answer, 1.6-1.8x sooner.
+- **Long texts are read in big pieces** (up to 8,192 tokens at a time): over 1,000 tokens per second.
 
-Want the full picture? The [details](docs/DETAILS.md#how-it-works) explain every part and its numbers, and the
-[paper](docs/paper/Strata-Paper.pdf) tells the whole story, with the measurements behind it.
+The longer explanation: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Every part and its numbers: [the
+details](docs/DETAILS.md#how-it-works) and the [paper](docs/paper/Strata-Paper.pdf).
 
-## Credits
+## Credits and license
 
-- Model: [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) by the Qwen team; compressed versions by
-  [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF);
-  [Swift 1.5](https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF) by UkisAI. Their licenses apply
-  to the model files.
-- Built with parts of [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp) (MIT). Ideas from
-  [Splash](https://github.com/incoai/splash), [ninfer](https://github.com/Neroued/ninfer) and
-  [HyperQwen](https://github.com/syv-ai/HyperQwen). More in the [details](docs/DETAILS.md#credits-and-licenses).
+The model is [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) by the Qwen team, compressed by
+[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5) and Unsloth;
+Strata is built with parts of [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). All credits:
+[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata is open source under the [MIT License](LICENSE); a few
+parts and every model carry their own licenses ([which ones](docs/HOW_IT_WORKS.md#license)).
 
-## License
+## Support Strata
 
-Strata is open source under the [MIT License](LICENSE). A few parts carry their own licenses: `third_party/ggml`
-(MIT, llama.cpp / ggml), the web app's font (SIL Open Font License 1.1) and the experimental speed projection's
-vector in `data/experimental-speed-projection` (Qwen Community License 1.0, from the model's activations). The
-models are not part of this repository; each model's own license applies to its files.
+Strata is free and open source. If it is useful to you, you can support its development:
+
+<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>

START-HERE.bat

@@ -1,6 +1,6 @@
 @echo off
 rem Strata for Windows: the first run installs everything and starts the model; later runs just start it.
-rem Needs only an NVIDIA driver. Python is installed for your user account if it is missing (no admin needed).
+rem Needs only an NVIDIA or AMD graphics driver. Python is installed for your user account if it is missing (no admin needed).
 setlocal
 title Strata
 cd /d "%~dp0"

bench/results/2026-09-29-rtx3090-epyc-milan/README.md

@@ -0,0 +1,46 @@
+# Preliminary native RTX 3090 / EPYC 7453 results
+
+Issue: [#165](https://github.com/Niko1221/Strata/issues/165)
+Engine: Strata v0.1.26, source build at `ac8b251`
+Status: preliminary; prompt set and harness guidance requested upstream.
+
+## Hardware
+
+| Component | Guest-visible details |
+|---|---|
+| CPU | AMD EPYC 7453, 20 vCPU, AVX2; AVX-512 not exposed |
+| RAM | 165 GiB visible to Ubuntu 24.04.5 KVM guest; physical DIMM layout and bandwidth not verified |
+| GPUs | 2x RTX 3090 24 GB; PCIe Gen4 x16 maximum reported per card; driver 595.91.07 |
+| Power limit | 280 W configured on each GPU; defaults are 350 W / 420 W, max-adjustable limits 366 W / 450 W |
+| Storage | 541 GiB ext4 guest data disk; host storage layout not verified |
+
+The power values above are configured caps, not measured workload draw. Idle draw during the hardware capture was approximately 35 W / 29 W.
+
+## Method
+
+Native Linux setup, local build from v0.1.26, `--prefill auto`, shipped expert profile, automatic expert cache, int8 KV, MTP on, images off, experimental speed projection off. The GPU power limit was 280 W per card. Native runs used a systemd service with `LimitMEMLOCK=infinity`; other llama-swap model containers were stopped.
+
+The local API harness used the public Strata source tree as its prompt corpus, with generated tasks, `temperature: 0`, `reasoning_effort: none`, and `max_tokens: 256`. It is not the upstream code-agent prompt set or harness. Runs use unique request IDs and no conversation reuse in speed mode. Three runs per speed cell are summarized by medians; actual `usage.prompt_tokens` are recorded because nominal context labels can drift. The first decode run is also shown. Several requests ended early on EOS, so actual generated counts are in the raw JSON.
+
+No calibration comparison, upstream prompt comparison, `STRATA_SPLIT_TIMING` stage timing, or under-load power-draw capture has been done. Treat all prompt/decode comparisons with upstream as indicative only until the requested upstream method is available.
+
+## Results
+
+See [`matrix.md`](matrix.md) for summary tables and [`matrix.json`](matrix.json) for all successful run records. Per-run source JSON, logs, the local harness, and scrubbed hardware facts are in [`data/`](data/).
+
+Highlights:
+
+- IQ3_XXS on one 3090: 1K–262K cells, three runs each; 262K prompts were 232,183 / 259,589 / 261,068 tokens and fit within the 262,144 context.
+- Coder IQ1_M at 32K: single-GPU median decode 66.8 tok/s; dual-GPU auto (K=19) median 98.6 tok/s.
+- Fixed Coder K sweep: K=22 was the highest observed three-run median (103.3 tok/s); independently generated prompts mean this is not a paired ranking.
+- IQ3_XXS and IQ3_S dual-GPU auto both selected K=24 at 32K.
+- IQ3_XXS and IQ3_S needles passed 3/3 at 8K / 16K / 28K on the tested single/dual configurations; two-turn checkpoint reuse passed for both.
+
+## Limits and follow-up
+
+- Upstream prompts/harness are not yet available; all current speed data uses a local source-tree corpus.
+- The guest exposes one NUMA node and 165 GiB RAM, but physical DIMM topology and memory bandwidth are unknown.
+- PCIe x16 is the reported maximum link width; the engine probes were about 23–26 GB/s. Verify PCIe generation/width under sustained load before claiming a direct x16-vs-x4 comparison.
+- Coder dual-GPU needle/two-turn checks and dual-GPU timing breakdowns are not run yet.
+- K=18 has one successful run at 83.1 tok/s; a repeated startup failed while uploading a dense weight with an out-of-memory error, so it is not a three-run median.
+- Calibration/default comparisons and IQ3_S single-GPU results remain outstanding.

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-1x3090/speed-32k-1.json

@@ -0,0 +1,360 @@
+{
+ "label": "native-v0.1.26-Coder-1x3090",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 1,
+ "run_id": "6c1252339f83",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:11:07",
+ "prompt_tokens": 28317,
+ "completion_tokens": 48,
+ "ttft_s": 12.868,
+ "e2e_s": 13.739,
+ "api_prompt_tps": 2200.5,
+ "api_decode_tps": 54.0,
+ "engine_prompt_tps": 2229.2,
+ "engine_decode_tps": 52.7,
+ "log_lines": [
+  "strata serve: prompt 28317 tokens = 0 reused + 28317 read in 12703 ms (2229.2 tok/s), 48 generated in 911 ms (52.7 tok/s), drafts accepted 31 of 38, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 96.3% (25001 hits / 25956 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what's there and identify the last file.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 8973,
+   "expert_cache_mib": 17503,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 369,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790716268.10927,
+    "duration_s": 13.6,
+    "finish": "stop",
+    "prompt_tokens": 28317,
+    "reused": 0,
+    "output_tokens": 48,
+    "engine_generated": 48,
+    "prompt_ms": 12702.6,
+    "decode_ms": 910.6,
+    "decode_tok_s": 52.7,
+    "hit_rate": 0.963
+   }
+  ],
+  "requests_kept": 1,
+  "totals": {
+   "since": 1790716256.7391467,
+   "requests": 1,
+   "prompt_tokens": 28317,
+   "reused": 0,
+   "output_tokens": 48,
+   "prompt_ms": 12702.6,
+   "decode_ms": 910.6
+  },
+  "hardware": {
+   "gpu_util": 88,
+   "gpu_mem_used": 25531711488,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 46,
+   "gpu_power": 220.362,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.28515625,
+   "gpu_pcie_tx_mb": 0.4287109375,
+   "cpu": 81.3,
+   "ram_used": 43964985344,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.9148478336133063,
+   "disk_write_mb": 0.0,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    1,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    99,
+    100,
+    100,
+    96,
+    100,
+    99,
+    100,
+    99,
+    99,
+    100,
+    100,
+    88
+   ],
+   "gpu_mem_used": [
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25407979520,
+    25418465280,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25531711488
+   ],
+   "gpu_temp": [
+    43,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    46,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    46
+   ],
+   "gpu_power": [
+    148.8,
+    140.48,
+    132.59,
+    119.03,
+    118.99,
+    119.03,
+    119.04,
+    119.03,
+    119.12,
+    119.17,
+    119.16,
+    119.15,
+    119.26,
+    191.57,
+    276.49,
+    276.88,
+    275.95,
+    277.04,
+    277.35,
+    267.91,
+    275.37,
+    276.16,
+    278.28,
+    279.07,
+    220.36
+   ],
+   "gpu_pcie_rx_mb": [
+    0.33,
+    0.33,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.29,
+    0.33,
+    0.29,
+    0.33,
+    0.29,
+    385.14,
+    19.88,
+    1892.23,
+    13280.39,
+    85.45,
+    16838.55,
+    680.06,
+    1060.72,
+    13.45,
+    15.74,
+    5013.75,
+    1194.52,
+    0.29
+   ],
+   "cpu": [
+    0.0,
+    0.7,
+    7.3,
+    15.7,
+    7.1,
+    2.0,
+    1.2,
+    2.0,
+    2.1,
+    1.6,
+    1.9,
+    7.4,
+    13.9,
+    18.3,
+    6.1,
+    6.0,
+    16.7,
+    24.9,
+    13.2,
+    12.0,
+    7.1,
+    6.2,
+    6.0,
+    8.8,
+    81.3
+   ],
+   "ram_used": [
+    43381886976,
+    43381694464,
+    43410796544,
+    43434872832,
+    43483377664,
+    43485900800,
+    43484897280,
+    43481210880,
+    43469135872,
+    43462295552,
+    43458363392,
+    43496140800,
+    43624353792,
+    43648593920,
+    43633405952,
+    43641139200,
+    43635249152,
+    43689267200,
+    43846070272,
+    43924541440,
+    43924467712,
+    43918565376,
+    43883311104,
+    44002877440,
+    43964985344
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    25.43,
+    204.93,
+    214.14,
+    7.41,
+    0.22,
+    184.19,
+    135.03,
+    0.0,
+    101.27,
+    0.0,
+    0.0,
+    0.0,
+    0.78,
+    2.91
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790716282.234284
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-1x3090/speed-32k-2.json

@@ -0,0 +1,491 @@
+{
+ "label": "native-v0.1.26-Coder-1x3090",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 2,
+ "run_id": "894af0ad69b7",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:11:22",
+ "prompt_tokens": 31630,
+ "completion_tokens": 26,
+ "ttft_s": 12.792,
+ "e2e_s": 13.164,
+ "api_prompt_tps": 2472.7,
+ "api_decode_tps": 67.3,
+ "engine_prompt_tps": 2506.6,
+ "engine_decode_tps": 66.8,
+ "log_lines": [
+  "strata serve: prompt 31630 tokens = 0 reused + 31630 read in 12619 ms (2506.6 tok/s), 26 generated in 389 ms (66.8 tok/s), drafts accepted 12 of 21, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 99.1% (16646 hits / 16791 lookups)"
+ ],
+ "answer_head": "I don't see any file content in the conversation above. Could you please share the file you'd like me to analyze?",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 8973,
+   "expert_cache_mib": 17503,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 369,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790716282.3818104,
+    "duration_s": 13.0,
+    "finish": "stop",
+    "prompt_tokens": 31630,
+    "reused": 0,
+    "output_tokens": 26,
+    "engine_generated": 26,
+    "prompt_ms": 12618.8,
+    "decode_ms": 389.0,
+    "decode_tok_s": 66.8,
+    "hit_rate": 0.991
+   },
+   {
+    "projection": null,
+    "time": 1790716268.10927,
+    "duration_s": 13.6,
+    "finish": "stop",
+    "prompt_tokens": 28317,
+    "reused": 0,
+    "output_tokens": 48,
+    "engine_generated": 48,
+    "prompt_ms": 12702.6,
+    "decode_ms": 910.6,
+    "decode_tok_s": 52.7,
+    "hit_rate": 0.963
+   }
+  ],
+  "requests_kept": 2,
+  "totals": {
+   "since": 1790716256.7391467,
+   "requests": 2,
+   "prompt_tokens": 59947,
+   "reused": 0,
+   "output_tokens": 74,
+   "prompt_ms": 25321.4,
+   "decode_ms": 1299.6
+  },
+  "hardware": {
+   "gpu_util": 81,
+   "gpu_mem_used": 25531711488,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 45,
+   "gpu_power": 256.109,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 9.7744140625,
+   "gpu_pcie_tx_mb": 25.1767578125,
+   "cpu": 46.1,
+   "ram_used": 43872071680,
+   "ram_total": 177175990272,
+   "disk_read_mb": 1.6222807345356267,
+   "disk_write_mb": 0.018689870213544086,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    1,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    99,
+    100,
+    100,
+    96,
+    100,
+    99,
+    100,
+    99,
+    99,
+    100,
+    100,
+    88,
+    0,
+    100,
+    99,
+    100,
+    99,
+    100,
+    98,
+    99,
+    99,
+    100,
+    99,
+    100,
+    81
+   ],
+   "gpu_mem_used": [
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25407979520,
+    25418465280,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488
+   ],
+   "gpu_temp": [
+    43,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    46,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    46,
+    44,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    48,
+    47,
+    45
+   ],
+   "gpu_power": [
+    148.8,
+    140.48,
+    132.59,
+    119.03,
+    118.99,
+    119.03,
+    119.04,
+    119.03,
+    119.12,
+    119.17,
+    119.16,
+    119.15,
+    119.26,
+    191.57,
+    276.49,
+    276.88,
+    275.95,
+    277.04,
+    277.35,
+    267.91,
+    275.37,
+    276.16,
+    278.28,
+    279.07,
+    220.36,
+    189.89,
+    258.72,
+    276.75,
+    276.67,
+    277.52,
+    275.34,
+    268.93,
+    277.69,
+    277.77,
+    276.91,
+    278.53,
+    278.56,
+    256.11
+   ],
+   "gpu_pcie_rx_mb": [
+    0.33,
+    0.33,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.29,
+    0.33,
+    0.29,
+    0.33,
+    0.29,
+    385.14,
+    19.88,
+    1892.23,
+    13280.39,
+    85.45,
+    16838.55,
+    680.06,
+    1060.72,
+    13.45,
+    15.74,
+    5013.75,
+    1194.52,
+    0.29,
+    2531.19,
+    14.02,
+    3715.09,
+    10.68,
+    13440.37,
+    8568.48,
+    14.92,
+    15.31,
+    15.69,
+    6486.56,
+    17.31,
+    9674.45,
+    9.77
+   ],
+   "cpu": [
+    0.0,
+    0.7,
+    7.3,
+    15.7,
+    7.1,
+    2.0,
+    1.2,
+    2.0,
+    2.1,
+    1.6,
+    1.9,
+    7.4,
+    13.9,
+    18.3,
+    6.1,
+    6.0,
+    16.7,
+    24.9,
+    13.2,
+    12.0,
+    7.1,
+    6.2,
+    6.0,
+    8.8,
+    81.3,
+    4.2,
+    6.8,
+    6.4,
+    7.6,
+    5.5,
+    6.0,
+    20.6,
+    12.9,
+    6.2,
+    7.0,
+    5.7,
+    5.9,
+    46.1
+   ],
+   "ram_used": [
+    43381886976,
+    43381694464,
+    43410796544,
+    43434872832,
+    43483377664,
+    43485900800,
+    43484897280,
+    43481210880,
+    43469135872,
+    43462295552,
+    43458363392,
+    43496140800,
+    43624353792,
+    43648593920,
+    43633405952,
+    43641139200,
+    43635249152,
+    43689267200,
+    43846070272,
+    43924541440,
+    43924467712,
+    43918565376,
+    43883311104,
+    44002877440,
+    43964985344,
+    43959267328,
+    43943862272,
+    43937861632,
+    43930320896,
+    43922284544,
+    43915001856,
+    43817238528,
+    43819147264,
+    43818729472,
+    43828027392,
+    43824201728,
+    43825434624,
+    43872071680
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    25.43,
+    204.93,
+    214.14,
+    7.41,
+    0.22,
+    184.19,
+    135.03,
+    0.0,
+    101.27,
+    0.0,
+    0.0,
+    0.0,
+    0.78,
+    2.91,
+    0.61,
+    0.0,
+    0.0,
+    0.02,
+    0.0,
+    0.0,
+    79.64,
+    41.92,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    1.62
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790716295.9009006
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-1x3090/speed-32k-3.json

@@ -0,0 +1,622 @@
+{
+ "label": "native-v0.1.26-Coder-1x3090",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 3,
+ "run_id": "a4c63bd599ab",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:11:35",
+ "prompt_tokens": 31601,
+ "completion_tokens": 45,
+ "ttft_s": 12.796,
+ "e2e_s": 13.243,
+ "api_prompt_tps": 2469.6,
+ "api_decode_tps": 98.5,
+ "engine_prompt_tps": 2502.4,
+ "engine_decode_tps": 97.4,
+ "log_lines": [
+  "strata serve: prompt 31601 tokens = 0 reused + 31601 read in 12628 ms (2502.4 tok/s), 45 generated in 462 ms (97.4 tok/s), drafts accepted 33 of 37, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 98.6% (23985 hits / 24333 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what we're working with.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 8973,
+   "expert_cache_mib": 17503,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 369,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790716296.0458817,
+    "duration_s": 13.1,
+    "finish": "stop",
+    "prompt_tokens": 31601,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 12628.3,
+    "decode_ms": 461.9,
+    "decode_tok_s": 97.4,
+    "hit_rate": 0.986
+   },
+   {
+    "projection": null,
+    "time": 1790716282.3818104,
+    "duration_s": 13.0,
+    "finish": "stop",
+    "prompt_tokens": 31630,
+    "reused": 0,
+    "output_tokens": 26,
+    "engine_generated": 26,
+    "prompt_ms": 12618.8,
+    "decode_ms": 389.0,
+    "decode_tok_s": 66.8,
+    "hit_rate": 0.991
+   },
+   {
+    "projection": null,
+    "time": 1790716268.10927,
+    "duration_s": 13.6,
+    "finish": "stop",
+    "prompt_tokens": 28317,
+    "reused": 0,
+    "output_tokens": 48,
+    "engine_generated": 48,
+    "prompt_ms": 12702.6,
+    "decode_ms": 910.6,
+    "decode_tok_s": 52.7,
+    "hit_rate": 0.963
+   }
+  ],
+  "requests_kept": 3,
+  "totals": {
+   "since": 1790716256.7391467,
+   "requests": 3,
+   "prompt_tokens": 91548,
+   "reused": 0,
+   "output_tokens": 119,
+   "prompt_ms": 37949.7,
+   "decode_ms": 1761.5
+  },
+  "hardware": {
+   "gpu_util": 96,
+   "gpu_mem_used": 25531711488,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 47,
+   "gpu_power": 245.399,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 414.037109375,
+   "gpu_pcie_tx_mb": 303.36328125,
+   "cpu": 35.1,
+   "ram_used": 43772702720,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.6087260569540994,
+   "disk_write_mb": 0.20539836277592313,
+   "tok_s": 97.29124144311484,
+   "tok_s_mean": 96.62332449971758
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    1,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    99,
+    100,
+    100,
+    96,
+    100,
+    99,
+    100,
+    99,
+    99,
+    100,
+    100,
+    88,
+    0,
+    100,
+    99,
+    100,
+    99,
+    100,
+    98,
+    99,
+    99,
+    100,
+    99,
+    100,
+    81,
+    99,
+    100,
+    100,
+    99,
+    100,
+    99,
+    17,
+    99,
+    100,
+    99,
+    99,
+    99,
+    96
+   ],
+   "gpu_mem_used": [
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25382813696,
+    25407979520,
+    25418465280,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25424756736,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488,
+    25531711488
+   ],
+   "gpu_temp": [
+    43,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    42,
+    46,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    46,
+    44,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    48,
+    47,
+    45,
+    47,
+    47,
+    47,
+    48,
+    48,
+    48,
+    46,
+    47,
+    48,
+    48,
+    48,
+    48,
+    47
+   ],
+   "gpu_power": [
+    148.8,
+    140.48,
+    132.59,
+    119.03,
+    118.99,
+    119.03,
+    119.04,
+    119.03,
+    119.12,
+    119.17,
+    119.16,
+    119.15,
+    119.26,
+    191.57,
+    276.49,
+    276.88,
+    275.95,
+    277.04,
+    277.35,
+    267.91,
+    275.37,
+    276.16,
+    278.28,
+    279.07,
+    220.36,
+    189.89,
+    258.72,
+    276.75,
+    276.67,
+    277.52,
+    275.34,
+    268.93,
+    277.69,
+    277.77,
+    276.91,
+    278.53,
+    278.56,
+    256.11,
+    199.6,
+    275.35,
+    275.49,
+    276.31,
+    275.46,
+    277.78,
+    266.35,
+    267.73,
+    276.71,
+    276.41,
+    275.34,
+    278.17,
+    245.4
+   ],
+   "gpu_pcie_rx_mb": [
+    0.33,
+    0.33,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.29,
+    0.33,
+    0.29,
+    0.33,
+    0.29,
+    385.14,
+    19.88,
+    1892.23,
+    13280.39,
+    85.45,
+    16838.55,
+    680.06,
+    1060.72,
+    13.45,
+    15.74,
+    5013.75,
+    1194.52,
+    0.29,
+    2531.19,
+    14.02,
+    3715.09,
+    10.68,
+    13440.37,
+    8568.48,
+    14.92,
+    15.31,
+    15.69,
+    6486.56,
+    17.31,
+    9674.45,
+    9.77,
+    2795.22,
+    4473.69,
+    14.5,
+    12.97,
+    14.92,
+    7716.66,
+    2819.78,
+    4278.33,
+    12009.48,
+    17.07,
+    9916.78,
+    8518.17,
+    414.04
+   ],
+   "cpu": [
+    0.0,
+    0.7,
+    7.3,
+    15.7,
+    7.1,
+    2.0,
+    1.2,
+    2.0,
+    2.1,
+    1.6,
+    1.9,
+    7.4,
+    13.9,
+    18.3,
+    6.1,
+    6.0,
+    16.7,
+    24.9,
+    13.2,
+    12.0,
+    7.1,
+    6.2,
+    6.0,
+    8.8,
+    81.3,
+    4.2,
+    6.8,
+    6.4,
+    7.6,
+    5.5,
+    6.0,
+    20.6,
+    12.9,
+    6.2,
+    7.0,
+    5.7,
+    5.9,
+    46.1,
+    4.0,
+    6.9,
+    5.7,
+    6.2,
+    7.1,
+    5.5,
+    5.6,
+    9.9,
+    13.7,
+    8.4,
+    7.7,
+    6.4,
+    35.1
+   ],
+   "ram_used": [
+    43381886976,
+    43381694464,
+    43410796544,
+    43434872832,
+    43483377664,
+    43485900800,
+    43484897280,
+    43481210880,
+    43469135872,
+    43462295552,
+    43458363392,
+    43496140800,
+    43624353792,
+    43648593920,
+    43633405952,
+    43641139200,
+    43635249152,
+    43689267200,
+    43846070272,
+    43924541440,
+    43924467712,
+    43918565376,
+    43883311104,
+    44002877440,
+    43964985344,
+    43959267328,
+    43943862272,
+    43937861632,
+    43930320896,
+    43922284544,
+    43915001856,
+    43817238528,
+    43819147264,
+    43818729472,
+    43828027392,
+    43824201728,
+    43825434624,
+    43872071680,
+    43734614016,
+    43728027648,
+    43719413760,
+    43703148544,
+    43695366144,
+    43687342080,
+    43744829440,
+    43791024128,
+    43789787136,
+    43766198272,
+    43761831936,
+    43750072320,
+    43772702720
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    25.43,
+    204.93,
+    214.14,
+    7.41,
+    0.22,
+    184.19,
+    135.03,
+    0.0,
+    101.27,
+    0.0,
+    0.0,
+    0.0,
+    0.78,
+    2.91,
+    0.61,
+    0.0,
+    0.0,
+    0.02,
+    0.0,
+    0.0,
+    79.64,
+    41.92,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    1.62,
+    0.72,
+    0.0,
+    0.0,
+    0.05,
+    0.0,
+    0.0,
+    0.18,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.61
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    97.29
+   ]
+  },
+  "time": 1790716309.6464093
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K18/speed-32k-1.json

@@ -0,0 +1,423 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K18",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 1,
+ "run_id": "052cbcbb7af7",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:21:48",
+ "prompt_tokens": 28313,
+ "completion_tokens": 57,
+ "ttft_s": 12.346,
+ "e2e_s": 12.991,
+ "api_prompt_tps": 2293.3,
+ "api_decode_tps": 86.9,
+ "engine_prompt_tps": 2324.6,
+ "engine_decode_tps": 83.1,
+ "log_lines": [
+  "strata serve: prompt 28313 tokens = 0 reused + 28313 read in 12180 ms (2324.6 tok/s), 57 generated in 686 ms (83.1 tok/s), drafts accepted 39 of 41, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 99.7% (28717 hits / 28800 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what's there and identify the last file.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 4608,
+   "expert_cache_mib": 8437,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 9557,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790716909.0337873,
+    "duration_s": 12.9,
+    "finish": "stop",
+    "prompt_tokens": 28313,
+    "reused": 0,
+    "output_tokens": 57,
+    "engine_generated": 57,
+    "prompt_ms": 12179.9,
+    "decode_ms": 685.7,
+    "decode_tok_s": 83.1,
+    "hit_rate": 0.997
+   }
+  ],
+  "requests_kept": 1,
+  "totals": {
+   "since": 1790716889.8200681,
+   "requests": 1,
+   "prompt_tokens": 28313,
+   "reused": 0,
+   "output_tokens": 57,
+   "prompt_ms": 12179.9,
+   "decode_ms": 685.7
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 0,
+     "mem_used": 15823994880,
+     "mem_total": 25769803776,
+     "temp": 44,
+     "power": 161.267
+    },
+    {
+     "index": 1,
+     "util": 90,
+     "mem_used": 24208408576,
+     "mem_total": 25769803776,
+     "temp": 51,
+     "power": 278.413
+    }
+   ],
+   "gpu_util": 45.0,
+   "gpu_mem_used": 40032403456,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 51,
+   "gpu_power": 439.68,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 5.33984375,
+   "gpu_pcie_tx_mb": 18.5009765625,
+   "cpu": 16.1,
+   "ram_used": 44574941184,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.9737281407762519,
+   "disk_write_mb": 0.11497896865254635,
+   "tok_s": 36.0,
+   "tok_s_mean": 64.01736582189179
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.5,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    41.5,
+    68.0,
+    50.0,
+    73.0,
+    87.5,
+    41.5,
+    47.0,
+    50.0,
+    78.0,
+    50.5,
+    45.0
+   ],
+   "gpu_mem_used": [
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39866728448,
+    39965294592,
+    39965294592,
+    39965294592,
+    39965294592,
+    39965294592,
+    39965294592,
+    39965294592,
+    39965294592,
+    39965294592,
+    39965294592,
+    40032403456
+   ],
+   "gpu_temp": [
+    45,
+    44,
+    44,
+    44,
+    44,
+    44,
+    45,
+    44,
+    44,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    42,
+    41,
+    41,
+    47,
+    50,
+    51,
+    51,
+    51,
+    52,
+    51,
+    52,
+    51,
+    52,
+    51
+   ],
+   "gpu_power": [
+    255.0,
+    240.14,
+    239.41,
+    239.38,
+    239.55,
+    239.75,
+    239.74,
+    239.65,
+    239.56,
+    239.63,
+    239.71,
+    239.58,
+    239.49,
+    239.62,
+    239.83,
+    239.9,
+    200.58,
+    138.26,
+    69.98,
+    230.69,
+    516.11,
+    485.77,
+    500.33,
+    518.82,
+    491.98,
+    467.22,
+    490.75,
+    506.65,
+    510.68,
+    439.68
+   ],
+   "gpu_pcie_rx_mb": [
+    0.71,
+    0.62,
+    0.67,
+    0.67,
+    0.67,
+    0.57,
+    0.71,
+    0.62,
+    0.62,
+    0.57,
+    0.67,
+    0.62,
+    0.71,
+    0.67,
+    0.67,
+    0.62,
+    0.76,
+    0.86,
+    0.71,
+    1243.59,
+    45.3,
+    2454.38,
+    77.06,
+    3382.78,
+    1099.87,
+    47.06,
+    2677.87,
+    79.25,
+    1143.26,
+    5.34
+   ],
+   "cpu": [
+    0.0,
+    1.0,
+    2.8,
+    3.4,
+    1.0,
+    1.6,
+    1.6,
+    1.0,
+    1.4,
+    0.7,
+    3.6,
+    8.3,
+    1.0,
+    8.2,
+    5.7,
+    2.6,
+    3.4,
+    1.4,
+    10.1,
+    12.9,
+    8.2,
+    10.3,
+    10.7,
+    11.6,
+    27.0,
+    14.9,
+    11.2,
+    18.3,
+    15.0,
+    16.1
+   ],
+   "ram_used": [
+    43962929152,
+    43965493248,
+    43981484032,
+    43968327680,
+    43968602112,
+    43968954368,
+    43964538880,
+    43964022784,
+    43964801024,
+    43964657664,
+    43961843712,
+    43958075392,
+    43958878208,
+    43967004672,
+    43967807488,
+    43951247360,
+    43945017344,
+    43945320448,
+    43971616768,
+    44103409664,
+    44112924672,
+    44112818176,
+    43979321344,
+    43978510336,
+    44145692672,
+    44276580352,
+    44275351552,
+    44327723008,
+    44296556544,
+    44574941184
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    64.21,
+    105.78,
+    24.45,
+    89.11,
+    67.32,
+    92.34,
+    145.4,
+    79.43,
+    98.92,
+    75.23,
+    0.0,
+    0.97
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    36.0
+   ]
+  },
+  "time": 1790716922.4098496
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K22/speed-32k-1.json

@@ -0,0 +1,270 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K22",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 1,
+ "run_id": "69d919ce1297",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:32:53",
+ "prompt_tokens": 28316,
+ "completion_tokens": 57,
+ "ttft_s": 10.575,
+ "e2e_s": 11.256,
+ "api_prompt_tps": 2677.7,
+ "api_decode_tps": 82.2,
+ "engine_prompt_tps": 2718.9,
+ "engine_decode_tps": 79.8,
+ "log_lines": [
+  "strata serve: prompt 28316 tokens = 0 reused + 28316 read in 10414 ms (2718.9 tok/s), 57 generated in 715 ms (79.8 tok/s), drafts accepted 37 of 48, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (34080 hits / 34080 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what's there and identify the last file.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 5632,
+   "expert_cache_mib": 10325,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 7669,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717573.1737459,
+    "duration_s": 11.1,
+    "finish": "stop",
+    "prompt_tokens": 28316,
+    "reused": 0,
+    "output_tokens": 57,
+    "engine_generated": 57,
+    "prompt_ms": 10414.4,
+    "decode_ms": 714.5,
+    "decode_tok_s": 79.8,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 1,
+  "totals": {
+   "since": 1790717571.262619,
+   "requests": 1,
+   "prompt_tokens": 28316,
+   "reused": 0,
+   "output_tokens": 57,
+   "prompt_ms": 10414.4,
+   "decode_ms": 714.5
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 34,
+     "mem_used": 17822580736,
+     "mem_total": 25769803776,
+     "temp": 49,
+     "power": 184.581
+    },
+    {
+     "index": 1,
+     "util": 54,
+     "mem_used": 22828482560,
+     "mem_total": 25769803776,
+     "temp": 52,
+     "power": 232.993
+    }
+   ],
+   "gpu_util": 44.0,
+   "gpu_mem_used": 40651063296,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 52,
+   "gpu_power": 417.57399999999996,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.666015625,
+   "gpu_pcie_tx_mb": 0.953125,
+   "cpu": 6.2,
+   "ram_used": 44281556992,
+   "ram_total": 177175990272,
+   "disk_read_mb": 3.421090719626586,
+   "disk_write_mb": 0.0,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    1.0,
+    0.0,
+    0.0,
+    73.5,
+    58.5,
+    96.0,
+    57.5,
+    95.5,
+    47.5,
+    93.0,
+    58.5,
+    50.0,
+    44.0
+   ],
+   "gpu_mem_used": [
+    40449736704,
+    40449736704,
+    40449736704,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40651063296
+   ],
+   "gpu_temp": [
+    48,
+    47,
+    47,
+    50,
+    53,
+    53,
+    54,
+    53,
+    54,
+    54,
+    54,
+    54,
+    52
+   ],
+   "gpu_power": [
+    256.98,
+    244.54,
+    244.03,
+    340.49,
+    532.78,
+    528.1,
+    525.31,
+    536.01,
+    504.78,
+    526.85,
+    529.86,
+    484.8,
+    417.57
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.62,
+    0.67,
+    78.72,
+    82.54,
+    76.1,
+    85.97,
+    55.17,
+    62.23,
+    87.12,
+    83.54,
+    85.83,
+    0.67
+   ],
+   "cpu": [
+    0.0,
+    1.2,
+    5.4,
+    10.7,
+    10.0,
+    9.8,
+    14.0,
+    18.3,
+    17.8,
+    18.6,
+    9.0,
+    5.8,
+    6.2
+   ],
+   "ram_used": [
+    43564908544,
+    43556737024,
+    43564126208,
+    43739512832,
+    43734302720,
+    43734560768,
+    43735560192,
+    43898122240,
+    44013469696,
+    43991674880,
+    43978436608,
+    43974189056,
+    44281556992
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    73.94,
+    96.25,
+    39.8,
+    73.77,
+    144.08,
+    110.85,
+    127.27,
+    131.68,
+    42.81,
+    0.0,
+    3.42
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790717584.813652
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K22/speed-32k-2.json

@@ -0,0 +1,383 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K22",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 2,
+ "run_id": "8506420b91e1",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:33:04",
+ "prompt_tokens": 31633,
+ "completion_tokens": 45,
+ "ttft_s": 10.85,
+ "e2e_s": 11.271,
+ "api_prompt_tps": 2915.4,
+ "api_decode_tps": 104.6,
+ "engine_prompt_tps": 2962.7,
+ "engine_decode_tps": 103.3,
+ "log_lines": [
+  "strata serve: prompt 31633 tokens = 0 reused + 31633 read in 10677 ms (2962.7 tok/s), 45 generated in 436 ms (103.3 tok/s), drafts accepted 32 of 37, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (24960 hits / 24960 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what the last file does.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 5632,
+   "expert_cache_mib": 10325,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 7669,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717584.9637764,
+    "duration_s": 11.1,
+    "finish": "stop",
+    "prompt_tokens": 31633,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 10677.2,
+    "decode_ms": 435.6,
+    "decode_tok_s": 103.3,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717573.1737459,
+    "duration_s": 11.1,
+    "finish": "stop",
+    "prompt_tokens": 28316,
+    "reused": 0,
+    "output_tokens": 57,
+    "engine_generated": 57,
+    "prompt_ms": 10414.4,
+    "decode_ms": 714.5,
+    "decode_tok_s": 79.8,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 2,
+  "totals": {
+   "since": 1790717571.262619,
+   "requests": 2,
+   "prompt_tokens": 59949,
+   "reused": 0,
+   "output_tokens": 102,
+   "prompt_ms": 21091.6,
+   "decode_ms": 1150.1
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 37,
+     "mem_used": 17822580736,
+     "mem_total": 25769803776,
+     "temp": 49,
+     "power": 187.064
+    },
+    {
+     "index": 1,
+     "util": 60,
+     "mem_used": 22828482560,
+     "mem_total": 25769803776,
+     "temp": 53,
+     "power": 243.674
+    }
+   ],
+   "gpu_util": 48.5,
+   "gpu_mem_used": 40651063296,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 53,
+   "gpu_power": 430.738,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.6181640625,
+   "gpu_pcie_tx_mb": 1.0,
+   "cpu": 6.1,
+   "ram_used": 44308209664,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.7959400039611647,
+   "disk_write_mb": 0.24130307812397303,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    1.0,
+    0.0,
+    0.0,
+    73.5,
+    58.5,
+    96.0,
+    57.5,
+    95.5,
+    47.5,
+    93.0,
+    58.5,
+    50.0,
+    44.0,
+    18.0,
+    52.0,
+    95.5,
+    50.0,
+    97.0,
+    24.5,
+    95.5,
+    82.0,
+    96.5,
+    50.0,
+    48.5
+   ],
+   "gpu_mem_used": [
+    40449736704,
+    40449736704,
+    40449736704,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296
+   ],
+   "gpu_temp": [
+    48,
+    47,
+    47,
+    50,
+    53,
+    53,
+    54,
+    53,
+    54,
+    54,
+    54,
+    54,
+    52,
+    49,
+    54,
+    54,
+    54,
+    54,
+    52,
+    54,
+    54,
+    54,
+    54,
+    53
+   ],
+   "gpu_power": [
+    256.98,
+    244.54,
+    244.03,
+    340.49,
+    532.78,
+    528.1,
+    525.31,
+    536.01,
+    504.78,
+    526.85,
+    529.86,
+    484.8,
+    417.57,
+    353.2,
+    539.11,
+    526.73,
+    525.83,
+    531.55,
+    507.76,
+    523.24,
+    527.24,
+    526.07,
+    501.32,
+    430.74
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.62,
+    0.67,
+    78.72,
+    82.54,
+    76.1,
+    85.97,
+    55.17,
+    62.23,
+    87.12,
+    83.54,
+    85.83,
+    0.67,
+    1765.25,
+    83.88,
+    85.4,
+    85.07,
+    88.59,
+    82.35,
+    78.3,
+    85.16,
+    61.94,
+    46.3,
+    0.62
+   ],
+   "cpu": [
+    0.0,
+    1.2,
+    5.4,
+    10.7,
+    10.0,
+    9.8,
+    14.0,
+    18.3,
+    17.8,
+    18.6,
+    9.0,
+    5.8,
+    6.2,
+    4.4,
+    6.6,
+    6.4,
+    5.8,
+    6.1,
+    7.4,
+    5.9,
+    6.8,
+    19.8,
+    14.2,
+    6.1
+   ],
+   "ram_used": [
+    43564908544,
+    43556737024,
+    43564126208,
+    43739512832,
+    43734302720,
+    43734560768,
+    43735560192,
+    43898122240,
+    44013469696,
+    43991674880,
+    43978436608,
+    43974189056,
+    44281556992,
+    44278607872,
+    44263776256,
+    44251099136,
+    44242841600,
+    44235354112,
+    44232339456,
+    44218642432,
+    44210057216,
+    44216344576,
+    44288196608,
+    44308209664
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    73.94,
+    96.25,
+    39.8,
+    73.77,
+    144.08,
+    110.85,
+    127.27,
+    131.68,
+    42.81,
+    0.0,
+    3.42,
+    0.48,
+    0.01,
+    0.04,
+    0.06,
+    0.02,
+    0.02,
+    0.02,
+    12.53,
+    104.78,
+    0.0,
+    0.8
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790717596.5877995
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K22/speed-32k-3.json

@@ -0,0 +1,496 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K22",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 3,
+ "run_id": "831d87a1dacc",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:33:16",
+ "prompt_tokens": 31601,
+ "completion_tokens": 60,
+ "ttft_s": 10.877,
+ "e2e_s": 11.43,
+ "api_prompt_tps": 2905.4,
+ "api_decode_tps": 106.7,
+ "engine_prompt_tps": 2951.6,
+ "engine_decode_tps": 105.7,
+ "log_lines": [
+  "strata serve: prompt 31601 tokens = 0 reused + 31601 read in 10706 ms (2951.6 tok/s), 60 generated in 568 ms (105.7 tok/s), drafts accepted 41 of 46, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (31680 hits / 31680 lookups)"
+ ],
+ "answer_head": "I'll start by examining the repository structure to understand what the last file does.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 5632,
+   "expert_cache_mib": 10325,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 7669,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717596.735188,
+    "duration_s": 11.3,
+    "finish": "stop",
+    "prompt_tokens": 31601,
+    "reused": 0,
+    "output_tokens": 60,
+    "engine_generated": 60,
+    "prompt_ms": 10706.3,
+    "decode_ms": 567.6,
+    "decode_tok_s": 105.7,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717584.9637764,
+    "duration_s": 11.1,
+    "finish": "stop",
+    "prompt_tokens": 31633,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 10677.2,
+    "decode_ms": 435.6,
+    "decode_tok_s": 103.3,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717573.1737459,
+    "duration_s": 11.1,
+    "finish": "stop",
+    "prompt_tokens": 28316,
+    "reused": 0,
+    "output_tokens": 57,
+    "engine_generated": 57,
+    "prompt_ms": 10414.4,
+    "decode_ms": 714.5,
+    "decode_tok_s": 79.8,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 3,
+  "totals": {
+   "since": 1790717571.262619,
+   "requests": 3,
+   "prompt_tokens": 91550,
+   "reused": 0,
+   "output_tokens": 162,
+   "prompt_ms": 31797.899999999998,
+   "decode_ms": 1717.6999999999998
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 25,
+     "mem_used": 17822580736,
+     "mem_total": 25769803776,
+     "temp": 50,
+     "power": 201.214
+    },
+    {
+     "index": 1,
+     "util": 61,
+     "mem_used": 22828482560,
+     "mem_total": 25769803776,
+     "temp": 53,
+     "power": 238.993
+    }
+   ],
+   "gpu_util": 43.0,
+   "gpu_mem_used": 40651063296,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 53,
+   "gpu_power": 440.207,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.666015625,
+   "gpu_pcie_tx_mb": 0.9052734375,
+   "cpu": 4.4,
+   "ram_used": 44291919872,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.7281688058408962,
+   "disk_write_mb": 0.01802398034259644,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    1.0,
+    0.0,
+    0.0,
+    73.5,
+    58.5,
+    96.0,
+    57.5,
+    95.5,
+    47.5,
+    93.0,
+    58.5,
+    50.0,
+    44.0,
+    18.0,
+    52.0,
+    95.5,
+    50.0,
+    97.0,
+    24.5,
+    95.5,
+    82.0,
+    96.5,
+    50.0,
+    48.5,
+    49.5,
+    96.0,
+    48.0,
+    96.0,
+    47.5,
+    94.5,
+    61.0,
+    97.5,
+    47.5,
+    49.5,
+    43.0
+   ],
+   "gpu_mem_used": [
+    40449736704,
+    40449736704,
+    40449736704,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296,
+    40651063296
+   ],
+   "gpu_temp": [
+    48,
+    47,
+    47,
+    50,
+    53,
+    53,
+    54,
+    53,
+    54,
+    54,
+    54,
+    54,
+    52,
+    49,
+    54,
+    54,
+    54,
+    54,
+    52,
+    54,
+    54,
+    54,
+    54,
+    53,
+    50,
+    54,
+    54,
+    54,
+    55,
+    54,
+    54,
+    54,
+    55,
+    55,
+    53
+   ],
+   "gpu_power": [
+    256.98,
+    244.54,
+    244.03,
+    340.49,
+    532.78,
+    528.1,
+    525.31,
+    536.01,
+    504.78,
+    526.85,
+    529.86,
+    484.8,
+    417.57,
+    353.2,
+    539.11,
+    526.73,
+    525.83,
+    531.55,
+    507.76,
+    523.24,
+    527.24,
+    526.07,
+    501.32,
+    430.74,
+    359.38,
+    510.67,
+    527.45,
+    526.85,
+    532.56,
+    501.47,
+    534.58,
+    526.16,
+    522.92,
+    475.95,
+    440.21
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.62,
+    0.67,
+    78.72,
+    82.54,
+    76.1,
+    85.97,
+    55.17,
+    62.23,
+    87.12,
+    83.54,
+    85.83,
+    0.67,
+    1765.25,
+    83.88,
+    85.4,
+    85.07,
+    88.59,
+    82.35,
+    78.3,
+    85.16,
+    61.94,
+    46.3,
+    0.62,
+    87.45,
+    41.53,
+    89.07,
+    84.21,
+    76.25,
+    52.17,
+    78.29,
+    83.59,
+    51.45,
+    47.92,
+    0.67
+   ],
+   "cpu": [
+    0.0,
+    1.2,
+    5.4,
+    10.7,
+    10.0,
+    9.8,
+    14.0,
+    18.3,
+    17.8,
+    18.6,
+    9.0,
+    5.8,
+    6.2,
+    4.4,
+    6.6,
+    6.4,
+    5.8,
+    6.1,
+    7.4,
+    5.9,
+    6.8,
+    19.8,
+    14.2,
+    6.1,
+    7.3,
+    6.6,
+    6.3,
+    5.7,
+    6.0,
+    6.2,
+    6.3,
+    6.1,
+    7.3,
+    6.4,
+    4.4
+   ],
+   "ram_used": [
+    43564908544,
+    43556737024,
+    43564126208,
+    43739512832,
+    43734302720,
+    43734560768,
+    43735560192,
+    43898122240,
+    44013469696,
+    43991674880,
+    43978436608,
+    43974189056,
+    44281556992,
+    44278607872,
+    44263776256,
+    44251099136,
+    44242841600,
+    44235354112,
+    44232339456,
+    44218642432,
+    44210057216,
+    44216344576,
+    44288196608,
+    44308209664,
+    44011671552,
+    43989819392,
+    43978940416,
+    43970682880,
+    43962933248,
+    44045533184,
+    44039798784,
+    44039798784,
+    44047568896,
+    44044705792,
+    44291919872
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    73.94,
+    96.25,
+    39.8,
+    73.77,
+    144.08,
+    110.85,
+    127.27,
+    131.68,
+    42.81,
+    0.0,
+    3.42,
+    0.48,
+    0.01,
+    0.04,
+    0.06,
+    0.02,
+    0.02,
+    0.02,
+    12.53,
+    104.78,
+    0.0,
+    0.8,
+    0.61,
+    0.05,
+    0.03,
+    0.09,
+    0.08,
+    0.04,
+    0.05,
+    0.04,
+    0.15,
+    0.0,
+    0.73
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790717608.5200934
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K24/speed-32k-1.json

@@ -0,0 +1,279 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K24",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 1,
+ "run_id": "87af67edfd2d",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:34:02",
+ "prompt_tokens": 28314,
+ "completion_tokens": 256,
+ "ttft_s": 10.029,
+ "e2e_s": 12.977,
+ "api_prompt_tps": 2823.1,
+ "api_decode_tps": 86.5,
+ "engine_prompt_tps": 2871.1,
+ "engine_decode_tps": 85.9,
+ "log_lines": [
+  "strata serve: prompt 28314 tokens = 0 reused + 28314 read in 9862 ms (2871.1 tok/s), 256 generated in 2981 ms (85.9 tok/s), drafts accepted 143 of 193, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (147360 hits / 147360 lookups)"
+ ],
+ "answer_head": "Looking at the last file above, `docs/AMD_HIP_PERFORMANCE.md`, it documents the performance evidence and correctness limits for the experimental AMD HIP backend targeting gfx1100 (RX 7900 XTX). It records:\n\n- The exact build/runtime configuration used for the benchmark (env vars, CLI flags, model pa",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 6144,
+   "expert_cache_mib": 11337,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 6657,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717642.434618,
+    "duration_s": 12.9,
+    "finish": "length",
+    "prompt_tokens": 28314,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 9861.8,
+    "decode_ms": 2981.5,
+    "decode_tok_s": 85.9,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 1,
+  "totals": {
+   "since": 1790717640.7776525,
+   "requests": 1,
+   "prompt_tokens": 28314,
+   "reused": 0,
+   "output_tokens": 256,
+   "prompt_ms": 9861.8,
+   "decode_ms": 2981.5
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 43,
+     "mem_used": 18898419712,
+     "mem_total": 25769803776,
+     "temp": 50,
+     "power": 244.874
+    },
+    {
+     "index": 1,
+     "util": 57,
+     "mem_used": 21758935040,
+     "mem_total": 25769803776,
+     "temp": 54,
+     "power": 254.71
+    }
+   ],
+   "gpu_util": 50.0,
+   "gpu_mem_used": 40657354752,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 54,
+   "gpu_power": 499.584,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 5.38671875,
+   "gpu_pcie_tx_mb": 87.7373046875,
+   "cpu": 11.7,
+   "ram_used": 44619116544,
+   "ram_total": 177175990272,
+   "disk_read_mb": 4.515214018548113,
+   "disk_write_mb": 0.17657261525048487,
+   "tok_s": 107.09643367230112,
+   "tok_s_mean": 83.26979350665239
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    1.0,
+    0.0,
+    1.0,
+    93.0,
+    94.0,
+    64.0,
+    94.5,
+    88.5,
+    95.5,
+    92.5,
+    48.5,
+    32.0,
+    48.0,
+    50.0
+   ],
+   "gpu_mem_used": [
+    40447639552,
+    40447639552,
+    40476999680,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40657354752,
+    40657354752,
+    40657354752
+   ],
+   "gpu_temp": [
+    49,
+    49,
+    49,
+    53,
+    54,
+    54,
+    55,
+    54,
+    55,
+    55,
+    55,
+    52,
+    54,
+    54
+   ],
+   "gpu_power": [
+    259.59,
+    247.18,
+    245.88,
+    414.53,
+    546.71,
+    547.76,
+    546.95,
+    533.57,
+    531.41,
+    545.46,
+    539.14,
+    402.04,
+    486.67,
+    499.58
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.62,
+    247.86,
+    81.11,
+    78.2,
+    80.92,
+    83.88,
+    57.6,
+    82.87,
+    79.92,
+    40.29,
+    5.39,
+    9.77,
+    5.39
+   ],
+   "cpu": [
+    0.0,
+    1.9,
+    9.1,
+    8.8,
+    9.5,
+    10.7,
+    13.9,
+    10.5,
+    13.9,
+    11.8,
+    6.0,
+    6.3,
+    12.4,
+    11.7
+   ],
+   "ram_used": [
+    43887411200,
+    43859918848,
+    43956232192,
+    44034678784,
+    44030210048,
+    44028076032,
+    44015280128,
+    44252610560,
+    44247789568,
+    44243210240,
+    44230340608,
+    44558966784,
+    44619108352,
+    44619116544
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    126.09,
+    44.05,
+    80.0,
+    93.94,
+    138.6,
+    104.01,
+    156.43,
+    97.38,
+    0.0,
+    1.76,
+    4.66,
+    4.52
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    78.74,
+    81.4,
+    107.1
+   ]
+  },
+  "time": 1790717655.7892108
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K24/speed-32k-2.json

@@ -0,0 +1,392 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K24",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 2,
+ "run_id": "b38394f620b3",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:34:15",
+ "prompt_tokens": 31635,
+ "completion_tokens": 45,
+ "ttft_s": 10.269,
+ "e2e_s": 10.686,
+ "api_prompt_tps": 3080.5,
+ "api_decode_tps": 105.5,
+ "engine_prompt_tps": 3130.4,
+ "engine_decode_tps": 104.2,
+ "log_lines": [
+  "strata serve: prompt 31635 tokens = 0 reused + 31635 read in 10106 ms (3130.4 tok/s), 45 generated in 432 ms (104.2 tok/s), drafts accepted 33 of 37, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (24480 hits / 24480 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what we're working with.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 6144,
+   "expert_cache_mib": 11337,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 6657,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717655.9301767,
+    "duration_s": 10.5,
+    "finish": "stop",
+    "prompt_tokens": 31635,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 10105.9,
+    "decode_ms": 431.8,
+    "decode_tok_s": 104.2,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717642.434618,
+    "duration_s": 12.9,
+    "finish": "length",
+    "prompt_tokens": 28314,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 9861.8,
+    "decode_ms": 2981.5,
+    "decode_tok_s": 85.9,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 2,
+  "totals": {
+   "since": 1790717640.7776525,
+   "requests": 2,
+   "prompt_tokens": 59949,
+   "reused": 0,
+   "output_tokens": 301,
+   "prompt_ms": 19967.699999999997,
+   "decode_ms": 3413.3
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 20,
+     "mem_used": 18898419712,
+     "mem_total": 25769803776,
+     "temp": 50,
+     "power": 191.174
+    },
+    {
+     "index": 1,
+     "util": 53,
+     "mem_used": 21758935040,
+     "mem_total": 25769803776,
+     "temp": 54,
+     "power": 238.032
+    }
+   ],
+   "gpu_util": 36.5,
+   "gpu_mem_used": 40657354752,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 54,
+   "gpu_power": 429.206,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.6181640625,
+   "gpu_pcie_tx_mb": 0.857421875,
+   "cpu": 5.3,
+   "ram_used": 44575621120,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.4688909410966033,
+   "disk_write_mb": 0.23427432287777988,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    1.0,
+    0.0,
+    1.0,
+    93.0,
+    94.0,
+    64.0,
+    94.5,
+    88.5,
+    95.5,
+    92.5,
+    48.5,
+    32.0,
+    48.0,
+    50.0,
+    0.0,
+    84.5,
+    96.0,
+    95.5,
+    79.0,
+    97.5,
+    97.0,
+    95.5,
+    96.0,
+    50.0,
+    36.5
+   ],
+   "gpu_mem_used": [
+    40447639552,
+    40447639552,
+    40476999680,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752
+   ],
+   "gpu_temp": [
+    49,
+    49,
+    49,
+    53,
+    54,
+    54,
+    55,
+    54,
+    55,
+    55,
+    55,
+    52,
+    54,
+    54,
+    50,
+    54,
+    55,
+    55,
+    55,
+    55,
+    55,
+    55,
+    55,
+    55,
+    54
+   ],
+   "gpu_power": [
+    259.59,
+    247.18,
+    245.88,
+    414.53,
+    546.71,
+    547.76,
+    546.95,
+    533.57,
+    531.41,
+    545.46,
+    539.14,
+    402.04,
+    486.67,
+    499.58,
+    409.59,
+    480.21,
+    551.08,
+    548.0,
+    547.82,
+    539.97,
+    537.72,
+    548.44,
+    548.33,
+    513.55,
+    429.21
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.62,
+    247.86,
+    81.11,
+    78.2,
+    80.92,
+    83.88,
+    57.6,
+    82.87,
+    79.92,
+    40.29,
+    5.39,
+    9.77,
+    5.39,
+    45.78,
+    64.8,
+    1284.55,
+    129.94,
+    83.06,
+    82.63,
+    84.78,
+    73.15,
+    82.44,
+    0.57,
+    0.62
+   ],
+   "cpu": [
+    0.0,
+    1.9,
+    9.1,
+    8.8,
+    9.5,
+    10.7,
+    13.9,
+    10.5,
+    13.9,
+    11.8,
+    6.0,
+    6.3,
+    12.4,
+    11.7,
+    6.7,
+    8.3,
+    6.5,
+    7.2,
+    6.1,
+    6.0,
+    6.1,
+    6.5,
+    11.2,
+    8.0,
+    5.3
+   ],
+   "ram_used": [
+    43887411200,
+    43859918848,
+    43956232192,
+    44034678784,
+    44030210048,
+    44028076032,
+    44015280128,
+    44252610560,
+    44247789568,
+    44243210240,
+    44230340608,
+    44558966784,
+    44619108352,
+    44619116544,
+    44154675200,
+    44152926208,
+    44151242752,
+    44149854208,
+    44145594368,
+    44375150592,
+    44366180352,
+    44365656064,
+    44362940416,
+    44400578560,
+    44575621120
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    126.09,
+    44.05,
+    80.0,
+    93.94,
+    138.6,
+    104.01,
+    156.43,
+    97.38,
+    0.0,
+    1.76,
+    4.66,
+    4.52,
+    0.63,
+    0.02,
+    0.04,
+    0.03,
+    0.05,
+    0.03,
+    0.02,
+    13.87,
+    103.6,
+    0.0,
+    2.47
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    78.74,
+    81.4,
+    107.1,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790717666.9788535
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K24/speed-32k-3.json

@@ -0,0 +1,496 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K24",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 3,
+ "run_id": "3bee1c1e482a",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:34:26",
+ "prompt_tokens": 31602,
+ "completion_tokens": 60,
+ "ttft_s": 10.281,
+ "e2e_s": 10.865,
+ "api_prompt_tps": 3073.8,
+ "api_decode_tps": 101.0,
+ "engine_prompt_tps": 3123.2,
+ "engine_decode_tps": 100.2,
+ "log_lines": [
+  "strata serve: prompt 31602 tokens = 0 reused + 31602 read in 10118 ms (3123.2 tok/s), 60 generated in 599 ms (100.2 tok/s), drafts accepted 40 of 52, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (34560 hits / 34560 lookups)"
+ ],
+ "answer_head": "I'll start by examining the repository structure to understand what the last file does.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 6144,
+   "expert_cache_mib": 11337,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 6657,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717667.1193056,
+    "duration_s": 10.7,
+    "finish": "stop",
+    "prompt_tokens": 31602,
+    "reused": 0,
+    "output_tokens": 60,
+    "engine_generated": 60,
+    "prompt_ms": 10118.5,
+    "decode_ms": 599.0,
+    "decode_tok_s": 100.2,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717655.9301767,
+    "duration_s": 10.5,
+    "finish": "stop",
+    "prompt_tokens": 31635,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 10105.9,
+    "decode_ms": 431.8,
+    "decode_tok_s": 104.2,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717642.434618,
+    "duration_s": 12.9,
+    "finish": "length",
+    "prompt_tokens": 28314,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 9861.8,
+    "decode_ms": 2981.5,
+    "decode_tok_s": 85.9,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 3,
+  "totals": {
+   "since": 1790717640.7776525,
+   "requests": 3,
+   "prompt_tokens": 91551,
+   "reused": 0,
+   "output_tokens": 361,
+   "prompt_ms": 30086.199999999997,
+   "decode_ms": 4012.3
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 43,
+     "mem_used": 18898419712,
+     "mem_total": 25769803776,
+     "temp": 51,
+     "power": 178.691
+    },
+    {
+     "index": 1,
+     "util": 51,
+     "mem_used": 21758935040,
+     "mem_total": 25769803776,
+     "temp": 54,
+     "power": 244.501
+    }
+   ],
+   "gpu_util": 47.0,
+   "gpu_mem_used": 40657354752,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 54,
+   "gpu_power": 423.192,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 5.1962890625,
+   "gpu_pcie_tx_mb": 103.138671875,
+   "cpu": 6.6,
+   "ram_used": 44596383744,
+   "ram_total": 177175990272,
+   "disk_read_mb": 1.1207609014346964,
+   "disk_write_mb": 0.0288298624163266,
+   "tok_s": 110.47676922161895,
+   "tok_s_mean": 99.0096070780873
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    1.0,
+    0.0,
+    1.0,
+    93.0,
+    94.0,
+    64.0,
+    94.5,
+    88.5,
+    95.5,
+    92.5,
+    48.5,
+    32.0,
+    48.0,
+    50.0,
+    0.0,
+    84.5,
+    96.0,
+    95.5,
+    79.0,
+    97.5,
+    97.0,
+    95.5,
+    96.0,
+    50.0,
+    36.5,
+    49.5,
+    96.5,
+    61.0,
+    94.5,
+    91.0,
+    97.0,
+    95.5,
+    98.5,
+    47.5,
+    47.0
+   ],
+   "gpu_mem_used": [
+    40447639552,
+    40447639552,
+    40476999680,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40546205696,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752,
+    40657354752
+   ],
+   "gpu_temp": [
+    49,
+    49,
+    49,
+    53,
+    54,
+    54,
+    55,
+    54,
+    55,
+    55,
+    55,
+    52,
+    54,
+    54,
+    50,
+    54,
+    55,
+    55,
+    55,
+    55,
+    55,
+    55,
+    55,
+    55,
+    54,
+    52,
+    55,
+    55,
+    55,
+    55,
+    56,
+    56,
+    55,
+    56,
+    54
+   ],
+   "gpu_power": [
+    259.59,
+    247.18,
+    245.88,
+    414.53,
+    546.71,
+    547.76,
+    546.95,
+    533.57,
+    531.41,
+    545.46,
+    539.14,
+    402.04,
+    486.67,
+    499.58,
+    409.59,
+    480.21,
+    551.08,
+    548.0,
+    547.82,
+    539.97,
+    537.72,
+    548.44,
+    548.33,
+    513.55,
+    429.21,
+    354.61,
+    553.75,
+    546.57,
+    547.59,
+    532.21,
+    535.5,
+    547.28,
+    548.49,
+    534.02,
+    423.19
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.62,
+    247.86,
+    81.11,
+    78.2,
+    80.92,
+    83.88,
+    57.6,
+    82.87,
+    79.92,
+    40.29,
+    5.39,
+    9.77,
+    5.39,
+    45.78,
+    64.8,
+    1284.55,
+    129.94,
+    83.06,
+    82.63,
+    84.78,
+    73.15,
+    82.44,
+    0.57,
+    0.62,
+    81.82,
+    83.45,
+    75.34,
+    84.16,
+    82.11,
+    81.49,
+    70.95,
+    73.91,
+    52.55,
+    5.2
+   ],
+   "cpu": [
+    0.0,
+    1.9,
+    9.1,
+    8.8,
+    9.5,
+    10.7,
+    13.9,
+    10.5,
+    13.9,
+    11.8,
+    6.0,
+    6.3,
+    12.4,
+    11.7,
+    6.7,
+    8.3,
+    6.5,
+    7.2,
+    6.1,
+    6.0,
+    6.1,
+    6.5,
+    11.2,
+    8.0,
+    5.3,
+    6.7,
+    12.7,
+    10.0,
+    5.9,
+    6.4,
+    6.0,
+    7.1,
+    5.7,
+    6.8,
+    6.6
+   ],
+   "ram_used": [
+    43887411200,
+    43859918848,
+    43956232192,
+    44034678784,
+    44030210048,
+    44028076032,
+    44015280128,
+    44252610560,
+    44247789568,
+    44243210240,
+    44230340608,
+    44558966784,
+    44619108352,
+    44619116544,
+    44154675200,
+    44152926208,
+    44151242752,
+    44149854208,
+    44145594368,
+    44375150592,
+    44366180352,
+    44365656064,
+    44362940416,
+    44400578560,
+    44575621120,
+    44112637952,
+    44134821888,
+    44114948096,
+    44112486400,
+    44240445440,
+    44343197696,
+    44341800960,
+    44344549376,
+    44344029184,
+    44596383744
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    126.09,
+    44.05,
+    80.0,
+    93.94,
+    138.6,
+    104.01,
+    156.43,
+    97.38,
+    0.0,
+    1.76,
+    4.66,
+    4.52,
+    0.63,
+    0.02,
+    0.04,
+    0.03,
+    0.05,
+    0.03,
+    0.02,
+    13.87,
+    103.6,
+    0.0,
+    2.47,
+    0.63,
+    0.05,
+    0.08,
+    0.08,
+    0.04,
+    0.07,
+    0.04,
+    0.17,
+    0.0,
+    1.12
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    78.74,
+    81.4,
+    107.1,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    110.48
+   ]
+  },
+  "time": 1790717678.3471994
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K26/speed-32k-1.json

@@ -0,0 +1,261 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K26",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 1,
+ "run_id": "5ca39dba7489",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:35:22",
+ "prompt_tokens": 28314,
+ "completion_tokens": 48,
+ "ttft_s": 9.79,
+ "e2e_s": 10.297,
+ "api_prompt_tps": 2892.2,
+ "api_decode_tps": 92.7,
+ "engine_prompt_tps": 2940.2,
+ "engine_decode_tps": 88.8,
+ "log_lines": [
+  "strata serve: prompt 28314 tokens = 0 reused + 28314 read in 9630 ms (2940.2 tok/s), 48 generated in 540 ms (88.8 tok/s), drafts accepted 33 of 37, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (25920 hits / 25920 lookups)"
+ ],
+ "answer_head": "I'll first look at the repository structure to understand what files exist and identify the last file.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 6656,
+   "expert_cache_mib": 12387,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 5607,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717722.129571,
+    "duration_s": 10.2,
+    "finish": "stop",
+    "prompt_tokens": 28314,
+    "reused": 0,
+    "output_tokens": 48,
+    "engine_generated": 48,
+    "prompt_ms": 9630.1,
+    "decode_ms": 540.5,
+    "decode_tok_s": 88.8,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 1,
+  "totals": {
+   "since": 1790717720.412441,
+   "requests": 1,
+   "prompt_tokens": 28314,
+   "reused": 0,
+   "output_tokens": 48,
+   "prompt_ms": 9630.1,
+   "decode_ms": 540.5
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 47,
+     "mem_used": 19980550144,
+     "mem_total": 25769803776,
+     "temp": 51,
+     "power": 179.665
+    },
+    {
+     "index": 1,
+     "util": 28,
+     "mem_used": 20641153024,
+     "mem_total": 25769803776,
+     "temp": 53,
+     "power": 236.004
+    }
+   ],
+   "gpu_util": 37.5,
+   "gpu_mem_used": 40621703168,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 53,
+   "gpu_power": 415.669,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.666015625,
+   "gpu_pcie_tx_mb": 0.9052734375,
+   "cpu": 6.0,
+   "ram_used": 44539072512,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.97776661958663,
+   "disk_write_mb": 0.0,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.5,
+    0.0,
+    0.5,
+    71.0,
+    95.5,
+    96.0,
+    91.5,
+    66.5,
+    95.5,
+    90.5,
+    84.5,
+    37.5
+   ],
+   "gpu_mem_used": [
+    40447639552,
+    40447639552,
+    40472805376,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40621703168
+   ],
+   "gpu_temp": [
+    50,
+    50,
+    50,
+    52,
+    55,
+    55,
+    56,
+    56,
+    56,
+    55,
+    56,
+    53
+   ],
+   "gpu_power": [
+    261.03,
+    248.02,
+    247.5,
+    351.95,
+    548.48,
+    553.77,
+    555.05,
+    536.77,
+    531.38,
+    552.37,
+    551.57,
+    415.67
+   ],
+   "gpu_pcie_rx_mb": [
+    0.76,
+    0.62,
+    224.64,
+    86.02,
+    83.16,
+    75.62,
+    86.69,
+    102.66,
+    65.66,
+    83.97,
+    38.86,
+    0.67
+   ],
+   "cpu": [
+    0.0,
+    1.1,
+    8.1,
+    9.1,
+    9.3,
+    12.2,
+    12.6,
+    12.6,
+    23.5,
+    14.7,
+    5.9,
+    6.0
+   ],
+   "ram_used": [
+    43871281152,
+    43867545600,
+    43958726656,
+    44040531968,
+    44041007104,
+    44033200128,
+    44032602112,
+    44244361216,
+    44250234880,
+    44236812288,
+    44233678848,
+    44539072512
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    122.8,
+    47.19,
+    79.99,
+    100.77,
+    127.09,
+    108.72,
+    156.54,
+    97.45,
+    0.0,
+    2.98
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790717732.8116624
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K26/speed-32k-2.json

@@ -0,0 +1,365 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K26",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 2,
+ "run_id": "14c7210ce0e9",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:35:32",
+ "prompt_tokens": 31635,
+ "completion_tokens": 45,
+ "ttft_s": 10.029,
+ "e2e_s": 10.496,
+ "api_prompt_tps": 3154.3,
+ "api_decode_tps": 94.3,
+ "engine_prompt_tps": 3205.2,
+ "engine_decode_tps": 93.5,
+ "log_lines": [
+  "strata serve: prompt 31635 tokens = 0 reused + 31635 read in 9870 ms (3205.2 tok/s), 45 generated in 481 ms (93.5 tok/s), drafts accepted 31 of 36, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (25440 hits / 25440 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what the last file does.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 6656,
+   "expert_cache_mib": 12387,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 5607,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717732.947479,
+    "duration_s": 10.4,
+    "finish": "stop",
+    "prompt_tokens": 31635,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 9870.0,
+    "decode_ms": 481.3,
+    "decode_tok_s": 93.5,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717722.129571,
+    "duration_s": 10.2,
+    "finish": "stop",
+    "prompt_tokens": 28314,
+    "reused": 0,
+    "output_tokens": 48,
+    "engine_generated": 48,
+    "prompt_ms": 9630.1,
+    "decode_ms": 540.5,
+    "decode_tok_s": 88.8,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 2,
+  "totals": {
+   "since": 1790717720.412441,
+   "requests": 2,
+   "prompt_tokens": 59949,
+   "reused": 0,
+   "output_tokens": 93,
+   "prompt_ms": 19500.1,
+   "decode_ms": 1021.8
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 51,
+     "mem_used": 19991035904,
+     "mem_total": 25769803776,
+     "temp": 51,
+     "power": 190.866
+    },
+    {
+     "index": 1,
+     "util": 56,
+     "mem_used": 20653735936,
+     "mem_total": 25769803776,
+     "temp": 55,
+     "power": 249.813
+    }
+   ],
+   "gpu_util": 53.5,
+   "gpu_mem_used": 40644771840,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 55,
+   "gpu_power": 440.679,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 16.44921875,
+   "gpu_pcie_tx_mb": 82.0625,
+   "cpu": 6.0,
+   "ram_used": 44553875456,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.6053434938228842,
+   "disk_write_mb": 0.0,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.5,
+    0.0,
+    0.5,
+    71.0,
+    95.5,
+    96.0,
+    91.5,
+    66.5,
+    95.5,
+    90.5,
+    84.5,
+    37.5,
+    1.5,
+    97.0,
+    96.0,
+    81.0,
+    86.5,
+    95.5,
+    95.0,
+    96.5,
+    95.0,
+    53.5
+   ],
+   "gpu_mem_used": [
+    40447639552,
+    40447639552,
+    40472805376,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40644771840
+   ],
+   "gpu_temp": [
+    50,
+    50,
+    50,
+    52,
+    55,
+    55,
+    56,
+    56,
+    56,
+    55,
+    56,
+    53,
+    51,
+    56,
+    56,
+    56,
+    56,
+    56,
+    56,
+    56,
+    56,
+    55
+   ],
+   "gpu_power": [
+    261.03,
+    248.02,
+    247.5,
+    351.95,
+    548.48,
+    553.77,
+    555.05,
+    536.77,
+    531.38,
+    552.37,
+    551.57,
+    415.67,
+    326.37,
+    532.86,
+    555.24,
+    552.15,
+    555.12,
+    507.8,
+    554.85,
+    554.0,
+    554.13,
+    440.68
+   ],
+   "gpu_pcie_rx_mb": [
+    0.76,
+    0.62,
+    224.64,
+    86.02,
+    83.16,
+    75.62,
+    86.69,
+    102.66,
+    65.66,
+    83.97,
+    38.86,
+    0.67,
+    3698.35,
+    132.46,
+    131.89,
+    82.06,
+    85.92,
+    83.02,
+    1279.88,
+    50.4,
+    86.21,
+    16.45
+   ],
+   "cpu": [
+    0.0,
+    1.1,
+    8.1,
+    9.1,
+    9.3,
+    12.2,
+    12.6,
+    12.6,
+    23.5,
+    14.7,
+    5.9,
+    6.0,
+    4.5,
+    6.1,
+    6.0,
+    6.4,
+    6.7,
+    5.9,
+    6.7,
+    9.9,
+    7.4,
+    6.0
+   ],
+   "ram_used": [
+    43871281152,
+    43867545600,
+    43958726656,
+    44040531968,
+    44041007104,
+    44033200128,
+    44032602112,
+    44244361216,
+    44250234880,
+    44236812288,
+    44233678848,
+    44539072512,
+    44064968704,
+    44058497024,
+    44058497024,
+    44058501120,
+    44175929344,
+    44294356992,
+    44294352896,
+    44295569408,
+    44291391488,
+    44553875456
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    122.8,
+    47.19,
+    79.99,
+    100.77,
+    127.09,
+    108.72,
+    156.54,
+    97.45,
+    0.0,
+    2.98,
+    0.47,
+    0.01,
+    0.04,
+    0.06,
+    0.03,
+    0.03,
+    12.57,
+    71.85,
+    33.12,
+    0.61
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790717743.8106396
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K26/speed-32k-3.json

@@ -0,0 +1,469 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K26",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 3,
+ "run_id": "3f243fdb63ee",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:35:43",
+ "prompt_tokens": 31601,
+ "completion_tokens": 60,
+ "ttft_s": 10.026,
+ "e2e_s": 10.61,
+ "api_prompt_tps": 3151.9,
+ "api_decode_tps": 101.0,
+ "engine_prompt_tps": 3208.3,
+ "engine_decode_tps": 100.2,
+ "log_lines": [
+  "strata serve: prompt 31601 tokens = 0 reused + 31601 read in 9850 ms (3208.3 tok/s), 60 generated in 599 ms (100.2 tok/s), drafts accepted 40 of 50, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (34080 hits / 34080 lookups)"
+ ],
+ "answer_head": "I'll start by examining the repository structure to understand what the last file is.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 6656,
+   "expert_cache_mib": 12387,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 5607,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717743.964174,
+    "duration_s": 10.5,
+    "finish": "stop",
+    "prompt_tokens": 31601,
+    "reused": 0,
+    "output_tokens": 60,
+    "engine_generated": 60,
+    "prompt_ms": 9849.9,
+    "decode_ms": 598.9,
+    "decode_tok_s": 100.2,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717732.947479,
+    "duration_s": 10.4,
+    "finish": "stop",
+    "prompt_tokens": 31635,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 9870.0,
+    "decode_ms": 481.3,
+    "decode_tok_s": 93.5,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717722.129571,
+    "duration_s": 10.2,
+    "finish": "stop",
+    "prompt_tokens": 28314,
+    "reused": 0,
+    "output_tokens": 48,
+    "engine_generated": 48,
+    "prompt_ms": 9630.1,
+    "decode_ms": 540.5,
+    "decode_tok_s": 88.8,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 3,
+  "totals": {
+   "since": 1790717720.412441,
+   "requests": 3,
+   "prompt_tokens": 91550,
+   "reused": 0,
+   "output_tokens": 153,
+   "prompt_ms": 29350.0,
+   "decode_ms": 1620.6999999999998
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 46,
+     "mem_used": 19991035904,
+     "mem_total": 25769803776,
+     "temp": 52,
+     "power": 189.922
+    },
+    {
+     "index": 1,
+     "util": 52,
+     "mem_used": 20653735936,
+     "mem_total": 25769803776,
+     "temp": 55,
+     "power": 246.398
+    }
+   ],
+   "gpu_util": 49.0,
+   "gpu_mem_used": 40644771840,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 55,
+   "gpu_power": 436.32,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 17.8330078125,
+   "gpu_pcie_tx_mb": 43.1533203125,
+   "cpu": 6.0,
+   "ram_used": 44614168576,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.4973320821900814,
+   "disk_write_mb": 0.05405783502066102,
+   "tok_s": 99.26679787882165,
+   "tok_s_mean": 97.997900113756
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.5,
+    0.0,
+    0.5,
+    71.0,
+    95.5,
+    96.0,
+    91.5,
+    66.5,
+    95.5,
+    90.5,
+    84.5,
+    37.5,
+    1.5,
+    97.0,
+    96.0,
+    81.0,
+    86.5,
+    95.5,
+    95.0,
+    96.5,
+    95.0,
+    53.5,
+    0.0,
+    95.0,
+    95.5,
+    82.0,
+    94.5,
+    93.0,
+    88.5,
+    96.0,
+    97.0,
+    49.0
+   ],
+   "gpu_mem_used": [
+    40447639552,
+    40447639552,
+    40472805376,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40535719936,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40621703168,
+    40644771840,
+    40644771840,
+    40644771840,
+    40644771840,
+    40644771840,
+    40644771840,
+    40644771840,
+    40644771840,
+    40644771840,
+    40644771840,
+    40644771840
+   ],
+   "gpu_temp": [
+    50,
+    50,
+    50,
+    52,
+    55,
+    55,
+    56,
+    56,
+    56,
+    55,
+    56,
+    53,
+    51,
+    56,
+    56,
+    56,
+    56,
+    56,
+    56,
+    56,
+    56,
+    55,
+    52,
+    55,
+    56,
+    56,
+    56,
+    56,
+    56,
+    56,
+    56,
+    55
+   ],
+   "gpu_power": [
+    261.03,
+    248.02,
+    247.5,
+    351.95,
+    548.48,
+    553.77,
+    555.05,
+    536.77,
+    531.38,
+    552.37,
+    551.57,
+    415.67,
+    326.37,
+    532.86,
+    555.24,
+    552.15,
+    555.12,
+    507.8,
+    554.85,
+    554.0,
+    554.13,
+    440.68,
+    363.75,
+    474.54,
+    552.72,
+    554.16,
+    554.3,
+    509.24,
+    555.11,
+    555.88,
+    555.36,
+    436.32
+   ],
+   "gpu_pcie_rx_mb": [
+    0.76,
+    0.62,
+    224.64,
+    86.02,
+    83.16,
+    75.62,
+    86.69,
+    102.66,
+    65.66,
+    83.97,
+    38.86,
+    0.67,
+    3698.35,
+    132.46,
+    131.89,
+    82.06,
+    85.92,
+    83.02,
+    1279.88,
+    50.4,
+    86.21,
+    16.45,
+    38.96,
+    81.63,
+    83.25,
+    84.4,
+    72.53,
+    127.22,
+    48.4,
+    96.51,
+    100.71,
+    17.83
+   ],
+   "cpu": [
+    0.0,
+    1.1,
+    8.1,
+    9.1,
+    9.3,
+    12.2,
+    12.6,
+    12.6,
+    23.5,
+    14.7,
+    5.9,
+    6.0,
+    4.5,
+    6.1,
+    6.0,
+    6.4,
+    6.7,
+    5.9,
+    6.7,
+    9.9,
+    7.4,
+    6.0,
+    11.7,
+    10.8,
+    6.2,
+    11.4,
+    7.0,
+    6.7,
+    5.6,
+    5.4,
+    6.0,
+    6.0
+   ],
+   "ram_used": [
+    43871281152,
+    43867545600,
+    43958726656,
+    44040531968,
+    44041007104,
+    44033200128,
+    44032602112,
+    44244361216,
+    44250234880,
+    44236812288,
+    44233678848,
+    44539072512,
+    44064968704,
+    44058497024,
+    44058497024,
+    44058501120,
+    44175929344,
+    44294356992,
+    44294352896,
+    44295569408,
+    44291391488,
+    44553875456,
+    44129808384,
+    44180946944,
+    44184666112,
+    44182659072,
+    44296114176,
+    44404658176,
+    44394999808,
+    44386783232,
+    44374441984,
+    44614168576
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    122.8,
+    47.19,
+    79.99,
+    100.77,
+    127.09,
+    108.72,
+    156.54,
+    97.45,
+    0.0,
+    2.98,
+    0.47,
+    0.01,
+    0.04,
+    0.06,
+    0.03,
+    0.03,
+    12.57,
+    71.85,
+    33.12,
+    0.61,
+    0.53,
+    0.06,
+    0.06,
+    0.09,
+    0.08,
+    0.04,
+    0.04,
+    0.04,
+    0.15,
+    0.5
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    99.27
+   ]
+  },
+  "time": 1790717754.9239163
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K28/speed-32k-1.json

@@ -0,0 +1,261 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K28",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 1,
+ "run_id": "0796a1ce12e4",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:36:29",
+ "prompt_tokens": 28316,
+ "completion_tokens": 57,
+ "ttft_s": 10.285,
+ "e2e_s": 10.86,
+ "api_prompt_tps": 2753.1,
+ "api_decode_tps": 97.4,
+ "engine_prompt_tps": 2798.5,
+ "engine_decode_tps": 93.7,
+ "log_lines": [
+  "strata serve: prompt 28316 tokens = 0 reused + 28316 read in 10118 ms (2798.5 tok/s), 57 generated in 608 ms (93.7 tok/s), drafts accepted 40 of 42, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (28800 hits / 28800 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what's there and identify the last file.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 7168,
+   "expert_cache_mib": 13437,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 4557,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717789.1846194,
+    "duration_s": 10.7,
+    "finish": "stop",
+    "prompt_tokens": 28316,
+    "reused": 0,
+    "output_tokens": 57,
+    "engine_generated": 57,
+    "prompt_ms": 10118.2,
+    "decode_ms": 608.0,
+    "decode_tok_s": 93.8,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 1,
+  "totals": {
+   "since": 1790717787.625935,
+   "requests": 1,
+   "prompt_tokens": 28316,
+   "reused": 0,
+   "output_tokens": 57,
+   "prompt_ms": 10118.2,
+   "decode_ms": 608.0
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 41,
+     "mem_used": 21104623616,
+     "mem_total": 25769803776,
+     "temp": 52,
+     "power": 189.832
+    },
+    {
+     "index": 1,
+     "util": 42,
+     "mem_used": 19548536832,
+     "mem_total": 25769803776,
+     "temp": 54,
+     "power": 230.563
+    }
+   ],
+   "gpu_util": 41.5,
+   "gpu_mem_used": 40653160448,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 54,
+   "gpu_power": 420.395,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 17.1650390625,
+   "gpu_pcie_tx_mb": 53.119140625,
+   "cpu": 6.1,
+   "ram_used": 44503613440,
+   "ram_total": 177175990272,
+   "disk_read_mb": 1.946188226381293,
+   "disk_write_mb": 0.0036040522710764685,
+   "tok_s": 81.42212134334822,
+   "tok_s_mean": 78.79383849818437
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.5,
+    0.0,
+    0.0,
+    46.5,
+    96.0,
+    72.5,
+    93.0,
+    90.0,
+    95.5,
+    93.5,
+    97.5,
+    41.5
+   ],
+   "gpu_mem_used": [
+    40447639552,
+    40447639552,
+    40495874048,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40653160448
+   ],
+   "gpu_temp": [
+    51,
+    50,
+    50,
+    52,
+    55,
+    54,
+    56,
+    54,
+    56,
+    56,
+    57,
+    54
+   ],
+   "gpu_power": [
+    261.46,
+    249.04,
+    248.59,
+    342.07,
+    531.07,
+    533.47,
+    545.68,
+    511.29,
+    533.34,
+    534.58,
+    541.51,
+    420.39
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.62,
+    182.48,
+    55.88,
+    87.02,
+    80.58,
+    57.5,
+    78.87,
+    83.88,
+    1294.42,
+    46.78,
+    17.17
+   ],
+   "cpu": [
+    0.0,
+    6.6,
+    18.0,
+    11.3,
+    10.5,
+    9.8,
+    11.5,
+    13.9,
+    11.4,
+    12.9,
+    7.4,
+    6.1
+   ],
+   "ram_used": [
+    43799613440,
+    43777974272,
+    43950305280,
+    43962880000,
+    43939622912,
+    43930345472,
+    43932086272,
+    44071780352,
+    44183257088,
+    44182880256,
+    44179062784,
+    44503613440
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    129.59,
+    40.83,
+    79.94,
+    84.92,
+    115.02,
+    136.42,
+    117.79,
+    106.04,
+    29.9,
+    1.95
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    81.42
+   ]
+  },
+  "time": 1790717800.421838
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K28/speed-32k-2.json

@@ -0,0 +1,374 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K28",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 2,
+ "run_id": "3d5bf2c2aa6f",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:36:40",
+ "prompt_tokens": 31631,
+ "completion_tokens": 45,
+ "ttft_s": 10.58,
+ "e2e_s": 10.972,
+ "api_prompt_tps": 2989.7,
+ "api_decode_tps": 112.3,
+ "engine_prompt_tps": 3034.9,
+ "engine_decode_tps": 110.6,
+ "log_lines": [
+  "strata serve: prompt 31631 tokens = 0 reused + 31631 read in 10423 ms (3034.9 tok/s), 45 generated in 407 ms (110.6 tok/s), drafts accepted 33 of 34, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (23040 hits / 23040 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what we're working with.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 7168,
+   "expert_cache_mib": 13437,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 4557,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717800.5571585,
+    "duration_s": 10.8,
+    "finish": "stop",
+    "prompt_tokens": 31631,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 10422.5,
+    "decode_ms": 406.8,
+    "decode_tok_s": 110.6,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717789.1846194,
+    "duration_s": 10.7,
+    "finish": "stop",
+    "prompt_tokens": 28316,
+    "reused": 0,
+    "output_tokens": 57,
+    "engine_generated": 57,
+    "prompt_ms": 10118.2,
+    "decode_ms": 608.0,
+    "decode_tok_s": 93.8,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 2,
+  "totals": {
+   "since": 1790717787.625935,
+   "requests": 2,
+   "prompt_tokens": 59947,
+   "reused": 0,
+   "output_tokens": 102,
+   "prompt_ms": 20540.7,
+   "decode_ms": 1014.8
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 46,
+     "mem_used": 21104623616,
+     "mem_total": 25769803776,
+     "temp": 51,
+     "power": 204.027
+    },
+    {
+     "index": 1,
+     "util": 12,
+     "mem_used": 19548536832,
+     "mem_total": 25769803776,
+     "temp": 54,
+     "power": 226.431
+    }
+   ],
+   "gpu_util": 29.0,
+   "gpu_mem_used": 40653160448,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 54,
+   "gpu_power": 430.45799999999997,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.6181640625,
+   "gpu_pcie_tx_mb": 0.9052734375,
+   "cpu": 4.3,
+   "ram_used": 44573519872,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.9768799607845553,
+   "disk_write_mb": 0.0,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.5,
+    0.0,
+    0.0,
+    46.5,
+    96.0,
+    72.5,
+    93.0,
+    90.0,
+    95.5,
+    93.5,
+    97.5,
+    41.5,
+    0.0,
+    96.0,
+    93.0,
+    97.5,
+    94.5,
+    97.0,
+    95.0,
+    97.0,
+    93.5,
+    67.0,
+    29.0
+   ],
+   "gpu_mem_used": [
+    40447639552,
+    40447639552,
+    40495874048,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448
+   ],
+   "gpu_temp": [
+    51,
+    50,
+    50,
+    52,
+    55,
+    54,
+    56,
+    54,
+    56,
+    56,
+    57,
+    54,
+    52,
+    55,
+    54,
+    56,
+    56,
+    56,
+    56,
+    57,
+    57,
+    56,
+    54
+   ],
+   "gpu_power": [
+    261.46,
+    249.04,
+    248.59,
+    342.07,
+    531.07,
+    533.47,
+    545.68,
+    511.29,
+    533.34,
+    534.58,
+    541.51,
+    420.39,
+    375.37,
+    452.84,
+    528.99,
+    547.53,
+    532.29,
+    524.3,
+    518.92,
+    543.34,
+    532.25,
+    534.03,
+    430.46
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.62,
+    182.48,
+    55.88,
+    87.02,
+    80.58,
+    57.5,
+    78.87,
+    83.88,
+    1294.42,
+    46.78,
+    17.17,
+    42.15,
+    1288.27,
+    41.05,
+    75.34,
+    70.76,
+    44.15,
+    79.87,
+    61.61,
+    75.63,
+    42.06,
+    0.62
+   ],
+   "cpu": [
+    0.0,
+    6.6,
+    18.0,
+    11.3,
+    10.5,
+    9.8,
+    11.5,
+    13.9,
+    11.4,
+    12.9,
+    7.4,
+    6.1,
+    4.2,
+    6.4,
+    6.0,
+    12.9,
+    11.2,
+    6.1,
+    8.2,
+    8.8,
+    14.0,
+    6.4,
+    4.3
+   ],
+   "ram_used": [
+    43799613440,
+    43777974272,
+    43950305280,
+    43962880000,
+    43939622912,
+    43930345472,
+    43932086272,
+    44071780352,
+    44183257088,
+    44182880256,
+    44179062784,
+    44503613440,
+    44015304704,
+    44019281920,
+    44021350400,
+    44034875392,
+    44111585280,
+    44348657664,
+    44344815616,
+    44359872512,
+    44334280704,
+    44327985152,
+    44573519872
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    129.59,
+    40.83,
+    79.94,
+    84.92,
+    115.02,
+    136.42,
+    117.79,
+    106.04,
+    29.9,
+    1.95,
+    1.72,
+    0.02,
+    0.02,
+    0.05,
+    0.06,
+    0.01,
+    0.03,
+    12.6,
+    101.64,
+    2.94,
+    0.98
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    81.42,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790717811.896799
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K28/speed-32k-3.json

@@ -0,0 +1,487 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K28",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 3,
+ "run_id": "d22ca9d8c7b8",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:36:51",
+ "prompt_tokens": 31602,
+ "completion_tokens": 59,
+ "ttft_s": 10.575,
+ "e2e_s": 11.17,
+ "api_prompt_tps": 2988.5,
+ "api_decode_tps": 97.4,
+ "engine_prompt_tps": 3037.5,
+ "engine_decode_tps": 96.6,
+ "log_lines": [
+  "strata serve: prompt 31602 tokens = 0 reused + 31602 read in 10404 ms (3037.5 tok/s), 59 generated in 611 ms (96.6 tok/s), drafts accepted 41 of 52, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (35040 hits / 35040 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what's there.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 7168,
+   "expert_cache_mib": 13437,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 4557,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717812.0454123,
+    "duration_s": 11.0,
+    "finish": "stop",
+    "prompt_tokens": 31602,
+    "reused": 0,
+    "output_tokens": 59,
+    "engine_generated": 59,
+    "prompt_ms": 10404.1,
+    "decode_ms": 610.6,
+    "decode_tok_s": 96.6,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717800.5571585,
+    "duration_s": 10.8,
+    "finish": "stop",
+    "prompt_tokens": 31631,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 10422.5,
+    "decode_ms": 406.8,
+    "decode_tok_s": 110.6,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717789.1846194,
+    "duration_s": 10.7,
+    "finish": "stop",
+    "prompt_tokens": 28316,
+    "reused": 0,
+    "output_tokens": 57,
+    "engine_generated": 57,
+    "prompt_ms": 10118.2,
+    "decode_ms": 608.0,
+    "decode_tok_s": 93.8,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 3,
+  "totals": {
+   "since": 1790717787.625935,
+   "requests": 3,
+   "prompt_tokens": 91549,
+   "reused": 0,
+   "output_tokens": 161,
+   "prompt_ms": 30944.800000000003,
+   "decode_ms": 1625.4
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 48,
+     "mem_used": 21104623616,
+     "mem_total": 25769803776,
+     "temp": 53,
+     "power": 214.931
+    },
+    {
+     "index": 1,
+     "util": 0,
+     "mem_used": 19548536832,
+     "mem_total": 25769803776,
+     "temp": 53,
+     "power": 199.379
+    }
+   ],
+   "gpu_util": 24.0,
+   "gpu_mem_used": 40653160448,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 53,
+   "gpu_power": 414.31,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.666015625,
+   "gpu_pcie_tx_mb": 0.857421875,
+   "cpu": 3.6,
+   "ram_used": 44545851392,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.7062443272525659,
+   "disk_write_mb": 0.11530519628613321,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.5,
+    0.0,
+    0.0,
+    46.5,
+    96.0,
+    72.5,
+    93.0,
+    90.0,
+    95.5,
+    93.5,
+    97.5,
+    41.5,
+    0.0,
+    96.0,
+    93.0,
+    97.5,
+    94.5,
+    97.0,
+    95.0,
+    97.0,
+    93.5,
+    67.0,
+    29.0,
+    50.0,
+    96.5,
+    47.5,
+    95.5,
+    96.5,
+    95.5,
+    62.5,
+    96.5,
+    95.5,
+    50.0,
+    24.0
+   ],
+   "gpu_mem_used": [
+    40447639552,
+    40447639552,
+    40495874048,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40542011392,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448,
+    40653160448
+   ],
+   "gpu_temp": [
+    51,
+    50,
+    50,
+    52,
+    55,
+    54,
+    56,
+    54,
+    56,
+    56,
+    57,
+    54,
+    52,
+    55,
+    54,
+    56,
+    56,
+    56,
+    56,
+    57,
+    57,
+    56,
+    54,
+    53,
+    56,
+    54,
+    57,
+    57,
+    57,
+    56,
+    57,
+    56,
+    57,
+    53
+   ],
+   "gpu_power": [
+    261.46,
+    249.04,
+    248.59,
+    342.07,
+    531.07,
+    533.47,
+    545.68,
+    511.29,
+    533.34,
+    534.58,
+    541.51,
+    420.39,
+    375.37,
+    452.84,
+    528.99,
+    547.53,
+    532.29,
+    524.3,
+    518.92,
+    543.34,
+    532.25,
+    534.03,
+    430.46,
+    354.3,
+    519.1,
+    530.52,
+    534.81,
+    532.5,
+    512.45,
+    532.86,
+    540.0,
+    532.72,
+    513.31,
+    414.31
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.62,
+    182.48,
+    55.88,
+    87.02,
+    80.58,
+    57.5,
+    78.87,
+    83.88,
+    1294.42,
+    46.78,
+    17.17,
+    42.15,
+    1288.27,
+    41.05,
+    75.34,
+    70.76,
+    44.15,
+    79.87,
+    61.61,
+    75.63,
+    42.06,
+    0.62,
+    84.64,
+    74.34,
+    85.5,
+    81.54,
+    0.67,
+    75.72,
+    81.82,
+    82.83,
+    28.8,
+    17.74,
+    0.67
+   ],
+   "cpu": [
+    0.0,
+    6.6,
+    18.0,
+    11.3,
+    10.5,
+    9.8,
+    11.5,
+    13.9,
+    11.4,
+    12.9,
+    7.4,
+    6.1,
+    4.2,
+    6.4,
+    6.0,
+    12.9,
+    11.2,
+    6.1,
+    8.2,
+    8.8,
+    14.0,
+    6.4,
+    4.3,
+    4.2,
+    6.6,
+    6.0,
+    17.8,
+    8.5,
+    6.2,
+    15.1,
+    14.3,
+    6.7,
+    6.9,
+    3.6
+   ],
+   "ram_used": [
+    43799613440,
+    43777974272,
+    43950305280,
+    43962880000,
+    43939622912,
+    43930345472,
+    43932086272,
+    44071780352,
+    44183257088,
+    44182880256,
+    44179062784,
+    44503613440,
+    44015304704,
+    44019281920,
+    44021350400,
+    44034875392,
+    44111585280,
+    44348657664,
+    44344815616,
+    44359872512,
+    44334280704,
+    44327985152,
+    44573519872,
+    44103626752,
+    44105744384,
+    44105732096,
+    44159885312,
+    44275363840,
+    44375539712,
+    44373200896,
+    44318019584,
+    44307988480,
+    44545863680,
+    44545851392
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    129.59,
+    40.83,
+    79.94,
+    84.92,
+    115.02,
+    136.42,
+    117.79,
+    106.04,
+    29.9,
+    1.95,
+    1.72,
+    0.02,
+    0.02,
+    0.05,
+    0.06,
+    0.01,
+    0.03,
+    12.6,
+    101.64,
+    2.94,
+    0.98,
+    0.74,
+    0.02,
+    0.06,
+    0.32,
+    0.08,
+    0.04,
+    0.05,
+    0.04,
+    0.1,
+    0.0,
+    0.71
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    81.42,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790717823.5700235
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K30/speed-32k-1.json

@@ -0,0 +1,608 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K30",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 1,
+ "run_id": "b4895f87f439",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:28:38",
+ "prompt_tokens": 28316,
+ "completion_tokens": 45,
+ "ttft_s": 9.936,
+ "e2e_s": 10.4,
+ "api_prompt_tps": 2849.8,
+ "api_decode_tps": 94.9,
+ "engine_prompt_tps": 2892.0,
+ "engine_decode_tps": 94.1,
+ "log_lines": [
+  "strata serve: prompt 28316 tokens = 0 reused + 28316 read in 9791 ms (2892.0 tok/s), 45 generated in 478 ms (94.1 tok/s), drafts accepted 31 of 37, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (25440 hits / 25440 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what's there.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 7680,
+   "expert_cache_mib": 14575,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 3419,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717318.4067442,
+    "duration_s": 10.3,
+    "finish": "stop",
+    "prompt_tokens": 28316,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 9791.3,
+    "decode_ms": 478.1,
+    "decode_tok_s": 94.1,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717288.2692974,
+    "duration_s": 11.1,
+    "finish": "stop",
+    "prompt_tokens": 28316,
+    "reused": 0,
+    "output_tokens": 56,
+    "engine_generated": 56,
+    "prompt_ms": 10484.4,
+    "decode_ms": 636.0,
+    "decode_tok_s": 88.1,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 2,
+  "totals": {
+   "since": 1790717276.5105639,
+   "requests": 2,
+   "prompt_tokens": 56632,
+   "reused": 0,
+   "output_tokens": 101,
+   "prompt_ms": 20275.699999999997,
+   "decode_ms": 1114.1
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 53,
+     "mem_used": 22306291712,
+     "mem_total": 25769803776,
+     "temp": 51,
+     "power": 200.427
+    },
+    {
+     "index": 1,
+     "util": 46,
+     "mem_used": 18353160192,
+     "mem_total": 25769803776,
+     "temp": 53,
+     "power": 223.879
+    }
+   ],
+   "gpu_util": 49.5,
+   "gpu_mem_used": 40659451904,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 53,
+   "gpu_power": 424.306,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 5.8642578125,
+   "gpu_pcie_tx_mb": 107.4306640625,
+   "cpu": 6.9,
+   "ram_used": 44604223488,
+   "ram_total": 177175990272,
+   "disk_read_mb": 1.1417042620069329,
+   "disk_write_mb": 0.04682067951447989,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    50.0,
+    94.0,
+    63.0,
+    95.0,
+    50.0,
+    92.0,
+    95.0,
+    82.5,
+    97.0,
+    35.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.5,
+    97.5,
+    49.0,
+    94.5,
+    50.0,
+    96.5,
+    53.0,
+    94.5,
+    97.0,
+    49.5
+   ],
+   "gpu_mem_used": [
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40542011392,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40659451904
+   ],
+   "gpu_temp": [
+    49,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    50,
+    52,
+    53,
+    53,
+    53,
+    52,
+    54,
+    51,
+    54,
+    52,
+    50,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    48,
+    49,
+    48,
+    48,
+    48,
+    49,
+    49,
+    49,
+    49,
+    46,
+    46,
+    53,
+    52,
+    54,
+    52,
+    54,
+    52,
+    54,
+    54,
+    53
+   ],
+   "gpu_power": [
+    257.9,
+    244.84,
+    244.4,
+    244.1,
+    244.29,
+    244.43,
+    244.32,
+    244.0,
+    243.79,
+    243.86,
+    244.04,
+    244.43,
+    304.5,
+    504.99,
+    527.59,
+    513.34,
+    519.55,
+    489.82,
+    537.3,
+    515.01,
+    534.71,
+    416.26,
+    390.46,
+    299.77,
+    260.05,
+    241.54,
+    241.56,
+    241.75,
+    241.63,
+    241.51,
+    241.46,
+    241.4,
+    241.29,
+    241.13,
+    241.31,
+    241.69,
+    241.72,
+    221.92,
+    136.12,
+    67.69,
+    469.97,
+    521.96,
+    512.77,
+    514.47,
+    496.64,
+    522.43,
+    513.55,
+    532.56,
+    424.31
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.67,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.67,
+    1.62,
+    0.67,
+    0.62,
+    0.62,
+    0.62,
+    4319.14,
+    134.51,
+    87.31,
+    96.37,
+    111.91,
+    79.87,
+    41.53,
+    80.25,
+    44.15,
+    15.88,
+    0.67,
+    0.57,
+    0.57,
+    0.57,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.57,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.57,
+    0.81,
+    0.81,
+    44.34,
+    97.08,
+    1308.87,
+    41.25,
+    84.97,
+    155.3,
+    46.68,
+    84.26,
+    3604.32,
+    5.86
+   ],
+   "cpu": [
+    0.0,
+    1.4,
+    4.7,
+    7.8,
+    13.7,
+    1.8,
+    1.4,
+    1.9,
+    1.6,
+    1.4,
+    1.5,
+    9.7,
+    13.3,
+    10.3,
+    10.0,
+    9.4,
+    23.0,
+    17.6,
+    10.6,
+    13.1,
+    6.5,
+    6.2,
+    1.3,
+    8.4,
+    5.4,
+    1.3,
+    1.2,
+    1.8,
+    1.6,
+    0.9,
+    10.0,
+    4.2,
+    0.8,
+    1.8,
+    1.6,
+    2.2,
+    2.6,
+    1.9,
+    2.8,
+    7.4,
+    8.7,
+    7.1,
+    5.6,
+    6.2,
+    14.3,
+    8.5,
+    5.8,
+    9.0,
+    6.9
+   ],
+   "ram_used": [
+    43796508672,
+    43800592384,
+    43805933568,
+    43809234944,
+    43832729600,
+    43832090624,
+    43837464576,
+    43835621376,
+    43830910976,
+    43829395456,
+    43825078272,
+    43852685312,
+    44052480000,
+    44036640768,
+    44033179648,
+    44028084224,
+    44053590016,
+    44291584000,
+    44290129920,
+    44269481984,
+    44257243136,
+    44556734464,
+    44502183936,
+    44524433408,
+    44526993408,
+    44523057152,
+    44523831296,
+    44519899136,
+    44508749824,
+    44508250112,
+    44527558656,
+    44529721344,
+    44527198208,
+    44529672192,
+    44530278400,
+    44537171968,
+    44538560512,
+    44524920832,
+    44552687616,
+    44075163648,
+    44090421248,
+    44086198272,
+    44085055488,
+    44084731904,
+    44364992512,
+    44360019968,
+    44359946240,
+    44359090176,
+    44604223488
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    56.63,
+    73.79,
+    64.08,
+    89.26,
+    71.49,
+    160.33,
+    111.7,
+    115.85,
+    97.53,
+    0.0,
+    3.5,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.56,
+    0.02,
+    0.04,
+    0.03,
+    0.05,
+    0.01,
+    0.03,
+    0.01,
+    0.0,
+    1.14
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790717329.186504
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K30/speed-32k-2.json

@@ -0,0 +1,721 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K30",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 2,
+ "run_id": "f81947893a8b",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:28:49",
+ "prompt_tokens": 31632,
+ "completion_tokens": 48,
+ "ttft_s": 11.011,
+ "e2e_s": 11.496,
+ "api_prompt_tps": 2872.7,
+ "api_decode_tps": 97.0,
+ "engine_prompt_tps": 2926.4,
+ "engine_decode_tps": 96.1,
+ "log_lines": [
+  "strata serve: prompt 31632 tokens = 0 reused + 31632 read in 10809 ms (2926.4 tok/s), 48 generated in 499 ms (96.1 tok/s), drafts accepted 32 of 38, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (27360 hits / 27360 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what's there and identify the last file.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 7680,
+   "expert_cache_mib": 14575,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 3419,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717329.364414,
+    "duration_s": 11.3,
+    "finish": "stop",
+    "prompt_tokens": 31632,
+    "reused": 0,
+    "output_tokens": 48,
+    "engine_generated": 48,
+    "prompt_ms": 10809.3,
+    "decode_ms": 499.4,
+    "decode_tok_s": 96.1,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717318.4067442,
+    "duration_s": 10.3,
+    "finish": "stop",
+    "prompt_tokens": 28316,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 9791.3,
+    "decode_ms": 478.1,
+    "decode_tok_s": 94.1,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717288.2692974,
+    "duration_s": 11.1,
+    "finish": "stop",
+    "prompt_tokens": 28316,
+    "reused": 0,
+    "output_tokens": 56,
+    "engine_generated": 56,
+    "prompt_ms": 10484.4,
+    "decode_ms": 636.0,
+    "decode_tok_s": 88.1,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 3,
+  "totals": {
+   "since": 1790717276.5105639,
+   "requests": 3,
+   "prompt_tokens": 88264,
+   "reused": 0,
+   "output_tokens": 149,
+   "prompt_ms": 31084.999999999996,
+   "decode_ms": 1613.5
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 48,
+     "mem_used": 22306291712,
+     "mem_total": 25769803776,
+     "temp": 51,
+     "power": 211.949
+    },
+    {
+     "index": 1,
+     "util": 41,
+     "mem_used": 18353160192,
+     "mem_total": 25769803776,
+     "temp": 54,
+     "power": 238.158
+    }
+   ],
+   "gpu_util": 44.5,
+   "gpu_mem_used": 40659451904,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 54,
+   "gpu_power": 450.10699999999997,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 16.3544921875,
+   "gpu_pcie_tx_mb": 94.6513671875,
+   "cpu": 12.9,
+   "ram_used": 44387663872,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.417992184024029,
+   "disk_write_mb": 0.0972912842124895,
+   "tok_s": 101.46942145535564,
+   "tok_s_mean": 93.06246895838419
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    50.0,
+    94.0,
+    63.0,
+    95.0,
+    50.0,
+    92.0,
+    95.0,
+    82.5,
+    97.0,
+    35.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.5,
+    97.5,
+    49.0,
+    94.5,
+    50.0,
+    96.5,
+    53.0,
+    94.5,
+    97.0,
+    49.5,
+    50.0,
+    97.5,
+    97.0,
+    98.5,
+    68.5,
+    96.0,
+    49.5,
+    96.0,
+    68.5,
+    97.0,
+    44.5
+   ],
+   "gpu_mem_used": [
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40449736704,
+    40542011392,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904
+   ],
+   "gpu_temp": [
+    49,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    50,
+    52,
+    53,
+    53,
+    53,
+    52,
+    54,
+    51,
+    54,
+    52,
+    50,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    48,
+    49,
+    48,
+    48,
+    48,
+    49,
+    49,
+    49,
+    49,
+    46,
+    46,
+    53,
+    52,
+    54,
+    52,
+    54,
+    52,
+    54,
+    54,
+    53,
+    50,
+    54,
+    54,
+    55,
+    54,
+    55,
+    52,
+    55,
+    54,
+    55,
+    54
+   ],
+   "gpu_power": [
+    257.9,
+    244.84,
+    244.4,
+    244.1,
+    244.29,
+    244.43,
+    244.32,
+    244.0,
+    243.79,
+    243.86,
+    244.04,
+    244.43,
+    304.5,
+    504.99,
+    527.59,
+    513.34,
+    519.55,
+    489.82,
+    537.3,
+    515.01,
+    534.71,
+    416.26,
+    390.46,
+    299.77,
+    260.05,
+    241.54,
+    241.56,
+    241.75,
+    241.63,
+    241.51,
+    241.46,
+    241.4,
+    241.29,
+    241.13,
+    241.31,
+    241.69,
+    241.72,
+    221.92,
+    136.12,
+    67.69,
+    469.97,
+    521.96,
+    512.77,
+    514.47,
+    496.64,
+    522.43,
+    513.55,
+    532.56,
+    424.31,
+    362.71,
+    481.71,
+    508.96,
+    525.74,
+    511.3,
+    510.79,
+    506.4,
+    523.98,
+    513.03,
+    516.57,
+    450.11
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.67,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.67,
+    1.62,
+    0.67,
+    0.62,
+    0.62,
+    0.62,
+    4319.14,
+    134.51,
+    87.31,
+    96.37,
+    111.91,
+    79.87,
+    41.53,
+    80.25,
+    44.15,
+    15.88,
+    0.67,
+    0.57,
+    0.57,
+    0.57,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.57,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.57,
+    0.81,
+    0.81,
+    44.34,
+    97.08,
+    1308.87,
+    41.25,
+    84.97,
+    155.3,
+    46.68,
+    84.26,
+    3604.32,
+    5.86,
+    45.25,
+    84.11,
+    42.39,
+    128.98,
+    67.52,
+    42.77,
+    92.46,
+    81.82,
+    43.3,
+    95.32,
+    16.35
+   ],
+   "cpu": [
+    0.0,
+    1.4,
+    4.7,
+    7.8,
+    13.7,
+    1.8,
+    1.4,
+    1.9,
+    1.6,
+    1.4,
+    1.5,
+    9.7,
+    13.3,
+    10.3,
+    10.0,
+    9.4,
+    23.0,
+    17.6,
+    10.6,
+    13.1,
+    6.5,
+    6.2,
+    1.3,
+    8.4,
+    5.4,
+    1.3,
+    1.2,
+    1.8,
+    1.6,
+    0.9,
+    10.0,
+    4.2,
+    0.8,
+    1.8,
+    1.6,
+    2.2,
+    2.6,
+    1.9,
+    2.8,
+    7.4,
+    8.7,
+    7.1,
+    5.6,
+    6.2,
+    14.3,
+    8.5,
+    5.8,
+    9.0,
+    6.9,
+    3.9,
+    5.5,
+    5.8,
+    5.9,
+    6.4,
+    7.5,
+    6.6,
+    6.1,
+    8.7,
+    17.4,
+    12.9
+   ],
+   "ram_used": [
+    43796508672,
+    43800592384,
+    43805933568,
+    43809234944,
+    43832729600,
+    43832090624,
+    43837464576,
+    43835621376,
+    43830910976,
+    43829395456,
+    43825078272,
+    43852685312,
+    44052480000,
+    44036640768,
+    44033179648,
+    44028084224,
+    44053590016,
+    44291584000,
+    44290129920,
+    44269481984,
+    44257243136,
+    44556734464,
+    44502183936,
+    44524433408,
+    44526993408,
+    44523057152,
+    44523831296,
+    44519899136,
+    44508749824,
+    44508250112,
+    44527558656,
+    44529721344,
+    44527198208,
+    44529672192,
+    44530278400,
+    44537171968,
+    44538560512,
+    44524920832,
+    44552687616,
+    44075163648,
+    44090421248,
+    44086198272,
+    44085055488,
+    44084731904,
+    44364992512,
+    44360019968,
+    44359946240,
+    44359090176,
+    44604223488,
+    44141965312,
+    44142555136,
+    44129452032,
+    44121694208,
+    44121939968,
+    44354928640,
+    44349378560,
+    44349894656,
+    44349890560,
+    44144226304,
+    44387663872
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    56.63,
+    73.79,
+    64.08,
+    89.26,
+    71.49,
+    160.33,
+    111.7,
+    115.85,
+    97.53,
+    0.0,
+    3.5,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.56,
+    0.02,
+    0.04,
+    0.03,
+    0.05,
+    0.01,
+    0.03,
+    0.01,
+    0.0,
+    1.14,
+    0.4,
+    0.01,
+    0.0,
+    0.03,
+    0.04,
+    0.01,
+    0.03,
+    12.58,
+    71.94,
+    32.9,
+    0.42
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    101.47
+   ]
+  },
+  "time": 1790717341.185252
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-K30/speed-32k-3.json

@@ -0,0 +1,735 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-K30",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 3,
+ "run_id": "cb947708c5cc",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:29:01",
+ "prompt_tokens": 31602,
+ "completion_tokens": 45,
+ "ttft_s": 10.976,
+ "e2e_s": 11.378,
+ "api_prompt_tps": 2879.2,
+ "api_decode_tps": 109.6,
+ "engine_prompt_tps": 2920.6,
+ "engine_decode_tps": 108.0,
+ "log_lines": [
+  "strata serve: prompt 31602 tokens = 0 reused + 31602 read in 10820 ms (2920.6 tok/s), 45 generated in 417 ms (108.0 tok/s), drafts accepted 33 of 37, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (24480 hits / 24480 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what we're working with.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 7680,
+   "expert_cache_mib": 14575,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 3419,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717341.3189692,
+    "duration_s": 11.2,
+    "finish": "stop",
+    "prompt_tokens": 31602,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 10820.3,
+    "decode_ms": 416.8,
+    "decode_tok_s": 108.0,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717329.364414,
+    "duration_s": 11.3,
+    "finish": "stop",
+    "prompt_tokens": 31632,
+    "reused": 0,
+    "output_tokens": 48,
+    "engine_generated": 48,
+    "prompt_ms": 10809.3,
+    "decode_ms": 499.4,
+    "decode_tok_s": 96.1,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717318.4067442,
+    "duration_s": 10.3,
+    "finish": "stop",
+    "prompt_tokens": 28316,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 9791.3,
+    "decode_ms": 478.1,
+    "decode_tok_s": 94.1,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790717288.2692974,
+    "duration_s": 11.1,
+    "finish": "stop",
+    "prompt_tokens": 28316,
+    "reused": 0,
+    "output_tokens": 56,
+    "engine_generated": 56,
+    "prompt_ms": 10484.4,
+    "decode_ms": 636.0,
+    "decode_tok_s": 88.1,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 4,
+  "totals": {
+   "since": 1790717276.5105639,
+   "requests": 4,
+   "prompt_tokens": 119866,
+   "reused": 0,
+   "output_tokens": 194,
+   "prompt_ms": 41905.299999999996,
+   "decode_ms": 2030.3
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 5,
+     "mem_used": 22306291712,
+     "mem_total": 25769803776,
+     "temp": 49,
+     "power": 227.165
+    },
+    {
+     "index": 1,
+     "util": 42,
+     "mem_used": 18353160192,
+     "mem_total": 25769803776,
+     "temp": 54,
+     "power": 235.691
+    }
+   ],
+   "gpu_util": 23.5,
+   "gpu_mem_used": 40659451904,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 54,
+   "gpu_power": 462.856,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 17.0703125,
+   "gpu_pcie_tx_mb": 56.361328125,
+   "cpu": 5.5,
+   "ram_used": 44383059968,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.38547172180233397,
+   "disk_write_mb": 0.09366602585851105,
+   "tok_s": 121.26075785511577,
+   "tok_s_mean": 112.03069063875886
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.0,
+    50.0,
+    94.0,
+    63.0,
+    95.0,
+    50.0,
+    92.0,
+    95.0,
+    82.5,
+    97.0,
+    35.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.5,
+    97.5,
+    49.0,
+    94.5,
+    50.0,
+    96.5,
+    53.0,
+    94.5,
+    97.0,
+    49.5,
+    50.0,
+    97.5,
+    97.0,
+    98.5,
+    68.5,
+    96.0,
+    49.5,
+    96.0,
+    68.5,
+    97.0,
+    44.5,
+    50.0,
+    85.5,
+    94.5,
+    73.0,
+    79.0,
+    96.0,
+    48.0,
+    95.0,
+    72.5,
+    95.5,
+    23.5
+   ],
+   "gpu_mem_used": [
+    40449736704,
+    40542011392,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40548302848,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40636383232,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904,
+    40659451904
+   ],
+   "gpu_temp": [
+    48,
+    50,
+    52,
+    53,
+    53,
+    53,
+    52,
+    54,
+    51,
+    54,
+    52,
+    50,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    48,
+    49,
+    48,
+    48,
+    48,
+    49,
+    49,
+    49,
+    49,
+    46,
+    46,
+    53,
+    52,
+    54,
+    52,
+    54,
+    52,
+    54,
+    54,
+    53,
+    50,
+    54,
+    54,
+    55,
+    54,
+    55,
+    52,
+    55,
+    54,
+    55,
+    54,
+    51,
+    53,
+    54,
+    53,
+    54,
+    55,
+    54,
+    55,
+    55,
+    55,
+    54
+   ],
+   "gpu_power": [
+    244.43,
+    304.5,
+    504.99,
+    527.59,
+    513.34,
+    519.55,
+    489.82,
+    537.3,
+    515.01,
+    534.71,
+    416.26,
+    390.46,
+    299.77,
+    260.05,
+    241.54,
+    241.56,
+    241.75,
+    241.63,
+    241.51,
+    241.46,
+    241.4,
+    241.29,
+    241.13,
+    241.31,
+    241.69,
+    241.72,
+    221.92,
+    136.12,
+    67.69,
+    469.97,
+    521.96,
+    512.77,
+    514.47,
+    496.64,
+    522.43,
+    513.55,
+    532.56,
+    424.31,
+    362.71,
+    481.71,
+    508.96,
+    525.74,
+    511.3,
+    510.79,
+    506.4,
+    523.98,
+    513.03,
+    516.57,
+    450.11,
+    369.53,
+    481.25,
+    509.29,
+    527.58,
+    511.62,
+    510.74,
+    510.07,
+    527.16,
+    513.42,
+    533.96,
+    462.86
+   ],
+   "gpu_pcie_rx_mb": [
+    0.62,
+    4319.14,
+    134.51,
+    87.31,
+    96.37,
+    111.91,
+    79.87,
+    41.53,
+    80.25,
+    44.15,
+    15.88,
+    0.67,
+    0.57,
+    0.57,
+    0.57,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.57,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.57,
+    0.81,
+    0.81,
+    44.34,
+    97.08,
+    1308.87,
+    41.25,
+    84.97,
+    155.3,
+    46.68,
+    84.26,
+    3604.32,
+    5.86,
+    45.25,
+    84.11,
+    42.39,
+    128.98,
+    67.52,
+    42.77,
+    92.46,
+    81.82,
+    43.3,
+    95.32,
+    16.35,
+    43.1,
+    79.92,
+    41.63,
+    65.23,
+    80.39,
+    37.1,
+    81.92,
+    1273.96,
+    41.01,
+    79.15,
+    17.07
+   ],
+   "cpu": [
+    9.7,
+    13.3,
+    10.3,
+    10.0,
+    9.4,
+    23.0,
+    17.6,
+    10.6,
+    13.1,
+    6.5,
+    6.2,
+    1.3,
+    8.4,
+    5.4,
+    1.3,
+    1.2,
+    1.8,
+    1.6,
+    0.9,
+    10.0,
+    4.2,
+    0.8,
+    1.8,
+    1.6,
+    2.2,
+    2.6,
+    1.9,
+    2.8,
+    7.4,
+    8.7,
+    7.1,
+    5.6,
+    6.2,
+    14.3,
+    8.5,
+    5.8,
+    9.0,
+    6.9,
+    3.9,
+    5.5,
+    5.8,
+    5.9,
+    6.4,
+    7.5,
+    6.6,
+    6.1,
+    8.7,
+    17.4,
+    12.9,
+    4.0,
+    6.3,
+    7.4,
+    6.1,
+    5.4,
+    6.6,
+    5.7,
+    6.7,
+    9.3,
+    6.7,
+    5.5
+   ],
+   "ram_used": [
+    43852685312,
+    44052480000,
+    44036640768,
+    44033179648,
+    44028084224,
+    44053590016,
+    44291584000,
+    44290129920,
+    44269481984,
+    44257243136,
+    44556734464,
+    44502183936,
+    44524433408,
+    44526993408,
+    44523057152,
+    44523831296,
+    44519899136,
+    44508749824,
+    44508250112,
+    44527558656,
+    44529721344,
+    44527198208,
+    44529672192,
+    44530278400,
+    44537171968,
+    44538560512,
+    44524920832,
+    44552687616,
+    44075163648,
+    44090421248,
+    44086198272,
+    44085055488,
+    44084731904,
+    44364992512,
+    44360019968,
+    44359946240,
+    44359090176,
+    44604223488,
+    44141965312,
+    44142555136,
+    44129452032,
+    44121694208,
+    44121939968,
+    44354928640,
+    44349378560,
+    44349894656,
+    44349890560,
+    44144226304,
+    44387663872,
+    44206063616,
+    44194983936,
+    44169220096,
+    44150177792,
+    44142342144,
+    44137152512,
+    44138057728,
+    44126892032,
+    44160794624,
+    44161884160,
+    44383059968
+   ],
+   "disk_read_mb": [
+    56.63,
+    73.79,
+    64.08,
+    89.26,
+    71.49,
+    160.33,
+    111.7,
+    115.85,
+    97.53,
+    0.0,
+    3.5,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.56,
+    0.02,
+    0.04,
+    0.03,
+    0.05,
+    0.01,
+    0.03,
+    0.01,
+    0.0,
+    1.14,
+    0.4,
+    0.01,
+    0.0,
+    0.03,
+    0.04,
+    0.01,
+    0.03,
+    12.58,
+    71.94,
+    32.9,
+    0.42,
+    0.53,
+    0.06,
+    0.03,
+    0.08,
+    0.09,
+    0.04,
+    0.08,
+    0.03,
+    0.02,
+    0.22,
+    0.39
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    101.47,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    121.26
+   ]
+  },
+  "time": 1790717353.0661266
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-auto/speed-32k-1.json

@@ -0,0 +1,585 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-auto",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 1,
+ "run_id": "3e44ccd7ec28",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:18:00",
+ "prompt_tokens": 28314,
+ "completion_tokens": 48,
+ "ttft_s": 11.385,
+ "e2e_s": 11.891,
+ "api_prompt_tps": 2486.9,
+ "api_decode_tps": 93.0,
+ "engine_prompt_tps": 2524.7,
+ "engine_decode_tps": 88.5,
+ "log_lines": [
+  "strata serve: prompt 28314 tokens = 0 reused + 28314 read in 11215 ms (2524.7 tok/s), 48 generated in 542 ms (88.5 tok/s), drafts accepted 34 of 36, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (24958 hits / 24960 lookups)"
+ ],
+ "answer_head": "I'll first look at the repository structure to understand what files exist and identify the last file.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 4864,
+   "expert_cache_mib": 8968,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 9025,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790716680.7795944,
+    "duration_s": 11.8,
+    "finish": "stop",
+    "prompt_tokens": 28314,
+    "reused": 0,
+    "output_tokens": 48,
+    "engine_generated": 48,
+    "prompt_ms": 11214.6,
+    "decode_ms": 542.2,
+    "decode_tok_s": 88.5,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 1,
+  "totals": {
+   "since": 1790716641.243247,
+   "requests": 1,
+   "prompt_tokens": 28314,
+   "reused": 0,
+   "output_tokens": 48,
+   "prompt_ms": 11214.6,
+   "decode_ms": 542.2
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 2,
+     "mem_used": 16383934464,
+     "mem_total": 25769803776,
+     "temp": 43,
+     "power": 147.336
+    },
+    {
+     "index": 1,
+     "util": 7,
+     "mem_used": 24227282944,
+     "mem_total": 25769803776,
+     "temp": 48,
+     "power": 266.007
+    }
+   ],
+   "gpu_util": 4.5,
+   "gpu_mem_used": 40611217408,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 48,
+   "gpu_power": 413.343,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 16.8310546875,
+   "gpu_pcie_tx_mb": 71.810546875,
+   "cpu": 8.4,
+   "ram_used": 44637356032,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.277384309569965,
+   "disk_write_mb": 0.07927603609262536,
+   "tok_s": 96.0473472439008,
+   "tok_s_mean": 90.3424282028167
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.5,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    2.0,
+    48.0,
+    91.5,
+    69.0,
+    77.0,
+    91.0,
+    93.0,
+    49.5,
+    94.0,
+    50.0,
+    4.5
+   ],
+   "gpu_mem_used": [
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40489582592,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40611217408
+   ],
+   "gpu_temp": [
+    44,
+    43,
+    43,
+    43,
+    43,
+    43,
+    44,
+    44,
+    43,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    41,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    43,
+    49,
+    50,
+    50,
+    50,
+    50,
+    50,
+    50,
+    51,
+    51,
+    48
+   ],
+   "gpu_power": [
+    253.73,
+    239.29,
+    238.99,
+    238.62,
+    238.58,
+    238.53,
+    238.39,
+    238.34,
+    238.4,
+    238.51,
+    238.61,
+    238.53,
+    238.49,
+    238.83,
+    238.96,
+    238.97,
+    226.8,
+    148.88,
+    73.97,
+    64.38,
+    63.94,
+    63.69,
+    63.56,
+    63.75,
+    63.53,
+    63.8,
+    63.68,
+    63.62,
+    63.84,
+    63.74,
+    63.81,
+    64.11,
+    63.92,
+    63.72,
+    63.85,
+    63.64,
+    63.78,
+    139.81,
+    418.68,
+    508.54,
+    520.36,
+    495.57,
+    522.4,
+    506.1,
+    501.02,
+    508.77,
+    509.32,
+    413.34
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.67,
+    0.71,
+    0.57,
+    0.62,
+    0.57,
+    0.67,
+    0.57,
+    0.67,
+    0.62,
+    0.57,
+    0.67,
+    0.62,
+    0.57,
+    1.05,
+    0.57,
+    0.76,
+    0.81,
+    0.81,
+    0.81,
+    0.76,
+    0.76,
+    0.81,
+    0.86,
+    1.43,
+    0.81,
+    0.81,
+    0.76,
+    0.86,
+    0.81,
+    0.76,
+    0.95,
+    0.86,
+    0.86,
+    0.76,
+    0.86,
+    0.86,
+    70.24,
+    68.14,
+    64.66,
+    75.24,
+    42.29,
+    84.73,
+    70.48,
+    75.05,
+    54.45,
+    44.44,
+    16.83
+   ],
+   "cpu": [
+    0.0,
+    0.6,
+    1.0,
+    1.6,
+    2.5,
+    1.0,
+    3.1,
+    8.1,
+    6.5,
+    1.2,
+    1.9,
+    2.1,
+    1.5,
+    3.2,
+    2.4,
+    1.3,
+    2.0,
+    1.4,
+    1.3,
+    0.9,
+    0.9,
+    14.0,
+    6.6,
+    2.5,
+    2.7,
+    1.7,
+    1.2,
+    0.6,
+    1.1,
+    1.2,
+    1.7,
+    1.3,
+    1.4,
+    2.7,
+    4.4,
+    12.3,
+    12.4,
+    11.4,
+    12.5,
+    10.3,
+    9.0,
+    12.2,
+    12.1,
+    12.5,
+    11.3,
+    10.4,
+    5.8,
+    8.4
+   ],
+   "ram_used": [
+    43649658880,
+    43652177920,
+    43664764928,
+    43666993152,
+    43672375296,
+    43665145856,
+    43695276032,
+    43759751168,
+    43760455680,
+    43757346816,
+    43754053632,
+    43711803392,
+    43717255168,
+    43692584960,
+    43693260800,
+    43687956480,
+    43682111488,
+    43635728384,
+    43632320512,
+    43636883456,
+    43633815552,
+    43735252992,
+    43881848832,
+    43881005056,
+    43875524608,
+    43879301120,
+    43878330368,
+    43866976256,
+    43862958080,
+    43863728128,
+    43854176256,
+    43855548416,
+    43854200832,
+    43859615744,
+    43895934976,
+    43929411584,
+    43962118144,
+    44169498624,
+    44107767808,
+    44110077952,
+    44109590528,
+    44111515648,
+    44233150464,
+    44346421248,
+    44340047872,
+    44335108096,
+    44329865216,
+    44637356032
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    129.56,
+    55.21,
+    65.53,
+    55.86,
+    128.68,
+    113.77,
+    117.74,
+    106.28,
+    68.13,
+    0.0,
+    2.28
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    96.05
+   ]
+  },
+  "time": 1790716693.048385
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-auto/speed-32k-2.json

@@ -0,0 +1,707 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-auto",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 2,
+ "run_id": "0b6b57dfa6a4",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:18:13",
+ "prompt_tokens": 31634,
+ "completion_tokens": 55,
+ "ttft_s": 11.855,
+ "e2e_s": 12.398,
+ "api_prompt_tps": 2668.4,
+ "api_decode_tps": 99.4,
+ "engine_prompt_tps": 2704.3,
+ "engine_decode_tps": 98.6,
+ "log_lines": [
+  "strata serve: prompt 31634 tokens = 0 reused + 31634 read in 11698 ms (2704.3 tok/s), 55 generated in 558 ms (98.6 tok/s), drafts accepted 38 of 44, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (29759 hits / 29760 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what we're working with.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 4864,
+   "expert_cache_mib": 8968,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 9025,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790716693.1835682,
+    "duration_s": 12.3,
+    "finish": "stop",
+    "prompt_tokens": 31634,
+    "reused": 0,
+    "output_tokens": 55,
+    "engine_generated": 55,
+    "prompt_ms": 11697.7,
+    "decode_ms": 557.8,
+    "decode_tok_s": 98.6,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790716680.7795944,
+    "duration_s": 11.8,
+    "finish": "stop",
+    "prompt_tokens": 28314,
+    "reused": 0,
+    "output_tokens": 48,
+    "engine_generated": 48,
+    "prompt_ms": 11214.6,
+    "decode_ms": 542.2,
+    "decode_tok_s": 88.5,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 2,
+  "totals": {
+   "since": 1790716641.243247,
+   "requests": 2,
+   "prompt_tokens": 59948,
+   "reused": 0,
+   "output_tokens": 103,
+   "prompt_ms": 22912.300000000003,
+   "decode_ms": 1100.0
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 30,
+     "mem_used": 16392323072,
+     "mem_total": 25769803776,
+     "temp": 45,
+     "power": 154.497
+    },
+    {
+     "index": 1,
+     "util": 0,
+     "mem_used": 24241963008,
+     "mem_total": 25769803776,
+     "temp": 47,
+     "power": 250.412
+    }
+   ],
+   "gpu_util": 15.0,
+   "gpu_mem_used": 40634286080,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 47,
+   "gpu_power": 404.909,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 19.310546875,
+   "gpu_pcie_tx_mb": 48.8271484375,
+   "cpu": 8.6,
+   "ram_used": 44481728512,
+   "ram_total": 177175990272,
+   "disk_read_mb": 1.583864660002197,
+   "disk_write_mb": 0.12238954190926069,
+   "tok_s": 112.53532236859756,
+   "tok_s_mean": 97.92814776699883
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.5,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    2.0,
+    48.0,
+    91.5,
+    69.0,
+    77.0,
+    91.0,
+    93.0,
+    49.5,
+    94.0,
+    50.0,
+    4.5,
+    0.5,
+    94.5,
+    73.5,
+    72.5,
+    96.5,
+    49.0,
+    97.5,
+    97.5,
+    48.5,
+    94.0,
+    48.5,
+    15.0
+   ],
+   "gpu_mem_used": [
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40489582592,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40634286080
+   ],
+   "gpu_temp": [
+    44,
+    43,
+    43,
+    43,
+    43,
+    43,
+    44,
+    44,
+    43,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    41,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    43,
+    49,
+    50,
+    50,
+    50,
+    50,
+    50,
+    50,
+    51,
+    51,
+    48,
+    45,
+    51,
+    51,
+    50,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    47
+   ],
+   "gpu_power": [
+    253.73,
+    239.29,
+    238.99,
+    238.62,
+    238.58,
+    238.53,
+    238.39,
+    238.34,
+    238.4,
+    238.51,
+    238.61,
+    238.53,
+    238.49,
+    238.83,
+    238.96,
+    238.97,
+    226.8,
+    148.88,
+    73.97,
+    64.38,
+    63.94,
+    63.69,
+    63.56,
+    63.75,
+    63.53,
+    63.8,
+    63.68,
+    63.62,
+    63.84,
+    63.74,
+    63.81,
+    64.11,
+    63.92,
+    63.72,
+    63.85,
+    63.64,
+    63.78,
+    139.81,
+    418.68,
+    508.54,
+    520.36,
+    495.57,
+    522.4,
+    506.1,
+    501.02,
+    508.77,
+    509.32,
+    413.34,
+    344.59,
+    491.91,
+    520.34,
+    491.14,
+    524.22,
+    495.09,
+    489.28,
+    512.02,
+    509.99,
+    497.54,
+    503.94,
+    404.91
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.67,
+    0.71,
+    0.57,
+    0.62,
+    0.57,
+    0.67,
+    0.57,
+    0.67,
+    0.62,
+    0.57,
+    0.67,
+    0.62,
+    0.57,
+    1.05,
+    0.57,
+    0.76,
+    0.81,
+    0.81,
+    0.81,
+    0.76,
+    0.76,
+    0.81,
+    0.86,
+    1.43,
+    0.81,
+    0.81,
+    0.76,
+    0.86,
+    0.81,
+    0.76,
+    0.95,
+    0.86,
+    0.86,
+    0.76,
+    0.86,
+    0.86,
+    70.24,
+    68.14,
+    64.66,
+    75.24,
+    42.29,
+    84.73,
+    70.48,
+    75.05,
+    54.45,
+    44.44,
+    16.83,
+    43.1,
+    73.43,
+    85.92,
+    85.16,
+    1285.6,
+    83.21,
+    5496.41,
+    73.67,
+    83.68,
+    89.36,
+    52.83,
+    19.31
+   ],
+   "cpu": [
+    0.0,
+    0.6,
+    1.0,
+    1.6,
+    2.5,
+    1.0,
+    3.1,
+    8.1,
+    6.5,
+    1.2,
+    1.9,
+    2.1,
+    1.5,
+    3.2,
+    2.4,
+    1.3,
+    2.0,
+    1.4,
+    1.3,
+    0.9,
+    0.9,
+    14.0,
+    6.6,
+    2.5,
+    2.7,
+    1.7,
+    1.2,
+    0.6,
+    1.1,
+    1.2,
+    1.7,
+    1.3,
+    1.4,
+    2.7,
+    4.4,
+    12.3,
+    12.4,
+    11.4,
+    12.5,
+    10.3,
+    9.0,
+    12.2,
+    12.1,
+    12.5,
+    11.3,
+    10.4,
+    5.8,
+    8.4,
+    5.5,
+    16.9,
+    8.9,
+    8.2,
+    6.7,
+    6.4,
+    6.4,
+    5.5,
+    7.3,
+    10.1,
+    6.1,
+    8.6
+   ],
+   "ram_used": [
+    43649658880,
+    43652177920,
+    43664764928,
+    43666993152,
+    43672375296,
+    43665145856,
+    43695276032,
+    43759751168,
+    43760455680,
+    43757346816,
+    43754053632,
+    43711803392,
+    43717255168,
+    43692584960,
+    43693260800,
+    43687956480,
+    43682111488,
+    43635728384,
+    43632320512,
+    43636883456,
+    43633815552,
+    43735252992,
+    43881848832,
+    43881005056,
+    43875524608,
+    43879301120,
+    43878330368,
+    43866976256,
+    43862958080,
+    43863728128,
+    43854176256,
+    43855548416,
+    43854200832,
+    43859615744,
+    43895934976,
+    43929411584,
+    43962118144,
+    44169498624,
+    44107767808,
+    44110077952,
+    44109590528,
+    44111515648,
+    44233150464,
+    44346421248,
+    44340047872,
+    44335108096,
+    44329865216,
+    44637356032,
+    44551315456,
+    44588969984,
+    44562165760,
+    44546224128,
+    44540850176,
+    44543295488,
+    44547391488,
+    44375429120,
+    44370026496,
+    44363567104,
+    44368113664,
+    44481728512
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    129.56,
+    55.21,
+    65.53,
+    55.86,
+    128.68,
+    113.77,
+    117.74,
+    106.28,
+    68.13,
+    0.0,
+    2.28,
+    1.11,
+    0.12,
+    0.02,
+    0.05,
+    0.05,
+    0.01,
+    0.03,
+    0.0,
+    25.35,
+    92.15,
+    0.01,
+    1.58
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    96.05,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    112.54
+   ]
+  },
+  "time": 1790716705.9498026
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-Coder-2x3090-auto/speed-32k-3.json

@@ -0,0 +1,721 @@
+{
+ "label": "native-v0.1.26-Coder-2x3090-auto",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 3,
+ "run_id": "caa0b992dd74",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:18:25",
+ "prompt_tokens": 31601,
+ "completion_tokens": 45,
+ "ttft_s": 11.847,
+ "e2e_s": 12.249,
+ "api_prompt_tps": 2667.3,
+ "api_decode_tps": 109.5,
+ "engine_prompt_tps": 2704.5,
+ "engine_decode_tps": 108.1,
+ "log_lines": [
+  "strata serve: prompt 31601 tokens = 0 reused + 31601 read in 11684 ms (2704.5 tok/s), 45 generated in 416 ms (108.1 tok/s), drafts accepted 33 of 36, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 100.0% (24000 hits / 24000 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what we're working with.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-coder-iq1_m",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 4864,
+   "expert_cache_mib": 8968,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 9025,
+   "cvec": 0,
+   "arena_mib": 23981,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790716706.0904436,
+    "duration_s": 12.1,
+    "finish": "stop",
+    "prompt_tokens": 31601,
+    "reused": 0,
+    "output_tokens": 45,
+    "engine_generated": 45,
+    "prompt_ms": 11684.5,
+    "decode_ms": 416.2,
+    "decode_tok_s": 108.1,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790716693.1835682,
+    "duration_s": 12.3,
+    "finish": "stop",
+    "prompt_tokens": 31634,
+    "reused": 0,
+    "output_tokens": 55,
+    "engine_generated": 55,
+    "prompt_ms": 11697.7,
+    "decode_ms": 557.8,
+    "decode_tok_s": 98.6,
+    "hit_rate": 1.0
+   },
+   {
+    "projection": null,
+    "time": 1790716680.7795944,
+    "duration_s": 11.8,
+    "finish": "stop",
+    "prompt_tokens": 28314,
+    "reused": 0,
+    "output_tokens": 48,
+    "engine_generated": 48,
+    "prompt_ms": 11214.6,
+    "decode_ms": 542.2,
+    "decode_tok_s": 88.5,
+    "hit_rate": 1.0
+   }
+  ],
+  "requests_kept": 3,
+  "totals": {
+   "since": 1790716641.243247,
+   "requests": 3,
+   "prompt_tokens": 91549,
+   "reused": 0,
+   "output_tokens": 148,
+   "prompt_ms": 34596.8,
+   "decode_ms": 1516.2
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 30,
+     "mem_used": 16392323072,
+     "mem_total": 25769803776,
+     "temp": 46,
+     "power": 161.526
+    },
+    {
+     "index": 1,
+     "util": 0,
+     "mem_used": 24241963008,
+     "mem_total": 25769803776,
+     "temp": 48,
+     "power": 223.689
+    }
+   ],
+   "gpu_util": 15.0,
+   "gpu_mem_used": 40634286080,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 48,
+   "gpu_power": 385.21500000000003,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.5703125,
+   "gpu_pcie_tx_mb": 0.857421875,
+   "cpu": 6.1,
+   "ram_used": 44274470912,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.14419269932051038,
+   "disk_write_mb": 0.2595468587769187,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    2.0,
+    48.0,
+    91.5,
+    69.0,
+    77.0,
+    91.0,
+    93.0,
+    49.5,
+    94.0,
+    50.0,
+    4.5,
+    0.5,
+    94.5,
+    73.5,
+    72.5,
+    96.5,
+    49.0,
+    97.5,
+    97.5,
+    48.5,
+    94.0,
+    48.5,
+    15.0,
+    0.0,
+    97.5,
+    49.0,
+    93.5,
+    72.5,
+    70.5,
+    97.0,
+    95.5,
+    50.0,
+    96.0,
+    50.0,
+    15.0
+   ],
+   "gpu_mem_used": [
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40426668032,
+    40489582592,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40521039872,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40611217408,
+    40634286080,
+    40634286080,
+    40634286080,
+    40634286080,
+    40634286080,
+    40634286080,
+    40634286080,
+    40634286080,
+    40634286080,
+    40634286080,
+    40634286080,
+    40634286080,
+    40634286080
+   ],
+   "gpu_temp": [
+    44,
+    44,
+    44,
+    44,
+    44,
+    41,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    43,
+    49,
+    50,
+    50,
+    50,
+    50,
+    50,
+    50,
+    51,
+    51,
+    48,
+    45,
+    51,
+    51,
+    50,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    47,
+    45,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    52,
+    52,
+    48
+   ],
+   "gpu_power": [
+    238.49,
+    238.83,
+    238.96,
+    238.97,
+    226.8,
+    148.88,
+    73.97,
+    64.38,
+    63.94,
+    63.69,
+    63.56,
+    63.75,
+    63.53,
+    63.8,
+    63.68,
+    63.62,
+    63.84,
+    63.74,
+    63.81,
+    64.11,
+    63.92,
+    63.72,
+    63.85,
+    63.64,
+    63.78,
+    139.81,
+    418.68,
+    508.54,
+    520.36,
+    495.57,
+    522.4,
+    506.1,
+    501.02,
+    508.77,
+    509.32,
+    413.34,
+    344.59,
+    491.91,
+    520.34,
+    491.14,
+    524.22,
+    495.09,
+    489.28,
+    512.02,
+    509.99,
+    497.54,
+    503.94,
+    404.91,
+    336.42,
+    502.95,
+    510.85,
+    498.42,
+    519.36,
+    492.86,
+    501.0,
+    522.94,
+    497.46,
+    508.62,
+    494.37,
+    385.22
+   ],
+   "gpu_pcie_rx_mb": [
+    0.62,
+    0.57,
+    1.05,
+    0.57,
+    0.76,
+    0.81,
+    0.81,
+    0.81,
+    0.76,
+    0.76,
+    0.81,
+    0.86,
+    1.43,
+    0.81,
+    0.81,
+    0.76,
+    0.86,
+    0.81,
+    0.76,
+    0.95,
+    0.86,
+    0.86,
+    0.76,
+    0.86,
+    0.86,
+    70.24,
+    68.14,
+    64.66,
+    75.24,
+    42.29,
+    84.73,
+    70.48,
+    75.05,
+    54.45,
+    44.44,
+    16.83,
+    43.1,
+    73.43,
+    85.92,
+    85.16,
+    1285.6,
+    83.21,
+    5496.41,
+    73.67,
+    83.68,
+    89.36,
+    52.83,
+    19.31,
+    45.11,
+    82.73,
+    85.31,
+    43.06,
+    94.36,
+    41.63,
+    73.48,
+    42.01,
+    87.26,
+    45.92,
+    121.97,
+    0.57
+   ],
+   "cpu": [
+    1.5,
+    3.2,
+    2.4,
+    1.3,
+    2.0,
+    1.4,
+    1.3,
+    0.9,
+    0.9,
+    14.0,
+    6.6,
+    2.5,
+    2.7,
+    1.7,
+    1.2,
+    0.6,
+    1.1,
+    1.2,
+    1.7,
+    1.3,
+    1.4,
+    2.7,
+    4.4,
+    12.3,
+    12.4,
+    11.4,
+    12.5,
+    10.3,
+    9.0,
+    12.2,
+    12.1,
+    12.5,
+    11.3,
+    10.4,
+    5.8,
+    8.4,
+    5.5,
+    16.9,
+    8.9,
+    8.2,
+    6.7,
+    6.4,
+    6.4,
+    5.5,
+    7.3,
+    10.1,
+    6.1,
+    8.6,
+    3.1,
+    6.2,
+    9.6,
+    17.9,
+    6.1,
+    7.4,
+    6.0,
+    6.4,
+    6.4,
+    5.7,
+    5.9,
+    6.1
+   ],
+   "ram_used": [
+    43717255168,
+    43692584960,
+    43693260800,
+    43687956480,
+    43682111488,
+    43635728384,
+    43632320512,
+    43636883456,
+    43633815552,
+    43735252992,
+    43881848832,
+    43881005056,
+    43875524608,
+    43879301120,
+    43878330368,
+    43866976256,
+    43862958080,
+    43863728128,
+    43854176256,
+    43855548416,
+    43854200832,
+    43859615744,
+    43895934976,
+    43929411584,
+    43962118144,
+    44169498624,
+    44107767808,
+    44110077952,
+    44109590528,
+    44111515648,
+    44233150464,
+    44346421248,
+    44340047872,
+    44335108096,
+    44329865216,
+    44637356032,
+    44551315456,
+    44588969984,
+    44562165760,
+    44546224128,
+    44540850176,
+    44543295488,
+    44547391488,
+    44375429120,
+    44370026496,
+    44363567104,
+    44368113664,
+    44481728512,
+    44224720896,
+    44216463360,
+    44188299264,
+    44025966592,
+    44017278976,
+    44014284800,
+    44060004352,
+    44042358784,
+    44044165120,
+    44044423168,
+    44044165120,
+    44274470912
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    129.56,
+    55.21,
+    65.53,
+    55.86,
+    128.68,
+    113.77,
+    117.74,
+    106.28,
+    68.13,
+    0.0,
+    2.28,
+    1.11,
+    0.12,
+    0.02,
+    0.05,
+    0.05,
+    0.01,
+    0.03,
+    0.0,
+    25.35,
+    92.15,
+    0.01,
+    1.58,
+    0.57,
+    0.06,
+    0.06,
+    0.05,
+    0.08,
+    0.04,
+    0.07,
+    0.01,
+    0.04,
+    0.15,
+    0.0,
+    0.14
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    96.05,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    112.54,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790716718.7022452
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_S-2x3090-auto-needle/needle-16k-1.json

@@ -0,0 +1,750 @@
+{
+ "label": "native-v0.1.26-IQ3_S-2x3090-auto-needle",
+ "mode": "needle",
+ "ctx": "16k",
+ "target_tokens": 16384,
+ "run": 1,
+ "run_id": "1e52cb484c7f",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:55:17",
+ "prompt_tokens": 17345,
+ "completion_tokens": 8,
+ "ttft_s": 23.389,
+ "e2e_s": 23.454,
+ "api_prompt_tps": 741.6,
+ "api_decode_tps": 106.6,
+ "engine_prompt_tps": 744.7,
+ "engine_decode_tps": 101.7,
+ "log_lines": [
+  "strata serve: prompt 17345 tokens = 0 reused + 17345 read in 23292 ms (744.7 tok/s), 8 generated in 79 ms (101.7 tok/s), drafts accepted 6 of 6, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 96.3% (4159 hits / 4317 lookups)"
+ ],
+ "answer_head": "A466BDE8",
+ "needle_found": true,
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_s",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 9376,
+   "expert_cache_mib": 17297,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 635,
+   "cvec": 0,
+   "arena_mib": 47962,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790718917.8830457,
+    "duration_s": 23.4,
+    "finish": "stop",
+    "prompt_tokens": 17345,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 23292.2,
+    "decode_ms": 78.7,
+    "decode_tok_s": 101.7,
+    "hit_rate": 0.963
+   },
+   {
+    "projection": null,
+    "time": 1790718906.5660036,
+    "duration_s": 10.7,
+    "finish": "stop",
+    "prompt_tokens": 7491,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 10625.3,
+    "decode_ms": 109.4,
+    "decode_tok_s": 73.1,
+    "hit_rate": 0.902
+   },
+   {
+    "projection": null,
+    "time": 1790718828.4807332,
+    "duration_s": 42.6,
+    "finish": "stop",
+    "prompt_tokens": 31604,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 42110.8,
+    "decode_ms": 460.9,
+    "decode_tok_s": 106.3,
+    "hit_rate": 0.991
+   },
+   {
+    "projection": null,
+    "time": 1790718784.8375118,
+    "duration_s": 43.0,
+    "finish": "stop",
+    "prompt_tokens": 31633,
+    "reused": 0,
+    "output_tokens": 88,
+    "engine_generated": 88,
+    "prompt_ms": 42216.7,
+    "decode_ms": 776.7,
+    "decode_tok_s": 113.3,
+    "hit_rate": 0.978
+   },
+   {
+    "projection": null,
+    "time": 1790718743.8685048,
+    "duration_s": 40.3,
+    "finish": "stop",
+    "prompt_tokens": 28315,
+    "reused": 0,
+    "output_tokens": 84,
+    "engine_generated": 84,
+    "prompt_ms": 39286.5,
+    "decode_ms": 1011.1,
+    "decode_tok_s": 83.1,
+    "hit_rate": 0.949
+   }
+  ],
+  "requests_kept": 5,
+  "totals": {
+   "since": 1790718725.4039552,
+   "requests": 5,
+   "prompt_tokens": 116388,
+   "reused": 0,
+   "output_tokens": 237,
+   "prompt_ms": 157531.5,
+   "decode_ms": 2436.8
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 5,
+     "mem_used": 25212944384,
+     "mem_total": 25769803776,
+     "temp": 46,
+     "power": 125.093
+    },
+    {
+     "index": 1,
+     "util": 0,
+     "mem_used": 24227282944,
+     "mem_total": 25769803776,
+     "temp": 51,
+     "power": 215.964
+    }
+   ],
+   "gpu_util": 2.5,
+   "gpu_mem_used": 49440227328,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 51,
+   "gpu_power": 341.057,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.666015625,
+   "gpu_pcie_tx_mb": 0.9052734375,
+   "cpu": 16.5,
+   "ram_used": 70477971456,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.17303348536954039,
+   "disk_write_mb": 0.0036048642785320916,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    37.0,
+    28.0,
+    22.5,
+    30.0,
+    37.0,
+    18.0,
+    29.0,
+    22.0,
+    21.0,
+    0.5,
+    50.0,
+    34.5,
+    25.0,
+    60.5,
+    31.5,
+    26.0,
+    33.5,
+    46.5,
+    25.0,
+    31.5,
+    21.0,
+    30.0,
+    35.0,
+    37.0,
+    32.5,
+    22.0,
+    26.5,
+    19.5,
+    49.5,
+    26.5,
+    21.0,
+    2.5
+   ],
+   "gpu_mem_used": [
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328
+   ],
+   "gpu_temp": [
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    45,
+    45,
+    45,
+    45,
+    45,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    50,
+    49,
+    51,
+    51,
+    51,
+    51,
+    50,
+    51,
+    52,
+    50,
+    49,
+    49,
+    51,
+    52,
+    50,
+    52,
+    51,
+    51,
+    51,
+    51,
+    52,
+    51,
+    52,
+    51,
+    51,
+    52,
+    52,
+    51,
+    52,
+    52,
+    53,
+    51
+   ],
+   "gpu_power": [
+    242.53,
+    242.44,
+    242.38,
+    242.52,
+    242.49,
+    242.46,
+    242.59,
+    242.37,
+    242.36,
+    242.5,
+    242.54,
+    140.5,
+    84.6,
+    65.59,
+    65.03,
+    64.81,
+    64.81,
+    64.9,
+    64.51,
+    64.65,
+    64.87,
+    64.67,
+    64.8,
+    64.5,
+    64.56,
+    64.64,
+    64.52,
+    64.48,
+    252.37,
+    388.02,
+    363.78,
+    375.15,
+    399.66,
+    368.5,
+    396.75,
+    362.75,
+    365.62,
+    336.59,
+    296.73,
+    407.69,
+    367.08,
+    376.76,
+    397.12,
+    384.8,
+    382.99,
+    408.25,
+    371.72,
+    397.41,
+    373.98,
+    388.78,
+    413.99,
+    362.47,
+    402.46,
+    358.68,
+    403.34,
+    387.12,
+    369.06,
+    426.5,
+    375.9,
+    341.06
+   ],
+   "gpu_pcie_rx_mb": [
+    0.62,
+    0.62,
+    0.67,
+    0.67,
+    0.67,
+    0.62,
+    0.62,
+    0.57,
+    0.62,
+    0.67,
+    0.71,
+    0.81,
+    0.81,
+    0.86,
+    0.81,
+    0.86,
+    0.81,
+    0.86,
+    0.76,
+    0.81,
+    0.86,
+    0.81,
+    0.95,
+    0.81,
+    0.86,
+    0.86,
+    0.86,
+    0.71,
+    7078.07,
+    5438.42,
+    7891.61,
+    5618.71,
+    5824.99,
+    6656.5,
+    6553.84,
+    1295.8,
+    3929.76,
+    0.67,
+    7089.76,
+    6013.73,
+    7916.07,
+    5803.39,
+    5749.08,
+    4295.01,
+    6229.4,
+    7158.52,
+    6448.94,
+    8972.07,
+    4133.89,
+    5762.81,
+    4218.58,
+    7081.13,
+    5446.96,
+    19399.45,
+    7164.1,
+    6657.41,
+    4340.22,
+    8272.65,
+    7551.43,
+    0.67
+   ],
+   "cpu": [
+    1.4,
+    3.7,
+    2.4,
+    2.6,
+    2.2,
+    1.6,
+    7.0,
+    7.8,
+    0.8,
+    3.8,
+    6.2,
+    1.4,
+    1.5,
+    1.3,
+    0.5,
+    0.8,
+    1.4,
+    1.7,
+    1.4,
+    1.5,
+    8.2,
+    9.2,
+    2.9,
+    5.7,
+    1.3,
+    5.6,
+    1.2,
+    7.3,
+    6.3,
+    9.1,
+    6.4,
+    7.4,
+    6.7,
+    6.4,
+    16.3,
+    9.8,
+    6.3,
+    19.7,
+    4.9,
+    6.7,
+    5.5,
+    6.3,
+    6.2,
+    6.4,
+    7.1,
+    6.3,
+    6.6,
+    6.4,
+    16.7,
+    8.7,
+    6.4,
+    6.7,
+    6.7,
+    6.4,
+    6.1,
+    7.1,
+    5.5,
+    9.0,
+    6.7,
+    16.5
+   ],
+   "ram_used": [
+    70616137728,
+    70634639360,
+    70632943616,
+    70619291648,
+    70622433280,
+    70624546816,
+    70651342848,
+    70754201600,
+    70754394112,
+    70772871168,
+    70811869184,
+    70806048768,
+    70788648960,
+    70775001088,
+    70770065408,
+    70761766912,
+    70762528768,
+    70758371328,
+    70767398912,
+    70768398336,
+    70723190784,
+    70684524544,
+    70713102336,
+    70694293504,
+    70656102400,
+    70685872128,
+    70679355392,
+    70251024384,
+    70250827776,
+    70240993280,
+    70217400320,
+    70216302592,
+    70220156928,
+    70211788800,
+    70218944512,
+    70240178176,
+    70232375296,
+    70472826880,
+    70432313344,
+    70401486848,
+    70392971264,
+    70382305280,
+    70365552640,
+    70356713472,
+    70348333056,
+    70341337088,
+    70134956032,
+    70120873984,
+    70109106176,
+    70101512192,
+    70084435968,
+    70076407808,
+    70066089984,
+    70059646976,
+    70027522048,
+    70087983104,
+    70087725056,
+    70115901440,
+    70239928320,
+    70477971456
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.04,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    1.46,
+    0.01,
+    0.0,
+    0.0,
+    0.82,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.2,
+    0.52,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.59,
+    0.0,
+    0.0,
+    0.02,
+    0.0,
+    0.04,
+    0.0,
+    0.0,
+    0.05,
+    0.0,
+    0.22,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.17
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790718941.76262
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_S-2x3090-auto-needle/needle-28k-1.json

@@ -0,0 +1,764 @@
+{
+ "label": "native-v0.1.26-IQ3_S-2x3090-auto-needle",
+ "mode": "needle",
+ "ctx": "28k",
+ "target_tokens": 28672,
+ "run": 1,
+ "run_id": "c61aafff2d90",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:55:41",
+ "prompt_tokens": 29827,
+ "completion_tokens": 8,
+ "ttft_s": 39.958,
+ "e2e_s": 40.036,
+ "api_prompt_tps": 746.5,
+ "api_decode_tps": 89.7,
+ "engine_prompt_tps": 749.2,
+ "engine_decode_tps": 85.9,
+ "log_lines": [
+  "strata serve: prompt 29827 tokens = 0 reused + 29827 read in 39812 ms (749.2 tok/s), 8 generated in 93 ms (85.9 tok/s), drafts accepted 6 of 6, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 96.2% (4154 hits / 4318 lookups)"
+ ],
+ "answer_head": "7BDBA3C1",
+ "needle_found": true,
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_s",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 9376,
+   "expert_cache_mib": 17297,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 635,
+   "cvec": 0,
+   "arena_mib": 47962,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790718941.8879943,
+    "duration_s": 39.9,
+    "finish": "stop",
+    "prompt_tokens": 29827,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 39812.1,
+    "decode_ms": 93.1,
+    "decode_tok_s": 85.9,
+    "hit_rate": 0.962
+   },
+   {
+    "projection": null,
+    "time": 1790718917.8830457,
+    "duration_s": 23.4,
+    "finish": "stop",
+    "prompt_tokens": 17345,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 23292.2,
+    "decode_ms": 78.7,
+    "decode_tok_s": 101.7,
+    "hit_rate": 0.963
+   },
+   {
+    "projection": null,
+    "time": 1790718906.5660036,
+    "duration_s": 10.7,
+    "finish": "stop",
+    "prompt_tokens": 7491,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 10625.3,
+    "decode_ms": 109.4,
+    "decode_tok_s": 73.1,
+    "hit_rate": 0.902
+   },
+   {
+    "projection": null,
+    "time": 1790718828.4807332,
+    "duration_s": 42.6,
+    "finish": "stop",
+    "prompt_tokens": 31604,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 42110.8,
+    "decode_ms": 460.9,
+    "decode_tok_s": 106.3,
+    "hit_rate": 0.991
+   },
+   {
+    "projection": null,
+    "time": 1790718784.8375118,
+    "duration_s": 43.0,
+    "finish": "stop",
+    "prompt_tokens": 31633,
+    "reused": 0,
+    "output_tokens": 88,
+    "engine_generated": 88,
+    "prompt_ms": 42216.7,
+    "decode_ms": 776.7,
+    "decode_tok_s": 113.3,
+    "hit_rate": 0.978
+   },
+   {
+    "projection": null,
+    "time": 1790718743.8685048,
+    "duration_s": 40.3,
+    "finish": "stop",
+    "prompt_tokens": 28315,
+    "reused": 0,
+    "output_tokens": 84,
+    "engine_generated": 84,
+    "prompt_ms": 39286.5,
+    "decode_ms": 1011.1,
+    "decode_tok_s": 83.1,
+    "hit_rate": 0.949
+   }
+  ],
+  "requests_kept": 6,
+  "totals": {
+   "since": 1790718725.4039552,
+   "requests": 6,
+   "prompt_tokens": 146215,
+   "reused": 0,
+   "output_tokens": 245,
+   "prompt_ms": 197343.6,
+   "decode_ms": 2529.9
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 0,
+     "mem_used": 25212944384,
+     "mem_total": 25769803776,
+     "temp": 47,
+     "power": 130.942
+    },
+    {
+     "index": 1,
+     "util": 42,
+     "mem_used": 24227282944,
+     "mem_total": 25769803776,
+     "temp": 53,
+     "power": 227.639
+    }
+   ],
+   "gpu_util": 21.0,
+   "gpu_mem_used": 49440227328,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 53,
+   "gpu_power": 358.581,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.666015625,
+   "gpu_pcie_tx_mb": 0.857421875,
+   "cpu": 6.3,
+   "ram_used": 70479159296,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.0,
+   "disk_write_mb": 0.014414807554752778,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.5,
+    50.0,
+    34.5,
+    25.0,
+    60.5,
+    31.5,
+    26.0,
+    33.5,
+    46.5,
+    25.0,
+    31.5,
+    21.0,
+    30.0,
+    35.0,
+    37.0,
+    32.5,
+    22.0,
+    26.5,
+    19.5,
+    49.5,
+    26.5,
+    21.0,
+    2.5,
+    50.5,
+    42.0,
+    43.0,
+    38.0,
+    32.0,
+    21.5,
+    29.0,
+    34.0,
+    13.5,
+    36.0,
+    23.0,
+    30.0,
+    11.0,
+    77.5,
+    34.5,
+    23.5,
+    29.0,
+    17.5,
+    69.0,
+    34.5,
+    23.5,
+    32.5,
+    24.5,
+    57.5,
+    32.0,
+    26.0,
+    33.5,
+    25.0,
+    32.0,
+    32.5,
+    22.5,
+    36.0,
+    37.0,
+    36.5,
+    28.5,
+    42.5,
+    21.0
+   ],
+   "gpu_mem_used": [
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328
+   ],
+   "gpu_temp": [
+    50,
+    49,
+    49,
+    51,
+    52,
+    50,
+    52,
+    51,
+    51,
+    51,
+    51,
+    52,
+    51,
+    52,
+    51,
+    51,
+    52,
+    52,
+    51,
+    52,
+    52,
+    53,
+    51,
+    50,
+    50,
+    51,
+    53,
+    51,
+    52,
+    52,
+    51,
+    52,
+    51,
+    52,
+    52,
+    52,
+    53,
+    52,
+    52,
+    51,
+    51,
+    53,
+    52,
+    52,
+    52,
+    52,
+    52,
+    51,
+    53,
+    52,
+    52,
+    52,
+    52,
+    53,
+    52,
+    53,
+    52,
+    53,
+    52,
+    53
+   ],
+   "gpu_power": [
+    336.59,
+    296.73,
+    407.69,
+    367.08,
+    376.76,
+    397.12,
+    384.8,
+    382.99,
+    408.25,
+    371.72,
+    397.41,
+    373.98,
+    388.78,
+    413.99,
+    362.47,
+    402.46,
+    358.68,
+    403.34,
+    387.12,
+    369.06,
+    426.5,
+    375.9,
+    341.06,
+    316.12,
+    396.45,
+    370.14,
+    378.61,
+    411.6,
+    381.87,
+    383.44,
+    399.03,
+    378.32,
+    399.6,
+    372.43,
+    400.43,
+    390.19,
+    384.29,
+    405.22,
+    398.65,
+    405.33,
+    377.77,
+    347.82,
+    408.44,
+    395.16,
+    399.76,
+    391.18,
+    383.68,
+    395.97,
+    381.79,
+    371.58,
+    387.34,
+    367.36,
+    390.18,
+    377.99,
+    404.54,
+    406.77,
+    371.21,
+    404.91,
+    389.57,
+    358.58
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    7089.76,
+    6013.73,
+    7916.07,
+    5803.39,
+    5749.08,
+    4295.01,
+    6229.4,
+    7158.52,
+    6448.94,
+    8972.07,
+    4133.89,
+    5762.81,
+    4218.58,
+    7081.13,
+    5446.96,
+    19399.45,
+    7164.1,
+    6657.41,
+    4340.22,
+    8272.65,
+    7551.43,
+    0.67,
+    5912.64,
+    6668.66,
+    4836.32,
+    15187.07,
+    6919.96,
+    7012.13,
+    4668.04,
+    8340.98,
+    5615.52,
+    6986.24,
+    8358.29,
+    8231.35,
+    7154.13,
+    6558.46,
+    5322.65,
+    14744.81,
+    6437.3,
+    7645.75,
+    4067.04,
+    5734.82,
+    1.62,
+    5507.04,
+    5994.7,
+    5715.7,
+    6093.12,
+    89.88,
+    6215.57,
+    4331.35,
+    6045.63,
+    5971.19,
+    6178.04,
+    5846.59,
+    7484.53,
+    5744.65,
+    5095.67,
+    7809.07,
+    0.67
+   ],
+   "cpu": [
+    19.7,
+    4.9,
+    6.7,
+    5.5,
+    6.3,
+    6.2,
+    6.4,
+    7.1,
+    6.3,
+    6.6,
+    6.4,
+    16.7,
+    8.7,
+    6.4,
+    6.7,
+    6.7,
+    6.4,
+    6.1,
+    7.1,
+    5.5,
+    9.0,
+    6.7,
+    16.5,
+    4.3,
+    8.5,
+    17.0,
+    10.5,
+    8.0,
+    8.5,
+    6.2,
+    6.8,
+    6.0,
+    6.0,
+    5.6,
+    6.4,
+    7.1,
+    5.6,
+    6.3,
+    11.3,
+    13.8,
+    13.6,
+    7.0,
+    5.8,
+    6.1,
+    6.1,
+    6.3,
+    6.2,
+    6.0,
+    9.3,
+    6.5,
+    6.1,
+    6.0,
+    14.5,
+    11.7,
+    5.6,
+    8.6,
+    7.7,
+    6.1,
+    6.7,
+    6.3
+   ],
+   "ram_used": [
+    70472826880,
+    70432313344,
+    70401486848,
+    70392971264,
+    70382305280,
+    70365552640,
+    70356713472,
+    70348333056,
+    70341337088,
+    70134956032,
+    70120873984,
+    70109106176,
+    70101512192,
+    70084435968,
+    70076407808,
+    70066089984,
+    70059646976,
+    70027522048,
+    70087983104,
+    70087725056,
+    70115901440,
+    70239928320,
+    70477971456,
+    70343557120,
+    70337265664,
+    70309584896,
+    70434922496,
+    70428250112,
+    70420987904,
+    70412926976,
+    70409207808,
+    70400454656,
+    70391336960,
+    70383079424,
+    70374731776,
+    70367465472,
+    70359465984,
+    70354010112,
+    70334316544,
+    70343401472,
+    70345613312,
+    70344155136,
+    70340018176,
+    70356168704,
+    70375772160,
+    70372450304,
+    70371860480,
+    70366474240,
+    70373842944,
+    70374035456,
+    70365605888,
+    70363926528,
+    70340509696,
+    70341251072,
+    70340546560,
+    70343122944,
+    70346379264,
+    70347120640,
+    70345048064,
+    70479159296
+   ],
+   "disk_read_mb": [
+    0.2,
+    0.52,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.59,
+    0.0,
+    0.0,
+    0.02,
+    0.0,
+    0.04,
+    0.0,
+    0.0,
+    0.05,
+    0.0,
+    0.22,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.17,
+    0.53,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.6,
+    0.0,
+    0.03,
+    0.0,
+    0.0,
+    0.05,
+    0.0,
+    0.0,
+    0.05,
+    0.0,
+    0.05,
+    0.0,
+    0.0,
+    0.03,
+    0.0,
+    0.04,
+    0.0,
+    0.0,
+    0.18,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790718982.302463
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_S-2x3090-auto-needle/needle-8k-1.json

@@ -0,0 +1,736 @@
+{
+ "label": "native-v0.1.26-IQ3_S-2x3090-auto-needle",
+ "mode": "needle",
+ "ctx": "8k",
+ "target_tokens": 8192,
+ "run": 1,
+ "run_id": "7c040dd4c6d7",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:55:06",
+ "prompt_tokens": 7491,
+ "completion_tokens": 8,
+ "ttft_s": 10.682,
+ "e2e_s": 10.776,
+ "api_prompt_tps": 701.3,
+ "api_decode_tps": 74.7,
+ "engine_prompt_tps": 705.0,
+ "engine_decode_tps": 73.1,
+ "log_lines": [
+  "strata serve: prompt 7491 tokens = 0 reused + 7491 read in 10625 ms (705.0 tok/s), 8 generated in 109 ms (73.1 tok/s), drafts accepted 6 of 6, 1 checkpoints",
+  "strata serve: decode expert cache hit rate: 90.2% (3891 hits / 4313 lookups)"
+ ],
+ "answer_head": "C43C98BD",
+ "needle_found": true,
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_s",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 9376,
+   "expert_cache_mib": 17297,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 635,
+   "cvec": 0,
+   "arena_mib": 47962,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790718906.5660036,
+    "duration_s": 10.7,
+    "finish": "stop",
+    "prompt_tokens": 7491,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 10625.3,
+    "decode_ms": 109.4,
+    "decode_tok_s": 73.1,
+    "hit_rate": 0.902
+   },
+   {
+    "projection": null,
+    "time": 1790718828.4807332,
+    "duration_s": 42.6,
+    "finish": "stop",
+    "prompt_tokens": 31604,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 42110.8,
+    "decode_ms": 460.9,
+    "decode_tok_s": 106.3,
+    "hit_rate": 0.991
+   },
+   {
+    "projection": null,
+    "time": 1790718784.8375118,
+    "duration_s": 43.0,
+    "finish": "stop",
+    "prompt_tokens": 31633,
+    "reused": 0,
+    "output_tokens": 88,
+    "engine_generated": 88,
+    "prompt_ms": 42216.7,
+    "decode_ms": 776.7,
+    "decode_tok_s": 113.3,
+    "hit_rate": 0.978
+   },
+   {
+    "projection": null,
+    "time": 1790718743.8685048,
+    "duration_s": 40.3,
+    "finish": "stop",
+    "prompt_tokens": 28315,
+    "reused": 0,
+    "output_tokens": 84,
+    "engine_generated": 84,
+    "prompt_ms": 39286.5,
+    "decode_ms": 1011.1,
+    "decode_tok_s": 83.1,
+    "hit_rate": 0.949
+   }
+  ],
+  "requests_kept": 4,
+  "totals": {
+   "since": 1790718725.4039552,
+   "requests": 4,
+   "prompt_tokens": 99043,
+   "reused": 0,
+   "output_tokens": 229,
+   "prompt_ms": 134239.3,
+   "decode_ms": 2358.1000000000004
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 1,
+     "mem_used": 25212944384,
+     "mem_total": 25769803776,
+     "temp": 46,
+     "power": 122.865
+    },
+    {
+     "index": 1,
+     "util": 0,
+     "mem_used": 24227282944,
+     "mem_total": 25769803776,
+     "temp": 50,
+     "power": 213.727
+    }
+   ],
+   "gpu_util": 0.5,
+   "gpu_mem_used": 49440227328,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 50,
+   "gpu_power": 336.592,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.666015625,
+   "gpu_pcie_tx_mb": 0.9052734375,
+   "cpu": 19.7,
+   "ram_used": 70472826880,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.2018157842351215,
+   "disk_write_mb": 0.28470440990311785,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    25.0,
+    33.5,
+    47.0,
+    15.5,
+    32.5,
+    24.0,
+    32.5,
+    29.0,
+    55.0,
+    34.0,
+    48.0,
+    33.0,
+    22.5,
+    31.5,
+    24.5,
+    20.0,
+    20.5,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    37.0,
+    28.0,
+    22.5,
+    30.0,
+    37.0,
+    18.0,
+    29.0,
+    22.0,
+    21.0,
+    0.5
+   ],
+   "gpu_mem_used": [
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328
+   ],
+   "gpu_temp": [
+    51,
+    50,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    50,
+    52,
+    52,
+    52,
+    51,
+    52,
+    52,
+    50,
+    49,
+    49,
+    49,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    45,
+    45,
+    45,
+    45,
+    45,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    50,
+    49,
+    51,
+    51,
+    51,
+    51,
+    50,
+    51,
+    52,
+    50
+   ],
+   "gpu_power": [
+    380.19,
+    386.69,
+    409.76,
+    359.41,
+    400.68,
+    368.54,
+    396.07,
+    393.99,
+    371.52,
+    407.92,
+    367.65,
+    416.33,
+    386.19,
+    394.79,
+    409.75,
+    373.56,
+    329.84,
+    371.48,
+    309.69,
+    279.06,
+    254.32,
+    242.47,
+    242.53,
+    242.44,
+    242.38,
+    242.52,
+    242.49,
+    242.46,
+    242.59,
+    242.37,
+    242.36,
+    242.5,
+    242.54,
+    140.5,
+    84.6,
+    65.59,
+    65.03,
+    64.81,
+    64.81,
+    64.9,
+    64.51,
+    64.65,
+    64.87,
+    64.67,
+    64.8,
+    64.5,
+    64.56,
+    64.64,
+    64.52,
+    64.48,
+    252.37,
+    388.02,
+    363.78,
+    375.15,
+    399.66,
+    368.5,
+    396.75,
+    362.75,
+    365.62,
+    336.59
+   ],
+   "gpu_pcie_rx_mb": [
+    5368.61,
+    7806.78,
+    7675.93,
+    8469.58,
+    5669.88,
+    14723.06,
+    5790.85,
+    6188.01,
+    6980.23,
+    7332.23,
+    12898.87,
+    6754.21,
+    6340.74,
+    6000.61,
+    6420.09,
+    4455.66,
+    19.74,
+    0.62,
+    0.62,
+    0.57,
+    0.67,
+    0.62,
+    0.62,
+    0.62,
+    0.67,
+    0.67,
+    0.67,
+    0.62,
+    0.62,
+    0.57,
+    0.62,
+    0.67,
+    0.71,
+    0.81,
+    0.81,
+    0.86,
+    0.81,
+    0.86,
+    0.81,
+    0.86,
+    0.76,
+    0.81,
+    0.86,
+    0.81,
+    0.95,
+    0.81,
+    0.86,
+    0.86,
+    0.86,
+    0.71,
+    7078.07,
+    5438.42,
+    7891.61,
+    5618.71,
+    5824.99,
+    6656.5,
+    6553.84,
+    1295.8,
+    3929.76,
+    0.67
+   ],
+   "cpu": [
+    7.3,
+    17.5,
+    8.0,
+    7.9,
+    6.6,
+    6.2,
+    6.2,
+    6.1,
+    6.6,
+    5.7,
+    6.8,
+    7.1,
+    7.8,
+    6.2,
+    9.8,
+    15.0,
+    32.2,
+    20.2,
+    4.3,
+    11.3,
+    2.0,
+    1.5,
+    1.4,
+    3.7,
+    2.4,
+    2.6,
+    2.2,
+    1.6,
+    7.0,
+    7.8,
+    0.8,
+    3.8,
+    6.2,
+    1.4,
+    1.5,
+    1.3,
+    0.5,
+    0.8,
+    1.4,
+    1.7,
+    1.4,
+    1.5,
+    8.2,
+    9.2,
+    2.9,
+    5.7,
+    1.3,
+    5.6,
+    1.2,
+    7.3,
+    6.3,
+    9.1,
+    6.4,
+    7.4,
+    6.7,
+    6.4,
+    16.3,
+    9.8,
+    6.3,
+    19.7
+   ],
+   "ram_used": [
+    70159552512,
+    70270476288,
+    70270668800,
+    70271184896,
+    70263926784,
+    70264184832,
+    70264291328,
+    70265167872,
+    70267035648,
+    70267035648,
+    70264389632,
+    70261067776,
+    70265782272,
+    70268100608,
+    70275043328,
+    70091866112,
+    70312075264,
+    70311395328,
+    70465576960,
+    70613889024,
+    70610006016,
+    70609461248,
+    70616137728,
+    70634639360,
+    70632943616,
+    70619291648,
+    70622433280,
+    70624546816,
+    70651342848,
+    70754201600,
+    70754394112,
+    70772871168,
+    70811869184,
+    70806048768,
+    70788648960,
+    70775001088,
+    70770065408,
+    70761766912,
+    70762528768,
+    70758371328,
+    70767398912,
+    70768398336,
+    70723190784,
+    70684524544,
+    70713102336,
+    70694293504,
+    70656102400,
+    70685872128,
+    70679355392,
+    70251024384,
+    70250827776,
+    70240993280,
+    70217400320,
+    70216302592,
+    70220156928,
+    70211788800,
+    70218944512,
+    70240178176,
+    70232375296,
+    70472826880
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.04,
+    0.0,
+    0.01,
+    0.0,
+    0.02,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.22,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.09,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.04,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    1.46,
+    0.01,
+    0.0,
+    0.0,
+    0.82,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.2
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    97.75,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790718917.804988
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_S-2x3090-auto-two-turn/two-turn-16k-1.json

@@ -0,0 +1,797 @@
+{
+ "label": "native-v0.1.26-IQ3_S-2x3090-auto-two-turn",
+ "mode": "two-turn",
+ "ctx": "16k",
+ "target_tokens": 16384,
+ "run": 1,
+ "run_id": "2fd93144a29a",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:56:47",
+ "prompt_tokens": 15639,
+ "completion_tokens": 83,
+ "ttft_s": 21.242,
+ "e2e_s": 22.071,
+ "api_prompt_tps": 736.2,
+ "api_decode_tps": 98.8,
+ "engine_prompt_tps": 739.4,
+ "engine_decode_tps": 98.3,
+ "log_lines": [
+  "strata serve: prompt 15639 tokens = 0 reused + 15639 read in 21151 ms (739.4 tok/s), 83 generated in 844 ms (98.3 tok/s), drafts accepted 53 of 73, 1 checkpoints",
+  "strata serve: decode expert cache hit rate: 98.5% (48689 hits / 49415 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what's there.",
+ "turn2_ttft_s": 0.627,
+ "turn2_prompt_tokens": 15676,
+ "turn2_log_lines": [
+  "strata serve: prompt 15676 tokens = 15632 reused + 44 read in 521 ms (84.4 tok/s), 64 generated in 739 ms (86.6 tok/s), drafts accepted 37 of 52, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 97.1% (36768 hits / 37868 lookups)"
+ ],
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_s",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 9376,
+   "expert_cache_mib": 17297,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 635,
+   "cvec": 0,
+   "arena_mib": 47962,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790719030.332211,
+    "duration_s": 1.3,
+    "finish": "length",
+    "prompt_tokens": 15676,
+    "reused": 15632,
+    "output_tokens": 64,
+    "engine_generated": 64,
+    "prompt_ms": 521.0,
+    "decode_ms": 739.1,
+    "decode_tok_s": 86.6,
+    "hit_rate": 0.971
+   },
+   {
+    "projection": null,
+    "time": 1790719007.7458334,
+    "duration_s": 22.0,
+    "finish": "stop",
+    "prompt_tokens": 15639,
+    "reused": 0,
+    "output_tokens": 83,
+    "engine_generated": 83,
+    "prompt_ms": 21150.9,
+    "decode_ms": 844.3,
+    "decode_tok_s": 98.3,
+    "hit_rate": 0.985
+   },
+   {
+    "projection": null,
+    "time": 1790718941.8879943,
+    "duration_s": 39.9,
+    "finish": "stop",
+    "prompt_tokens": 29827,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 39812.1,
+    "decode_ms": 93.1,
+    "decode_tok_s": 85.9,
+    "hit_rate": 0.962
+   },
+   {
+    "projection": null,
+    "time": 1790718917.8830457,
+    "duration_s": 23.4,
+    "finish": "stop",
+    "prompt_tokens": 17345,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 23292.2,
+    "decode_ms": 78.7,
+    "decode_tok_s": 101.7,
+    "hit_rate": 0.963
+   },
+   {
+    "projection": null,
+    "time": 1790718906.5660036,
+    "duration_s": 10.7,
+    "finish": "stop",
+    "prompt_tokens": 7491,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 10625.3,
+    "decode_ms": 109.4,
+    "decode_tok_s": 73.1,
+    "hit_rate": 0.902
+   },
+   {
+    "projection": null,
+    "time": 1790718828.4807332,
+    "duration_s": 42.6,
+    "finish": "stop",
+    "prompt_tokens": 31604,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 42110.8,
+    "decode_ms": 460.9,
+    "decode_tok_s": 106.3,
+    "hit_rate": 0.991
+   },
+   {
+    "projection": null,
+    "time": 1790718784.8375118,
+    "duration_s": 43.0,
+    "finish": "stop",
+    "prompt_tokens": 31633,
+    "reused": 0,
+    "output_tokens": 88,
+    "engine_generated": 88,
+    "prompt_ms": 42216.7,
+    "decode_ms": 776.7,
+    "decode_tok_s": 113.3,
+    "hit_rate": 0.978
+   },
+   {
+    "projection": null,
+    "time": 1790718743.8685048,
+    "duration_s": 40.3,
+    "finish": "stop",
+    "prompt_tokens": 28315,
+    "reused": 0,
+    "output_tokens": 84,
+    "engine_generated": 84,
+    "prompt_ms": 39286.5,
+    "decode_ms": 1011.1,
+    "decode_tok_s": 83.1,
+    "hit_rate": 0.949
+   }
+  ],
+  "requests_kept": 8,
+  "totals": {
+   "since": 1790718725.4039552,
+   "requests": 8,
+   "prompt_tokens": 177530,
+   "reused": 15632,
+   "output_tokens": 392,
+   "prompt_ms": 219015.5,
+   "decode_ms": 4113.3
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 41,
+     "mem_used": 25212944384,
+     "mem_total": 25769803776,
+     "temp": 50,
+     "power": 209.466
+    },
+    {
+     "index": 1,
+     "util": 49,
+     "mem_used": 24227282944,
+     "mem_total": 25769803776,
+     "temp": 53,
+     "power": 186.867
+    }
+   ],
+   "gpu_util": 45.0,
+   "gpu_mem_used": 49440227328,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 53,
+   "gpu_power": 396.33299999999997,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 2090.7392578125,
+   "gpu_pcie_tx_mb": 103.1865234375,
+   "cpu": 82.0,
+   "ram_used": 70765735936,
+   "ram_total": 177175990272,
+   "disk_read_mb": 3.548661248401337,
+   "disk_write_mb": 0.003595401467478558,
+   "tok_s": 85.02069488627261,
+   "tok_s_mean": 85.60127593635436
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    57.5,
+    32.0,
+    26.0,
+    33.5,
+    25.0,
+    32.0,
+    32.5,
+    22.5,
+    36.0,
+    37.0,
+    36.5,
+    28.5,
+    42.5,
+    21.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    32.0,
+    30.5,
+    25.5,
+    63.5,
+    34.5,
+    24.5,
+    36.0,
+    38.0,
+    28.0,
+    33.0,
+    21.5,
+    25.0,
+    41.5,
+    33.0,
+    30.0,
+    22.0,
+    29.5,
+    19.5,
+    23.5,
+    48.0,
+    0.0,
+    45.0
+   ],
+   "gpu_mem_used": [
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328
+   ],
+   "gpu_temp": [
+    52,
+    51,
+    53,
+    52,
+    52,
+    52,
+    52,
+    53,
+    52,
+    53,
+    52,
+    53,
+    52,
+    53,
+    51,
+    50,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    47,
+    46,
+    46,
+    46,
+    46,
+    45,
+    45,
+    45,
+    51,
+    51,
+    51,
+    52,
+    51,
+    53,
+    52,
+    53,
+    52,
+    52,
+    53,
+    52,
+    53,
+    52,
+    52,
+    53,
+    53,
+    53,
+    53,
+    53,
+    51,
+    53
+   ],
+   "gpu_power": [
+    383.68,
+    395.97,
+    381.79,
+    371.58,
+    387.34,
+    367.36,
+    390.18,
+    377.99,
+    404.54,
+    406.77,
+    371.21,
+    404.91,
+    389.57,
+    358.58,
+    310.16,
+    278.39,
+    242.88,
+    242.69,
+    242.56,
+    242.41,
+    242.25,
+    242.45,
+    242.44,
+    242.38,
+    242.25,
+    242.18,
+    242.52,
+    242.57,
+    242.25,
+    242.23,
+    197.61,
+    110.58,
+    65.55,
+    65.12,
+    65.0,
+    65.02,
+    65.01,
+    64.82,
+    155.28,
+    410.36,
+    361.72,
+    382.57,
+    404.0,
+    377.89,
+    389.35,
+    407.89,
+    389.48,
+    386.49,
+    371.41,
+    379.85,
+    411.79,
+    387.4,
+    395.89,
+    369.99,
+    400.03,
+    390.77,
+    393.61,
+    354.25,
+    413.98,
+    396.33
+   ],
+   "gpu_pcie_rx_mb": [
+    5715.7,
+    6093.12,
+    89.88,
+    6215.57,
+    4331.35,
+    6045.63,
+    5971.19,
+    6178.04,
+    5846.59,
+    7484.53,
+    5744.65,
+    5095.67,
+    7809.07,
+    0.67,
+    0.71,
+    0.57,
+    0.67,
+    0.62,
+    0.62,
+    0.57,
+    0.71,
+    0.62,
+    0.57,
+    0.67,
+    0.62,
+    0.62,
+    0.57,
+    0.62,
+    0.62,
+    0.71,
+    0.91,
+    0.81,
+    0.76,
+    0.81,
+    0.76,
+    0.81,
+    0.86,
+    0.86,
+    6219.24,
+    5437.52,
+    6447.08,
+    6610.58,
+    6506.2,
+    88.93,
+    6106.14,
+    6218.53,
+    5297.85,
+    8021.88,
+    67.85,
+    4886.82,
+    6395.77,
+    5794.95,
+    5961.99,
+    92.79,
+    4561.42,
+    5886.93,
+    4115.39,
+    15.07,
+    6036.38,
+    2090.74
+   ],
+   "cpu": [
+    6.2,
+    6.0,
+    9.3,
+    6.5,
+    6.1,
+    6.0,
+    14.5,
+    11.7,
+    5.6,
+    8.6,
+    7.7,
+    6.1,
+    6.7,
+    6.3,
+    15.2,
+    4.0,
+    11.7,
+    1.7,
+    0.8,
+    1.6,
+    10.1,
+    8.8,
+    1.8,
+    3.0,
+    2.0,
+    1.1,
+    3.9,
+    2.5,
+    1.2,
+    1.0,
+    4.4,
+    2.2,
+    2.1,
+    1.1,
+    9.2,
+    4.0,
+    0.7,
+    12.8,
+    6.2,
+    7.1,
+    6.6,
+    6.1,
+    5.7,
+    6.8,
+    7.9,
+    5.9,
+    6.1,
+    7.1,
+    16.6,
+    14.8,
+    6.4,
+    6.7,
+    6.6,
+    7.0,
+    6.5,
+    7.1,
+    6.4,
+    44.5,
+    46.9,
+    82.0
+   ],
+   "ram_used": [
+    70371860480,
+    70366474240,
+    70373842944,
+    70374035456,
+    70365605888,
+    70363926528,
+    70340509696,
+    70341251072,
+    70340546560,
+    70343122944,
+    70346379264,
+    70347120640,
+    70345048064,
+    70479159296,
+    70572703744,
+    70652428288,
+    70736793600,
+    70729584640,
+    70733873152,
+    70729728000,
+    70753632256,
+    70894960640,
+    70882795520,
+    70887088128,
+    70877409280,
+    70896922624,
+    70917816320,
+    70944993280,
+    70949908480,
+    70948433920,
+    70969233408,
+    70968532992,
+    70973911040,
+    70965530624,
+    70981591040,
+    70982193152,
+    70979321856,
+    71007891456,
+    70533447680,
+    70536957952,
+    70444630016,
+    70438658048,
+    70435995648,
+    70428848128,
+    70422986752,
+    70424240128,
+    70404268032,
+    70411444224,
+    70518763520,
+    70541369344,
+    70539395072,
+    70538240000,
+    70528540672,
+    70514954240,
+    70316679168,
+    70304641024,
+    70310481920,
+    70539440128,
+    70546964480,
+    70765735936
+   ],
+   "disk_read_mb": [
+    0.05,
+    0.0,
+    0.0,
+    0.03,
+    0.0,
+    0.04,
+    0.0,
+    0.0,
+    0.18,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.15,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.48,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.02,
+    0.0,
+    0.01,
+    0.0,
+    0.03,
+    0.0,
+    0.0,
+    0.21,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.21,
+    0.47,
+    3.55
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    106.42,
+    0.0,
+    85.02
+   ]
+  },
+  "time": 1790719032.1000419
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_S-2x3090-auto/speed-32k-1.json

@@ -0,0 +1,648 @@
+{
+ "label": "native-v0.1.26-IQ3_S-2x3090-auto",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 1,
+ "run_id": "e13c9cdd6797",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:52:23",
+ "prompt_tokens": 28315,
+ "completion_tokens": 84,
+ "ttft_s": 39.45,
+ "e2e_s": 40.422,
+ "api_prompt_tps": 717.7,
+ "api_decode_tps": 85.4,
+ "engine_prompt_tps": 720.7,
+ "engine_decode_tps": 83.1,
+ "log_lines": [
+  "strata serve: prompt 28315 tokens = 0 reused + 28315 read in 39287 ms (720.7 tok/s), 84 generated in 1011 ms (83.1 tok/s), drafts accepted 56 of 67, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 94.9% (43073 hits / 45377 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand the context and find the relevant files.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_s",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 9376,
+   "expert_cache_mib": 17297,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 635,
+   "cvec": 0,
+   "arena_mib": 47962,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790718743.8685048,
+    "duration_s": 40.3,
+    "finish": "stop",
+    "prompt_tokens": 28315,
+    "reused": 0,
+    "output_tokens": 84,
+    "engine_generated": 84,
+    "prompt_ms": 39286.5,
+    "decode_ms": 1011.1,
+    "decode_tok_s": 83.1,
+    "hit_rate": 0.949
+   }
+  ],
+  "requests_kept": 1,
+  "totals": {
+   "since": 1790718725.4039552,
+   "requests": 1,
+   "prompt_tokens": 28315,
+   "reused": 0,
+   "output_tokens": 84,
+   "prompt_ms": 39286.5,
+   "decode_ms": 1011.1
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 39,
+     "mem_used": 25212944384,
+     "mem_total": 25769803776,
+     "temp": 45,
+     "power": 141.739
+    },
+    {
+     "index": 1,
+     "util": 48,
+     "mem_used": 24227282944,
+     "mem_total": 25769803776,
+     "temp": 50,
+     "power": 205.268
+    }
+   ],
+   "gpu_util": 43.5,
+   "gpu_mem_used": 49440227328,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 50,
+   "gpu_power": 347.007,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 6170.3671875,
+   "gpu_pcie_tx_mb": 126.7900390625,
+   "cpu": 85.9,
+   "ram_used": 70739390464,
+   "ram_total": 177175990272,
+   "disk_read_mb": 3.315986845192439,
+   "disk_write_mb": 0.09340808014626588,
+   "tok_s": 93.51552437654362,
+   "tok_s_mean": 85.24938371584437
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    1.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    1.0,
+    57.0,
+    32.5,
+    23.5,
+    25.0,
+    22.0,
+    51.0,
+    35.5,
+    33.0,
+    27.5,
+    26.0,
+    41.0,
+    33.0,
+    23.5,
+    29.5,
+    27.5,
+    71.0,
+    31.5,
+    21.5,
+    64.0,
+    36.5,
+    20.5,
+    32.5,
+    30.5,
+    22.0,
+    29.0,
+    24.5,
+    30.0,
+    17.5,
+    76.0,
+    25.0,
+    20.5,
+    30.0,
+    23.0,
+    19.0,
+    24.0,
+    43.5
+   ],
+   "gpu_mem_used": [
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49224220672,
+    49316495360,
+    49326981120,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49440227328
+   ],
+   "gpu_temp": [
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    42,
+    44,
+    47,
+    46,
+    49,
+    47,
+    49,
+    49,
+    49,
+    49,
+    48,
+    49,
+    48,
+    48,
+    49,
+    48,
+    48,
+    50,
+    48,
+    48,
+    48,
+    48,
+    49,
+    48,
+    48,
+    49,
+    48,
+    50,
+    49,
+    49,
+    51,
+    48,
+    50,
+    48,
+    49,
+    50,
+    50,
+    50
+   ],
+   "gpu_power": [
+    239.92,
+    240.19,
+    239.81,
+    239.35,
+    239.3,
+    239.28,
+    239.24,
+    239.29,
+    239.41,
+    239.46,
+    239.45,
+    239.36,
+    239.21,
+    239.26,
+    239.32,
+    239.37,
+    225.06,
+    145.42,
+    158.12,
+    371.86,
+    384.05,
+    358.93,
+    405.61,
+    381.25,
+    384.16,
+    389.93,
+    395.3,
+    399.41,
+    384.35,
+    363.41,
+    386.79,
+    392.7,
+    344.05,
+    389.76,
+    383.66,
+    397.8,
+    373.05,
+    366.9,
+    391.72,
+    366.3,
+    381.76,
+    387.41,
+    366.67,
+    395.88,
+    403.84,
+    386.94,
+    392.83,
+    352.42,
+    402.48,
+    392.8,
+    357.94,
+    383.58,
+    359.24,
+    350.73,
+    347.01
+   ],
+   "gpu_pcie_rx_mb": [
+    0.62,
+    0.67,
+    0.67,
+    0.62,
+    0.62,
+    0.57,
+    0.57,
+    0.57,
+    0.62,
+    0.67,
+    0.67,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.62,
+    0.76,
+    14.92,
+    6198.41,
+    7236.0,
+    8724.26,
+    3654.86,
+    5959.32,
+    6449.94,
+    6329.34,
+    7285.26,
+    4763.84,
+    6812.52,
+    7321.69,
+    6494.14,
+    5619.86,
+    6789.97,
+    5793.67,
+    5562.64,
+    2331.69,
+    6177.52,
+    8354.0,
+    7435.75,
+    4811.38,
+    1.43,
+    7506.18,
+    7562.16,
+    2886.44,
+    7184.17,
+    6752.54,
+    7546.95,
+    7499.69,
+    4610.3,
+    6852.1,
+    6167.22,
+    5107.31,
+    7326.55,
+    4237.65,
+    0.67,
+    6170.37
+   ],
+   "cpu": [
+    100.0,
+    1.4,
+    1.1,
+    0.9,
+    1.2,
+    2.3,
+    0.9,
+    1.3,
+    9.5,
+    5.4,
+    6.6,
+    4.7,
+    1.3,
+    1.2,
+    1.8,
+    2.0,
+    0.8,
+    11.3,
+    10.6,
+    10.4,
+    5.7,
+    6.8,
+    17.0,
+    10.2,
+    8.1,
+    7.1,
+    6.9,
+    7.6,
+    9.8,
+    9.9,
+    5.8,
+    6.7,
+    11.0,
+    6.0,
+    9.4,
+    6.2,
+    15.2,
+    12.9,
+    8.4,
+    7.2,
+    10.2,
+    6.1,
+    10.5,
+    6.4,
+    6.0,
+    9.0,
+    7.4,
+    8.9,
+    6.9,
+    8.7,
+    15.6,
+    12.0,
+    6.3,
+    6.8,
+    85.9
+   ],
+   "ram_used": [
+    70239571968,
+    70088245248,
+    70096302080,
+    70097154048,
+    70103457792,
+    70078648320,
+    70084857856,
+    70088122368,
+    70148214784,
+    70162341888,
+    70191161344,
+    70190673920,
+    70184566784,
+    70184079360,
+    70179979264,
+    70173462528,
+    70171906048,
+    70119723008,
+    70266146816,
+    70263320576,
+    70263308288,
+    70262693888,
+    70372823040,
+    70378594304,
+    70380400640,
+    70383910912,
+    70389587968,
+    70366543872,
+    70385344512,
+    70382903296,
+    70379986944,
+    70372298752,
+    70358667264,
+    70367678464,
+    70356070400,
+    70353756160,
+    70321405952,
+    70284603392,
+    70262280192,
+    70387703808,
+    70314864640,
+    70312120320,
+    70299967488,
+    70299987968,
+    70294355968,
+    70301368320,
+    70293327872,
+    70290792448,
+    70280925184,
+    70300200960,
+    70377836544,
+    70448394240,
+    70448910336,
+    70574612480,
+    70739390464
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    4.75,
+    124.76,
+    40.78,
+    0.0,
+    7.5,
+    16.84,
+    0.0,
+    55.72,
+    0.0,
+    13.03,
+    20.5,
+    0.0,
+    67.32,
+    0.0,
+    0.0,
+    83.74,
+    0.0,
+    80.89,
+    0.0,
+    0.0,
+    71.3,
+    0.0,
+    0.0,
+    79.43,
+    0.0,
+    72.4,
+    4.53,
+    0.0,
+    54.62,
+    0.0,
+    42.73,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    3.32
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    93.52
+   ]
+  },
+  "time": 1790718784.6773934
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_S-2x3090-auto/speed-32k-2.json

@@ -0,0 +1,707 @@
+{
+ "label": "native-v0.1.26-IQ3_S-2x3090-auto",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 2,
+ "run_id": "f953f67441b4",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:53:04",
+ "prompt_tokens": 31633,
+ "completion_tokens": 88,
+ "ttft_s": 42.403,
+ "e2e_s": 43.165,
+ "api_prompt_tps": 746.0,
+ "api_decode_tps": 114.2,
+ "engine_prompt_tps": 749.3,
+ "engine_decode_tps": 113.3,
+ "log_lines": [
+  "strata serve: prompt 31633 tokens = 0 reused + 31633 read in 42217 ms (749.3 tok/s), 88 generated in 777 ms (113.3 tok/s), drafts accepted 61 of 69, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 97.8% (45031 hits / 46048 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what's there and identify the \"last file\" being referenced.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_s",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 9376,
+   "expert_cache_mib": 17297,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 635,
+   "cvec": 0,
+   "arena_mib": 47962,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790718784.8375118,
+    "duration_s": 43.0,
+    "finish": "stop",
+    "prompt_tokens": 31633,
+    "reused": 0,
+    "output_tokens": 88,
+    "engine_generated": 88,
+    "prompt_ms": 42216.7,
+    "decode_ms": 776.7,
+    "decode_tok_s": 113.3,
+    "hit_rate": 0.978
+   },
+   {
+    "projection": null,
+    "time": 1790718743.8685048,
+    "duration_s": 40.3,
+    "finish": "stop",
+    "prompt_tokens": 28315,
+    "reused": 0,
+    "output_tokens": 84,
+    "engine_generated": 84,
+    "prompt_ms": 39286.5,
+    "decode_ms": 1011.1,
+    "decode_tok_s": 83.1,
+    "hit_rate": 0.949
+   }
+  ],
+  "requests_kept": 2,
+  "totals": {
+   "since": 1790718725.4039552,
+   "requests": 2,
+   "prompt_tokens": 59948,
+   "reused": 0,
+   "output_tokens": 172,
+   "prompt_ms": 81503.2,
+   "decode_ms": 1787.8000000000002
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 43,
+     "mem_used": 25212944384,
+     "mem_total": 25769803776,
+     "temp": 46,
+     "power": 127.404
+    },
+    {
+     "index": 1,
+     "util": 22,
+     "mem_used": 24227282944,
+     "mem_total": 25769803776,
+     "temp": 50,
+     "power": 202.736
+    }
+   ],
+   "gpu_util": 32.5,
+   "gpu_mem_used": 49440227328,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 50,
+   "gpu_power": 330.14,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 50.97265625,
+   "gpu_pcie_tx_mb": 74.43359375,
+   "cpu": 47.4,
+   "ram_used": 70762872832,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.9085358249169909,
+   "disk_write_mb": 0.21546304148229034,
+   "tok_s": 110.51273259801776,
+   "tok_s_mean": 104.51731108020398
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    31.5,
+    21.5,
+    64.0,
+    36.5,
+    20.5,
+    32.5,
+    30.5,
+    22.0,
+    29.0,
+    24.5,
+    30.0,
+    17.5,
+    76.0,
+    25.0,
+    20.5,
+    30.0,
+    23.0,
+    19.0,
+    24.0,
+    43.5,
+    0.0,
+    41.5,
+    33.5,
+    27.5,
+    34.0,
+    22.0,
+    61.5,
+    37.0,
+    23.5,
+    24.5,
+    28.5,
+    54.0,
+    41.5,
+    89.5,
+    30.5,
+    21.0,
+    32.0,
+    33.5,
+    24.5,
+    26.0,
+    25.5,
+    38.5,
+    31.5,
+    24.5,
+    30.0,
+    26.0,
+    31.5,
+    19.0,
+    86.5,
+    28.0,
+    22.5,
+    32.0,
+    26.0,
+    77.5,
+    22.5,
+    22.5,
+    34.0,
+    26.0,
+    20.5,
+    32.5
+   ],
+   "gpu_mem_used": [
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49333272576,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328
+   ],
+   "gpu_temp": [
+    48,
+    48,
+    48,
+    48,
+    49,
+    48,
+    48,
+    49,
+    48,
+    50,
+    49,
+    49,
+    51,
+    48,
+    50,
+    48,
+    49,
+    50,
+    50,
+    50,
+    47,
+    48,
+    48,
+    50,
+    49,
+    50,
+    49,
+    49,
+    50,
+    49,
+    50,
+    49,
+    50,
+    51,
+    49,
+    50,
+    50,
+    49,
+    50,
+    50,
+    51,
+    50,
+    49,
+    50,
+    49,
+    51,
+    49,
+    50,
+    51,
+    49,
+    50,
+    50,
+    51,
+    52,
+    50,
+    50,
+    50,
+    51,
+    52,
+    50
+   ],
+   "gpu_power": [
+    397.8,
+    373.05,
+    366.9,
+    391.72,
+    366.3,
+    381.76,
+    387.41,
+    366.67,
+    395.88,
+    403.84,
+    386.94,
+    392.83,
+    352.42,
+    402.48,
+    392.8,
+    357.94,
+    383.58,
+    359.24,
+    350.73,
+    347.01,
+    325.66,
+    372.55,
+    370.97,
+    352.05,
+    397.27,
+    391.66,
+    343.17,
+    403.13,
+    366.16,
+    402.14,
+    390.68,
+    396.82,
+    401.28,
+    360.7,
+    398.3,
+    366.48,
+    382.49,
+    395.03,
+    366.16,
+    401.29,
+    385.57,
+    368.88,
+    396.03,
+    374.4,
+    393.23,
+    385.24,
+    406.68,
+    391.86,
+    364.28,
+    409.58,
+    388.38,
+    389.1,
+    390.65,
+    372.81,
+    415.25,
+    390.44,
+    384.85,
+    382.05,
+    377.98,
+    330.14
+   ],
+   "gpu_pcie_rx_mb": [
+    6177.52,
+    8354.0,
+    7435.75,
+    4811.38,
+    1.43,
+    7506.18,
+    7562.16,
+    2886.44,
+    7184.17,
+    6752.54,
+    7546.95,
+    7499.69,
+    4610.3,
+    6852.1,
+    6167.22,
+    5107.31,
+    7326.55,
+    4237.65,
+    0.67,
+    6170.37,
+    5107.54,
+    5192.28,
+    6336.26,
+    3422.45,
+    4911.8,
+    7759.14,
+    6146.0,
+    4671.1,
+    9668.92,
+    5712.17,
+    6061.89,
+    4888.06,
+    5886.08,
+    9090.42,
+    5683.33,
+    6294.68,
+    7625.39,
+    4995.82,
+    12979.6,
+    4392.43,
+    6062.41,
+    5581.95,
+    5691.53,
+    8234.83,
+    8344.89,
+    7880.73,
+    6094.07,
+    8294.58,
+    5858.85,
+    5465.17,
+    6922.82,
+    4432.29,
+    6296.59,
+    5837.44,
+    7103.97,
+    7604.84,
+    4756.35,
+    4899.02,
+    5517.96,
+    50.97
+   ],
+   "cpu": [
+    6.2,
+    15.2,
+    12.9,
+    8.4,
+    7.2,
+    10.2,
+    6.1,
+    10.5,
+    6.4,
+    6.0,
+    9.0,
+    7.4,
+    8.9,
+    6.9,
+    8.7,
+    15.6,
+    12.0,
+    6.3,
+    6.8,
+    85.9,
+    9.8,
+    9.7,
+    6.7,
+    6.6,
+    6.8,
+    7.4,
+    7.1,
+    7.1,
+    12.2,
+    13.8,
+    6.7,
+    8.5,
+    6.5,
+    6.1,
+    6.4,
+    6.5,
+    6.1,
+    7.2,
+    6.6,
+    7.4,
+    6.1,
+    6.8,
+    12.8,
+    11.1,
+    6.2,
+    6.4,
+    6.4,
+    6.1,
+    6.3,
+    8.7,
+    7.4,
+    9.4,
+    6.3,
+    7.9,
+    6.5,
+    6.4,
+    15.4,
+    9.4,
+    6.3,
+    47.4
+   ],
+   "ram_used": [
+    70353756160,
+    70321405952,
+    70284603392,
+    70262280192,
+    70387703808,
+    70314864640,
+    70312120320,
+    70299967488,
+    70299987968,
+    70294355968,
+    70301368320,
+    70293327872,
+    70290792448,
+    70280925184,
+    70300200960,
+    70377836544,
+    70448394240,
+    70448910336,
+    70574612480,
+    70739390464,
+    70512533504,
+    70533234688,
+    70513684480,
+    70496215040,
+    70484873216,
+    70469386240,
+    70452416512,
+    70445555712,
+    70424002560,
+    70434533376,
+    70424702976,
+    70408409088,
+    70395523072,
+    70388035584,
+    70353256448,
+    70333558784,
+    70326820864,
+    70357315584,
+    70361923584,
+    70355750912,
+    70357520384,
+    70490038272,
+    70530985984,
+    70524338176,
+    70524243968,
+    70525259776,
+    70528827392,
+    70526148608,
+    70526631936,
+    70541975552,
+    70541778944,
+    70536822784,
+    70537621504,
+    70537134080,
+    70534877184,
+    70532685824,
+    70529556480,
+    70530998272,
+    70534545408,
+    70762872832
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    71.3,
+    0.0,
+    0.0,
+    79.43,
+    0.0,
+    72.4,
+    4.53,
+    0.0,
+    54.62,
+    0.0,
+    42.73,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    3.32,
+    0.6,
+    0.01,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.02,
+    0.0,
+    0.02,
+    0.0,
+    0.0,
+    0.03,
+    0.0,
+    0.03,
+    0.0,
+    0.0,
+    0.73,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.02,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    12.58,
+    0.0,
+    0.0,
+    71.88,
+    0.0,
+    33.04,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.91
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    93.52,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    110.51
+   ]
+  },
+  "time": 1790718828.3451
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_S-2x3090-auto/speed-32k-3.json

@@ -0,0 +1,721 @@
+{
+ "label": "native-v0.1.26-IQ3_S-2x3090-auto",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 3,
+ "run_id": "4e71f26e7fd8",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:53:48",
+ "prompt_tokens": 31604,
+ "completion_tokens": 49,
+ "ttft_s": 42.269,
+ "e2e_s": 42.716,
+ "api_prompt_tps": 747.7,
+ "api_decode_tps": 107.5,
+ "engine_prompt_tps": 750.5,
+ "engine_decode_tps": 106.3,
+ "log_lines": [
+  "strata serve: prompt 31604 tokens = 0 reused + 31604 read in 42111 ms (750.5 tok/s), 49 generated in 461 ms (106.3 tok/s), drafts accepted 33 of 37, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 99.1% (26163 hits / 26396 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand what's there and identify the \"last file\" being referenced.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_s",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 9376,
+   "expert_cache_mib": 17297,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 635,
+   "cvec": 0,
+   "arena_mib": 47962,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790718828.4807332,
+    "duration_s": 42.6,
+    "finish": "stop",
+    "prompt_tokens": 31604,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 42110.8,
+    "decode_ms": 460.9,
+    "decode_tok_s": 106.3,
+    "hit_rate": 0.991
+   },
+   {
+    "projection": null,
+    "time": 1790718784.8375118,
+    "duration_s": 43.0,
+    "finish": "stop",
+    "prompt_tokens": 31633,
+    "reused": 0,
+    "output_tokens": 88,
+    "engine_generated": 88,
+    "prompt_ms": 42216.7,
+    "decode_ms": 776.7,
+    "decode_tok_s": 113.3,
+    "hit_rate": 0.978
+   },
+   {
+    "projection": null,
+    "time": 1790718743.8685048,
+    "duration_s": 40.3,
+    "finish": "stop",
+    "prompt_tokens": 28315,
+    "reused": 0,
+    "output_tokens": 84,
+    "engine_generated": 84,
+    "prompt_ms": 39286.5,
+    "decode_ms": 1011.1,
+    "decode_tok_s": 83.1,
+    "hit_rate": 0.949
+   }
+  ],
+  "requests_kept": 3,
+  "totals": {
+   "since": 1790718725.4039552,
+   "requests": 3,
+   "prompt_tokens": 91552,
+   "reused": 0,
+   "output_tokens": 221,
+   "prompt_ms": 123614.0,
+   "decode_ms": 2248.7000000000003
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 39,
+     "mem_used": 25212944384,
+     "mem_total": 25769803776,
+     "temp": 47,
+     "power": 127.526
+    },
+    {
+     "index": 1,
+     "util": 2,
+     "mem_used": 24227282944,
+     "mem_total": 25769803776,
+     "temp": 50,
+     "power": 202.315
+    }
+   ],
+   "gpu_util": 20.5,
+   "gpu_mem_used": 49440227328,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 50,
+   "gpu_power": 329.841,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 19.740234375,
+   "gpu_pcie_tx_mb": 54.21484375,
+   "cpu": 32.2,
+   "ram_used": 70312075264,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.0,
+   "disk_write_mb": 0.09328657441898458,
+   "tok_s": 97.74566314411967,
+   "tok_s_mean": 97.74268443960047
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    25.5,
+    38.5,
+    31.5,
+    24.5,
+    30.0,
+    26.0,
+    31.5,
+    19.0,
+    86.5,
+    28.0,
+    22.5,
+    32.0,
+    26.0,
+    77.5,
+    22.5,
+    22.5,
+    34.0,
+    26.0,
+    20.5,
+    32.5,
+    0.0,
+    36.5,
+    37.0,
+    26.0,
+    32.0,
+    37.0,
+    24.0,
+    32.5,
+    20.5,
+    29.0,
+    25.0,
+    27.5,
+    30.5,
+    24.0,
+    35.0,
+    28.5,
+    29.0,
+    30.0,
+    29.5,
+    33.5,
+    44.5,
+    55.5,
+    36.0,
+    25.0,
+    33.5,
+    47.0,
+    15.5,
+    32.5,
+    24.0,
+    32.5,
+    29.0,
+    55.0,
+    34.0,
+    48.0,
+    33.0,
+    22.5,
+    31.5,
+    24.5,
+    20.0,
+    20.5
+   ],
+   "gpu_mem_used": [
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328,
+    49440227328
+   ],
+   "gpu_temp": [
+    51,
+    50,
+    49,
+    50,
+    49,
+    51,
+    49,
+    50,
+    51,
+    49,
+    50,
+    50,
+    51,
+    52,
+    50,
+    50,
+    50,
+    51,
+    52,
+    50,
+    49,
+    49,
+    49,
+    51,
+    50,
+    50,
+    50,
+    50,
+    51,
+    50,
+    50,
+    51,
+    50,
+    51,
+    50,
+    51,
+    51,
+    50,
+    52,
+    50,
+    51,
+    50,
+    50,
+    51,
+    50,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    50,
+    52,
+    52,
+    52,
+    51,
+    52,
+    52,
+    50
+   ],
+   "gpu_power": [
+    385.57,
+    368.88,
+    396.03,
+    374.4,
+    393.23,
+    385.24,
+    406.68,
+    391.86,
+    364.28,
+    409.58,
+    388.38,
+    389.1,
+    390.65,
+    372.81,
+    415.25,
+    390.44,
+    384.85,
+    382.05,
+    377.98,
+    330.14,
+    415.76,
+    348.22,
+    407.06,
+    354.03,
+    392.05,
+    392.72,
+    370.96,
+    399.97,
+    402.84,
+    387.68,
+    394.33,
+    359.55,
+    419.05,
+    403.04,
+    384.68,
+    390.64,
+    393.93,
+    389.6,
+    362.93,
+    373.48,
+    405.22,
+    378.6,
+    399.07,
+    380.19,
+    386.69,
+    409.76,
+    359.41,
+    400.68,
+    368.54,
+    396.07,
+    393.99,
+    371.52,
+    407.92,
+    367.65,
+    416.33,
+    386.19,
+    394.79,
+    409.75,
+    373.56,
+    329.84
+   ],
+   "gpu_pcie_rx_mb": [
+    6062.41,
+    5581.95,
+    5691.53,
+    8234.83,
+    8344.89,
+    7880.73,
+    6094.07,
+    8294.58,
+    5858.85,
+    5465.17,
+    6922.82,
+    4432.29,
+    6296.59,
+    5837.44,
+    7103.97,
+    7604.84,
+    4756.35,
+    4899.02,
+    5517.96,
+    50.97,
+    0.67,
+    11557.34,
+    5125.33,
+    6536.1,
+    6042.86,
+    7341.43,
+    5299.47,
+    6137.51,
+    6431.72,
+    5859.47,
+    4887.1,
+    5919.46,
+    5763.62,
+    94.17,
+    6438.59,
+    7535.84,
+    6171.89,
+    7355.36,
+    4306.32,
+    5857.37,
+    4681.59,
+    5568.03,
+    5823.33,
+    5368.61,
+    7806.78,
+    7675.93,
+    8469.58,
+    5669.88,
+    14723.06,
+    5790.85,
+    6188.01,
+    6980.23,
+    7332.23,
+    12898.87,
+    6754.21,
+    6340.74,
+    6000.61,
+    6420.09,
+    4455.66,
+    19.74
+   ],
+   "cpu": [
+    6.1,
+    6.8,
+    12.8,
+    11.1,
+    6.2,
+    6.4,
+    6.4,
+    6.1,
+    6.3,
+    8.7,
+    7.4,
+    9.4,
+    6.3,
+    7.9,
+    6.5,
+    6.4,
+    15.4,
+    9.4,
+    6.3,
+    47.4,
+    36.1,
+    6.3,
+    6.7,
+    6.6,
+    5.7,
+    7.4,
+    6.2,
+    7.4,
+    6.0,
+    6.6,
+    15.5,
+    14.6,
+    5.8,
+    6.4,
+    6.5,
+    6.2,
+    11.6,
+    8.8,
+    7.8,
+    6.2,
+    6.9,
+    6.0,
+    5.5,
+    7.3,
+    17.5,
+    8.0,
+    7.9,
+    6.6,
+    6.2,
+    6.2,
+    6.1,
+    6.6,
+    5.7,
+    6.8,
+    7.1,
+    7.8,
+    6.2,
+    9.8,
+    15.0,
+    32.2
+   ],
+   "ram_used": [
+    70357520384,
+    70490038272,
+    70530985984,
+    70524338176,
+    70524243968,
+    70525259776,
+    70528827392,
+    70526148608,
+    70526631936,
+    70541975552,
+    70541778944,
+    70536822784,
+    70537621504,
+    70537134080,
+    70534877184,
+    70532685824,
+    70529556480,
+    70530998272,
+    70534545408,
+    70762872832,
+    70460293120,
+    70452703232,
+    70444961792,
+    70428729344,
+    70420193280,
+    70418821120,
+    70406471680,
+    70396342272,
+    70389112832,
+    70380965888,
+    70312722432,
+    70348423168,
+    70338981888,
+    70328791040,
+    70326034432,
+    70312820736,
+    70085947392,
+    69987848192,
+    70072119296,
+    70060589056,
+    70056861696,
+    70179471360,
+    70176550912,
+    70159552512,
+    70270476288,
+    70270668800,
+    70271184896,
+    70263926784,
+    70264184832,
+    70264291328,
+    70265167872,
+    70267035648,
+    70267035648,
+    70264389632,
+    70261067776,
+    70265782272,
+    70268100608,
+    70275043328,
+    70091866112,
+    70312075264
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.01,
+    0.0,
+    0.02,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    12.58,
+    0.0,
+    0.0,
+    71.88,
+    0.0,
+    33.04,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.91,
+    0.83,
+    0.04,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.03,
+    0.0,
+    0.0,
+    0.03,
+    0.0,
+    0.05,
+    0.0,
+    0.0,
+    0.04,
+    0.0,
+    0.04,
+    0.0,
+    0.04,
+    0.0,
+    0.0,
+    0.04,
+    0.0,
+    0.0,
+    0.04,
+    0.0,
+    0.01,
+    0.0,
+    0.02,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.22,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    110.51,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    97.75
+   ]
+  },
+  "time": 1790718871.5639336
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090-needle/needle-16k-1.json

@@ -0,0 +1,830 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090-needle",
+ "mode": "needle",
+ "ctx": "16k",
+ "target_tokens": 16384,
+ "run": 1,
+ "run_id": "e39790e5649d",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:47:57",
+ "prompt_tokens": 17346,
+ "completion_tokens": 8,
+ "ttft_s": 9.075,
+ "e2e_s": 9.178,
+ "api_prompt_tps": 1911.3,
+ "api_decode_tps": 68.2,
+ "engine_prompt_tps": 1933.6,
+ "engine_decode_tps": 67.4,
+ "log_lines": [
+  "strata serve: prompt 17346 tokens = 0 reused + 17346 read in 8971 ms (1933.6 tok/s), 8 generated in 119 ms (67.4 tok/s), drafts accepted 6 of 6, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 80.6% (3014 hits / 3740 lookups)"
+ ],
+ "answer_head": "37E3DB14",
+ "needle_found": true,
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714877.4157934,
+    "duration_s": 9.1,
+    "finish": "stop",
+    "prompt_tokens": 17346,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 8971.0,
+    "decode_ms": 118.7,
+    "decode_tok_s": 67.4,
+    "hit_rate": 0.806
+   },
+   {
+    "projection": null,
+    "time": 1790714872.57582,
+    "duration_s": 4.3,
+    "finish": "stop",
+    "prompt_tokens": 7493,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 4124.7,
+    "decode_ms": 123.2,
+    "decode_tok_s": 64.9,
+    "hit_rate": 0.748
+   },
+   {
+    "projection": null,
+    "time": 1790714759.725435,
+    "duration_s": 63.5,
+    "finish": "length",
+    "prompt_tokens": 129913,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 59900.8,
+    "decode_ms": 3556.3,
+    "decode_tok_s": 72.0,
+    "hit_rate": 0.956
+   },
+   {
+    "projection": null,
+    "time": 1790714695.4286683,
+    "duration_s": 63.2,
+    "finish": "length",
+    "prompt_tokens": 130360,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 59973.9,
+    "decode_ms": 3226.1,
+    "decode_tok_s": 79.4,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714636.537164,
+    "duration_s": 57.8,
+    "finish": "length",
+    "prompt_tokens": 119256,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 54577.1,
+    "decode_ms": 3229.8,
+    "decode_tok_s": 79.3,
+    "hit_rate": 0.955
+   },
+   {
+    "projection": null,
+    "time": 1790714507.7469585,
+    "duration_s": 17.4,
+    "finish": "length",
+    "prompt_tokens": 31602,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14617.7,
+    "decode_ms": 2785.2,
+    "decode_tok_s": 91.9,
+    "hit_rate": 0.965
+   },
+   {
+    "projection": null,
+    "time": 1790714489.1532733,
+    "duration_s": 17.9,
+    "finish": "length",
+    "prompt_tokens": 31582,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14618.7,
+    "decode_ms": 3270.7,
+    "decode_tok_s": 78.3,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714469.1007972,
+    "duration_s": 19.4,
+    "finish": "length",
+    "prompt_tokens": 34932,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 16553.9,
+    "decode_ms": 2826.7,
+    "decode_tok_s": 90.6,
+    "hit_rate": 0.941
+   },
+   {
+    "projection": null,
+    "time": 1790714463.1913157,
+    "duration_s": 5.2,
+    "finish": "length",
+    "prompt_tokens": 4201,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2365.6,
+    "decode_ms": 2868.4,
+    "decode_tok_s": 89.2,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714458.6299415,
+    "duration_s": 4.0,
+    "finish": "stop",
+    "prompt_tokens": 4200,
+    "reused": 0,
+    "output_tokens": 142,
+    "engine_generated": 142,
+    "prompt_ms": 2370.0,
+    "decode_ms": 1660.0,
+    "decode_tok_s": 85.5,
+    "hit_rate": 0.896
+   },
+   {
+    "projection": null,
+    "time": 1790714453.0791314,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 3881,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2223.8,
+    "decode_ms": 2790.5,
+    "decode_tok_s": 91.7,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790714448.5537117,
+    "duration_s": 4.0,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1305.1,
+    "decode_ms": 2690.8,
+    "decode_tok_s": 95.1,
+    "hit_rate": 0.972
+   }
+  ],
+  "requests_kept": 24,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 24,
+   "prompt_tokens": 631825,
+   "reused": 0,
+   "output_tokens": 4662,
+   "prompt_ms": 302982.9,
+   "decode_ms": 55029.19999999998
+  },
+  "hardware": {
+   "gpu_util": 100,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 49,
+   "gpu_power": 265.798,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 22367.23828125,
+   "gpu_pcie_tx_mb": 80.8232421875,
+   "cpu": 6.2,
+   "ram_used": 63624777728,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.0,
+   "disk_write_mb": 0.31474488451743216,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    99,
+    100,
+    99,
+    100,
+    0,
+    100,
+    99,
+    99,
+    100,
+    100,
+    99,
+    90,
+    100
+   ],
+   "gpu_mem_used": [
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    46,
+    45,
+    45,
+    44,
+    45,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    42,
+    42,
+    42,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    47,
+    48,
+    49,
+    49,
+    46,
+    49,
+    49,
+    48,
+    49,
+    49,
+    49,
+    48,
+    49
+   ],
+   "gpu_power": [
+    162.82,
+    148.08,
+    135.02,
+    121.86,
+    121.63,
+    121.78,
+    121.51,
+    121.24,
+    121.16,
+    121.27,
+    121.24,
+    121.23,
+    121.05,
+    121.06,
+    121.26,
+    121.23,
+    101.81,
+    59.35,
+    35.84,
+    35.18,
+    35.01,
+    35.2,
+    35.0,
+    34.94,
+    35.11,
+    35.02,
+    35.03,
+    35.11,
+    34.94,
+    34.9,
+    35.04,
+    35.01,
+    35.02,
+    35.13,
+    34.98,
+    34.93,
+    35.03,
+    34.9,
+    34.91,
+    35.06,
+    34.94,
+    34.94,
+    35.11,
+    35.01,
+    35.09,
+    34.99,
+    34.98,
+    69.69,
+    278.48,
+    278.38,
+    276.27,
+    201.82,
+    237.39,
+    277.05,
+    276.41,
+    278.18,
+    277.05,
+    276.73,
+    274.45,
+    265.8
+   ],
+   "gpu_pcie_rx_mb": [
+    0.33,
+    0.33,
+    0.33,
+    0.29,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.38,
+    0.33,
+    0.29,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.48,
+    0.52,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.57,
+    0.43,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.48,
+    0.43,
+    0.43,
+    0.67,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    12932.54,
+    9.63,
+    46.83,
+    24945.45,
+    2343.61,
+    13238.91,
+    26700.59,
+    116.78,
+    454.76,
+    16.55,
+    25179.15,
+    16555.31,
+    22367.24
+   ],
+   "cpu": [
+    1.7,
+    6.8,
+    2.8,
+    2.2,
+    1.5,
+    0.9,
+    1.8,
+    5.2,
+    2.3,
+    5.5,
+    8.8,
+    1.8,
+    3.7,
+    1.8,
+    2.8,
+    1.4,
+    3.7,
+    3.3,
+    1.7,
+    1.3,
+    4.2,
+    1.2,
+    1.6,
+    1.7,
+    9.9,
+    2.6,
+    0.5,
+    2.0,
+    1.8,
+    2.4,
+    1.5,
+    0.8,
+    1.6,
+    1.2,
+    0.7,
+    1.6,
+    1.6,
+    2.3,
+    9.4,
+    4.7,
+    1.1,
+    3.5,
+    10.5,
+    1.6,
+    1.2,
+    1.3,
+    4.9,
+    9.2,
+    6.2,
+    7.5,
+    8.5,
+    22.5,
+    12.5,
+    14.4,
+    12.6,
+    8.8,
+    6.5,
+    7.5,
+    6.4,
+    6.2
+   ],
+   "ram_used": [
+    64108949504,
+    64264835072,
+    64269873152,
+    64217538560,
+    64220250112,
+    64209580032,
+    64204328960,
+    64140402688,
+    64131928064,
+    64163700736,
+    64240553984,
+    64240898048,
+    64248393728,
+    64243052544,
+    64234790912,
+    64233521152,
+    64232390656,
+    64308101120,
+    64309403648,
+    64297762816,
+    64329601024,
+    64305721344,
+    64307060736,
+    64304783360,
+    64363835392,
+    64356347904,
+    64354099200,
+    64356909056,
+    64359067648,
+    64329744384,
+    64330743808,
+    64332496896,
+    64325115904,
+    64319610880,
+    64320966656,
+    64320552960,
+    64315297792,
+    64318357504,
+    64285126656,
+    64085762048,
+    64088686592,
+    64091111424,
+    64247578624,
+    64251699200,
+    64241737728,
+    64244289536,
+    64283144192,
+    63581208576,
+    63583793152,
+    63583657984,
+    63716282368,
+    63680327680,
+    63688806400,
+    63692079104,
+    63653728256,
+    63611449344,
+    63604965376,
+    63573368832,
+    63636836352,
+    63624777728
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.02,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    59.38,
+    0.0,
+    0.0,
+    0.0,
+    34.69,
+    51.95,
+    0.0,
+    0.0,
+    11.38,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790714887.0144231
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090-needle/needle-28k-1.json

@@ -0,0 +1,830 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090-needle",
+ "mode": "needle",
+ "ctx": "28k",
+ "target_tokens": 28672,
+ "run": 1,
+ "run_id": "94aa92e58ada",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:48:07",
+ "prompt_tokens": 29820,
+ "completion_tokens": 8,
+ "ttft_s": 14.138,
+ "e2e_s": 14.241,
+ "api_prompt_tps": 2109.2,
+ "api_decode_tps": 68.2,
+ "engine_prompt_tps": 2134.7,
+ "engine_decode_tps": 66.5,
+ "log_lines": [
+  "strata serve: prompt 29820 tokens = 0 reused + 29820 read in 13969 ms (2134.7 tok/s), 8 generated in 120 ms (66.5 tok/s), drafts accepted 6 of 6, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 80.6% (3033 hits / 3764 lookups)"
+ ],
+ "answer_head": "64EC0999",
+ "needle_found": true,
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714887.1579242,
+    "duration_s": 14.1,
+    "finish": "stop",
+    "prompt_tokens": 29820,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 13969.3,
+    "decode_ms": 120.3,
+    "decode_tok_s": 66.5,
+    "hit_rate": 0.806
+   },
+   {
+    "projection": null,
+    "time": 1790714877.4157934,
+    "duration_s": 9.1,
+    "finish": "stop",
+    "prompt_tokens": 17346,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 8971.0,
+    "decode_ms": 118.7,
+    "decode_tok_s": 67.4,
+    "hit_rate": 0.806
+   },
+   {
+    "projection": null,
+    "time": 1790714872.57582,
+    "duration_s": 4.3,
+    "finish": "stop",
+    "prompt_tokens": 7493,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 4124.7,
+    "decode_ms": 123.2,
+    "decode_tok_s": 64.9,
+    "hit_rate": 0.748
+   },
+   {
+    "projection": null,
+    "time": 1790714759.725435,
+    "duration_s": 63.5,
+    "finish": "length",
+    "prompt_tokens": 129913,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 59900.8,
+    "decode_ms": 3556.3,
+    "decode_tok_s": 72.0,
+    "hit_rate": 0.956
+   },
+   {
+    "projection": null,
+    "time": 1790714695.4286683,
+    "duration_s": 63.2,
+    "finish": "length",
+    "prompt_tokens": 130360,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 59973.9,
+    "decode_ms": 3226.1,
+    "decode_tok_s": 79.4,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714636.537164,
+    "duration_s": 57.8,
+    "finish": "length",
+    "prompt_tokens": 119256,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 54577.1,
+    "decode_ms": 3229.8,
+    "decode_tok_s": 79.3,
+    "hit_rate": 0.955
+   },
+   {
+    "projection": null,
+    "time": 1790714507.7469585,
+    "duration_s": 17.4,
+    "finish": "length",
+    "prompt_tokens": 31602,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14617.7,
+    "decode_ms": 2785.2,
+    "decode_tok_s": 91.9,
+    "hit_rate": 0.965
+   },
+   {
+    "projection": null,
+    "time": 1790714489.1532733,
+    "duration_s": 17.9,
+    "finish": "length",
+    "prompt_tokens": 31582,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14618.7,
+    "decode_ms": 3270.7,
+    "decode_tok_s": 78.3,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714469.1007972,
+    "duration_s": 19.4,
+    "finish": "length",
+    "prompt_tokens": 34932,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 16553.9,
+    "decode_ms": 2826.7,
+    "decode_tok_s": 90.6,
+    "hit_rate": 0.941
+   },
+   {
+    "projection": null,
+    "time": 1790714463.1913157,
+    "duration_s": 5.2,
+    "finish": "length",
+    "prompt_tokens": 4201,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2365.6,
+    "decode_ms": 2868.4,
+    "decode_tok_s": 89.2,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714458.6299415,
+    "duration_s": 4.0,
+    "finish": "stop",
+    "prompt_tokens": 4200,
+    "reused": 0,
+    "output_tokens": 142,
+    "engine_generated": 142,
+    "prompt_ms": 2370.0,
+    "decode_ms": 1660.0,
+    "decode_tok_s": 85.5,
+    "hit_rate": 0.896
+   },
+   {
+    "projection": null,
+    "time": 1790714453.0791314,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 3881,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2223.8,
+    "decode_ms": 2790.5,
+    "decode_tok_s": 91.7,
+    "hit_rate": 0.973
+   }
+  ],
+  "requests_kept": 25,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 25,
+   "prompt_tokens": 661645,
+   "reused": 0,
+   "output_tokens": 4670,
+   "prompt_ms": 316952.2,
+   "decode_ms": 55149.499999999985
+  },
+  "hardware": {
+   "gpu_util": 99,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 49,
+   "gpu_power": 278.531,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 86.068359375,
+   "gpu_pcie_tx_mb": 76.1982421875,
+   "cpu": 6.7,
+   "ram_used": 63515471872,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.0,
+   "disk_write_mb": 0.08241686128693117,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    99,
+    100,
+    99,
+    100,
+    0,
+    100,
+    99,
+    99,
+    100,
+    100,
+    99,
+    90,
+    100,
+    0,
+    99,
+    99,
+    100,
+    96,
+    100,
+    99,
+    99,
+    99,
+    100,
+    99,
+    100,
+    99,
+    99
+   ],
+   "gpu_mem_used": [
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    44,
+    44,
+    42,
+    42,
+    42,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    47,
+    48,
+    49,
+    49,
+    46,
+    49,
+    49,
+    48,
+    49,
+    49,
+    49,
+    48,
+    49,
+    46,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49
+   ],
+   "gpu_power": [
+    121.26,
+    121.23,
+    101.81,
+    59.35,
+    35.84,
+    35.18,
+    35.01,
+    35.2,
+    35.0,
+    34.94,
+    35.11,
+    35.02,
+    35.03,
+    35.11,
+    34.94,
+    34.9,
+    35.04,
+    35.01,
+    35.02,
+    35.13,
+    34.98,
+    34.93,
+    35.03,
+    34.9,
+    34.91,
+    35.06,
+    34.94,
+    34.94,
+    35.11,
+    35.01,
+    35.09,
+    34.99,
+    34.98,
+    69.69,
+    278.48,
+    278.38,
+    276.27,
+    201.82,
+    237.39,
+    277.05,
+    276.41,
+    278.18,
+    277.05,
+    276.73,
+    274.45,
+    265.8,
+    225.08,
+    215.47,
+    277.53,
+    277.2,
+    277.77,
+    276.7,
+    277.63,
+    275.8,
+    266.11,
+    275.21,
+    277.53,
+    275.92,
+    279.07,
+    278.53
+   ],
+   "gpu_pcie_rx_mb": [
+    0.33,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.48,
+    0.52,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.57,
+    0.43,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.48,
+    0.43,
+    0.43,
+    0.67,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    12932.54,
+    9.63,
+    46.83,
+    24945.45,
+    2343.61,
+    13238.91,
+    26700.59,
+    116.78,
+    454.76,
+    16.55,
+    25179.15,
+    16555.31,
+    22367.24,
+    0.29,
+    6039.81,
+    15.31,
+    2050.78,
+    8.77,
+    51.02,
+    14.83,
+    11.11,
+    14.88,
+    25541.35,
+    21966.98,
+    22781.28,
+    20080.71,
+    86.07
+   ],
+   "cpu": [
+    2.8,
+    1.4,
+    3.7,
+    3.3,
+    1.7,
+    1.3,
+    4.2,
+    1.2,
+    1.6,
+    1.7,
+    9.9,
+    2.6,
+    0.5,
+    2.0,
+    1.8,
+    2.4,
+    1.5,
+    0.8,
+    1.6,
+    1.2,
+    0.7,
+    1.6,
+    1.6,
+    2.3,
+    9.4,
+    4.7,
+    1.1,
+    3.5,
+    10.5,
+    1.6,
+    1.2,
+    1.3,
+    4.9,
+    9.2,
+    6.2,
+    7.5,
+    8.5,
+    22.5,
+    12.5,
+    14.4,
+    12.6,
+    8.8,
+    6.5,
+    7.5,
+    6.4,
+    6.2,
+    20.7,
+    7.5,
+    5.6,
+    6.7,
+    10.9,
+    6.4,
+    7.2,
+    16.4,
+    12.9,
+    5.7,
+    8.2,
+    6.5,
+    6.1,
+    6.7
+   ],
+   "ram_used": [
+    64234790912,
+    64233521152,
+    64232390656,
+    64308101120,
+    64309403648,
+    64297762816,
+    64329601024,
+    64305721344,
+    64307060736,
+    64304783360,
+    64363835392,
+    64356347904,
+    64354099200,
+    64356909056,
+    64359067648,
+    64329744384,
+    64330743808,
+    64332496896,
+    64325115904,
+    64319610880,
+    64320966656,
+    64320552960,
+    64315297792,
+    64318357504,
+    64285126656,
+    64085762048,
+    64088686592,
+    64091111424,
+    64247578624,
+    64251699200,
+    64241737728,
+    64244289536,
+    64283144192,
+    63581208576,
+    63583793152,
+    63583657984,
+    63716282368,
+    63680327680,
+    63688806400,
+    63692079104,
+    63653728256,
+    63611449344,
+    63604965376,
+    63573368832,
+    63636836352,
+    63624777728,
+    63754727424,
+    63698673664,
+    63690715136,
+    63681667072,
+    63675179008,
+    63536058368,
+    63521837056,
+    63517446144,
+    63523024896,
+    63518388224,
+    63517155328,
+    63521734656,
+    63516585984,
+    63515471872
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    0.02,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    59.38,
+    0.0,
+    0.0,
+    0.0,
+    34.69,
+    51.95,
+    0.0,
+    0.0,
+    11.38,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.11,
+    4.58,
+    0.0,
+    0.0,
+    97.16,
+    0.0,
+    0.0,
+    0.0,
+    52.31,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790714901.7591343
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090-needle/needle-8k-1.json

@@ -0,0 +1,830 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090-needle",
+ "mode": "needle",
+ "ctx": "8k",
+ "target_tokens": 8192,
+ "run": 1,
+ "run_id": "3c83d69c2581",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:47:52",
+ "prompt_tokens": 7493,
+ "completion_tokens": 8,
+ "ttft_s": 4.2,
+ "e2e_s": 4.307,
+ "api_prompt_tps": 1783.9,
+ "api_decode_tps": 65.8,
+ "engine_prompt_tps": 1816.6,
+ "engine_decode_tps": 64.9,
+ "log_lines": [
+  "strata serve: prompt 7493 tokens = 0 reused + 7493 read in 4125 ms (1816.6 tok/s), 8 generated in 123 ms (64.9 tok/s), drafts accepted 6 of 6, 1 checkpoints",
+  "strata serve: decode expert cache hit rate: 74.8% (2704 hits / 3615 lookups)"
+ ],
+ "answer_head": "0E6415FF",
+ "needle_found": true,
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714872.57582,
+    "duration_s": 4.3,
+    "finish": "stop",
+    "prompt_tokens": 7493,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 4124.7,
+    "decode_ms": 123.2,
+    "decode_tok_s": 64.9,
+    "hit_rate": 0.748
+   },
+   {
+    "projection": null,
+    "time": 1790714759.725435,
+    "duration_s": 63.5,
+    "finish": "length",
+    "prompt_tokens": 129913,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 59900.8,
+    "decode_ms": 3556.3,
+    "decode_tok_s": 72.0,
+    "hit_rate": 0.956
+   },
+   {
+    "projection": null,
+    "time": 1790714695.4286683,
+    "duration_s": 63.2,
+    "finish": "length",
+    "prompt_tokens": 130360,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 59973.9,
+    "decode_ms": 3226.1,
+    "decode_tok_s": 79.4,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714636.537164,
+    "duration_s": 57.8,
+    "finish": "length",
+    "prompt_tokens": 119256,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 54577.1,
+    "decode_ms": 3229.8,
+    "decode_tok_s": 79.3,
+    "hit_rate": 0.955
+   },
+   {
+    "projection": null,
+    "time": 1790714507.7469585,
+    "duration_s": 17.4,
+    "finish": "length",
+    "prompt_tokens": 31602,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14617.7,
+    "decode_ms": 2785.2,
+    "decode_tok_s": 91.9,
+    "hit_rate": 0.965
+   },
+   {
+    "projection": null,
+    "time": 1790714489.1532733,
+    "duration_s": 17.9,
+    "finish": "length",
+    "prompt_tokens": 31582,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14618.7,
+    "decode_ms": 3270.7,
+    "decode_tok_s": 78.3,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714469.1007972,
+    "duration_s": 19.4,
+    "finish": "length",
+    "prompt_tokens": 34932,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 16553.9,
+    "decode_ms": 2826.7,
+    "decode_tok_s": 90.6,
+    "hit_rate": 0.941
+   },
+   {
+    "projection": null,
+    "time": 1790714463.1913157,
+    "duration_s": 5.2,
+    "finish": "length",
+    "prompt_tokens": 4201,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2365.6,
+    "decode_ms": 2868.4,
+    "decode_tok_s": 89.2,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714458.6299415,
+    "duration_s": 4.0,
+    "finish": "stop",
+    "prompt_tokens": 4200,
+    "reused": 0,
+    "output_tokens": 142,
+    "engine_generated": 142,
+    "prompt_ms": 2370.0,
+    "decode_ms": 1660.0,
+    "decode_tok_s": 85.5,
+    "hit_rate": 0.896
+   },
+   {
+    "projection": null,
+    "time": 1790714453.0791314,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 3881,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2223.8,
+    "decode_ms": 2790.5,
+    "decode_tok_s": 91.7,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790714448.5537117,
+    "duration_s": 4.0,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1305.1,
+    "decode_ms": 2690.8,
+    "decode_tok_s": 95.1,
+    "hit_rate": 0.972
+   },
+   {
+    "projection": null,
+    "time": 1790714443.8943996,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1149,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1297.6,
+    "decode_ms": 2847.5,
+    "decode_tok_s": 89.9,
+    "hit_rate": 0.971
+   }
+  ],
+  "requests_kept": 23,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 23,
+   "prompt_tokens": 614479,
+   "reused": 0,
+   "output_tokens": 4654,
+   "prompt_ms": 294011.9,
+   "decode_ms": 54910.499999999985
+  },
+  "hardware": {
+   "gpu_util": 100,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 49,
+   "gpu_power": 276.273,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 24945.44921875,
+   "gpu_pcie_tx_mb": 2986.287109375,
+   "cpu": 8.5,
+   "ram_used": 63716282368,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.0,
+   "disk_write_mb": 0.0,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    100,
+    99,
+    100,
+    99,
+    99,
+    85,
+    91,
+    96,
+    95,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    99,
+    100,
+    99,
+    100
+   ],
+   "gpu_mem_used": [
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    46,
+    45,
+    45,
+    44,
+    45,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    42,
+    42,
+    42,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    47,
+    48,
+    49,
+    49
+   ],
+   "gpu_power": [
+    277.85,
+    276.09,
+    278.51,
+    279.06,
+    278.13,
+    239.73,
+    262.71,
+    270.73,
+    273.54,
+    162.82,
+    148.08,
+    135.02,
+    121.86,
+    121.63,
+    121.78,
+    121.51,
+    121.24,
+    121.16,
+    121.27,
+    121.24,
+    121.23,
+    121.05,
+    121.06,
+    121.26,
+    121.23,
+    101.81,
+    59.35,
+    35.84,
+    35.18,
+    35.01,
+    35.2,
+    35.0,
+    34.94,
+    35.11,
+    35.02,
+    35.03,
+    35.11,
+    34.94,
+    34.9,
+    35.04,
+    35.01,
+    35.02,
+    35.13,
+    34.98,
+    34.93,
+    35.03,
+    34.9,
+    34.91,
+    35.06,
+    34.94,
+    34.94,
+    35.11,
+    35.01,
+    35.09,
+    34.99,
+    34.98,
+    69.69,
+    278.48,
+    278.38,
+    276.27
+   ],
+   "gpu_pcie_rx_mb": [
+    12025.83,
+    22118.66,
+    18928.05,
+    18.36,
+    86.12,
+    284.24,
+    9240.67,
+    7805.97,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.29,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.38,
+    0.33,
+    0.29,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.48,
+    0.52,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.57,
+    0.43,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.48,
+    0.43,
+    0.43,
+    0.67,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    12932.54,
+    9.63,
+    46.83,
+    24945.45
+   ],
+   "cpu": [
+    8.2,
+    6.7,
+    6.2,
+    6.0,
+    13.2,
+    64.5,
+    99.9,
+    99.0,
+    86.8,
+    1.7,
+    6.8,
+    2.8,
+    2.2,
+    1.5,
+    0.9,
+    1.8,
+    5.2,
+    2.3,
+    5.5,
+    8.8,
+    1.8,
+    3.7,
+    1.8,
+    2.8,
+    1.4,
+    3.7,
+    3.3,
+    1.7,
+    1.3,
+    4.2,
+    1.2,
+    1.6,
+    1.7,
+    9.9,
+    2.6,
+    0.5,
+    2.0,
+    1.8,
+    2.4,
+    1.5,
+    0.8,
+    1.6,
+    1.2,
+    0.7,
+    1.6,
+    1.6,
+    2.3,
+    9.4,
+    4.7,
+    1.1,
+    3.5,
+    10.5,
+    1.6,
+    1.2,
+    1.3,
+    4.9,
+    9.2,
+    6.2,
+    7.5,
+    8.5
+   ],
+   "ram_used": [
+    64100171776,
+    64087904256,
+    64069447680,
+    64067096576,
+    64095731712,
+    64224346112,
+    64185745408,
+    64170352640,
+    64157536256,
+    64108949504,
+    64264835072,
+    64269873152,
+    64217538560,
+    64220250112,
+    64209580032,
+    64204328960,
+    64140402688,
+    64131928064,
+    64163700736,
+    64240553984,
+    64240898048,
+    64248393728,
+    64243052544,
+    64234790912,
+    64233521152,
+    64232390656,
+    64308101120,
+    64309403648,
+    64297762816,
+    64329601024,
+    64305721344,
+    64307060736,
+    64304783360,
+    64363835392,
+    64356347904,
+    64354099200,
+    64356909056,
+    64359067648,
+    64329744384,
+    64330743808,
+    64332496896,
+    64325115904,
+    64319610880,
+    64320966656,
+    64320552960,
+    64315297792,
+    64318357504,
+    64285126656,
+    64085762048,
+    64088686592,
+    64091111424,
+    64247578624,
+    64251699200,
+    64241737728,
+    64244289536,
+    64283144192,
+    63581208576,
+    63583793152,
+    63583657984,
+    63716282368
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.89,
+    2.52,
+    3.25,
+    2.83,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.02,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    59.38,
+    0.0,
+    0.0,
+    0.0
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    61.23,
+    76.01,
+    95.36,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790714877.3325372
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090-two-turn/two-turn-16k-1.json

@@ -0,0 +1,835 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090-two-turn",
+ "mode": "two-turn",
+ "ctx": "16k",
+ "target_tokens": 16384,
+ "run": 1,
+ "run_id": "7aed17dd03cd",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:48:48",
+ "prompt_tokens": 15636,
+ "completion_tokens": 256,
+ "ttft_s": 7.488,
+ "e2e_s": 10.882,
+ "api_prompt_tps": 2088.0,
+ "api_decode_tps": 75.1,
+ "engine_prompt_tps": 2114.8,
+ "engine_decode_tps": 75.1,
+ "log_lines": [
+  "strata serve: prompt 15636 tokens = 0 reused + 15636 read in 7394 ms (2114.8 tok/s), 256 generated in 3410 ms (75.1 tok/s), drafts accepted 152 of 215, 1 checkpoints",
+  "strata serve: decode expert cache hit rate: 93.0% (136865 hits / 147168 lookups)"
+ ],
+ "answer_head": "## What `README.md` does\n\n`README.md` is the project's landing page and primary user-facing documentation. It serves several purposes:\n\n1. **Marketing/overview** \u2014 Opens with a headline claim (\"Run a 125-billion-parameter AI model on a normal gaming PC\") and a visual demo (the pagoda video) to immed",
+ "turn2_ttft_s": 0.438,
+ "turn2_prompt_tokens": 15915,
+ "turn2_log_lines": [
+  "strata serve: prompt 15915 tokens = 15892 reused + 23 read in 332 ms (69.3 tok/s), 46 generated in 661 ms (69.6 tok/s), drafts accepted 27 of 44, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 96.4% (29047 hits / 30124 lookups)"
+ ],
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714939.7647502,
+    "duration_s": 1.0,
+    "finish": "stop",
+    "prompt_tokens": 15915,
+    "reused": 15892,
+    "output_tokens": 46,
+    "engine_generated": 46,
+    "prompt_ms": 331.9,
+    "decode_ms": 660.8,
+    "decode_tok_s": 69.6,
+    "hit_rate": 0.964
+   },
+   {
+    "projection": null,
+    "time": 1790714928.3706348,
+    "duration_s": 10.8,
+    "finish": "length",
+    "prompt_tokens": 15636,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 7393.6,
+    "decode_ms": 3409.5,
+    "decode_tok_s": 75.1,
+    "hit_rate": 0.93
+   },
+   {
+    "projection": null,
+    "time": 1790714887.1579242,
+    "duration_s": 14.1,
+    "finish": "stop",
+    "prompt_tokens": 29820,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 13969.3,
+    "decode_ms": 120.3,
+    "decode_tok_s": 66.5,
+    "hit_rate": 0.806
+   },
+   {
+    "projection": null,
+    "time": 1790714877.4157934,
+    "duration_s": 9.1,
+    "finish": "stop",
+    "prompt_tokens": 17346,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 8971.0,
+    "decode_ms": 118.7,
+    "decode_tok_s": 67.4,
+    "hit_rate": 0.806
+   },
+   {
+    "projection": null,
+    "time": 1790714872.57582,
+    "duration_s": 4.3,
+    "finish": "stop",
+    "prompt_tokens": 7493,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 4124.7,
+    "decode_ms": 123.2,
+    "decode_tok_s": 64.9,
+    "hit_rate": 0.748
+   },
+   {
+    "projection": null,
+    "time": 1790714759.725435,
+    "duration_s": 63.5,
+    "finish": "length",
+    "prompt_tokens": 129913,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 59900.8,
+    "decode_ms": 3556.3,
+    "decode_tok_s": 72.0,
+    "hit_rate": 0.956
+   },
+   {
+    "projection": null,
+    "time": 1790714695.4286683,
+    "duration_s": 63.2,
+    "finish": "length",
+    "prompt_tokens": 130360,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 59973.9,
+    "decode_ms": 3226.1,
+    "decode_tok_s": 79.4,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714636.537164,
+    "duration_s": 57.8,
+    "finish": "length",
+    "prompt_tokens": 119256,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 54577.1,
+    "decode_ms": 3229.8,
+    "decode_tok_s": 79.3,
+    "hit_rate": 0.955
+   },
+   {
+    "projection": null,
+    "time": 1790714507.7469585,
+    "duration_s": 17.4,
+    "finish": "length",
+    "prompt_tokens": 31602,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14617.7,
+    "decode_ms": 2785.2,
+    "decode_tok_s": 91.9,
+    "hit_rate": 0.965
+   },
+   {
+    "projection": null,
+    "time": 1790714489.1532733,
+    "duration_s": 17.9,
+    "finish": "length",
+    "prompt_tokens": 31582,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14618.7,
+    "decode_ms": 3270.7,
+    "decode_tok_s": 78.3,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714469.1007972,
+    "duration_s": 19.4,
+    "finish": "length",
+    "prompt_tokens": 34932,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 16553.9,
+    "decode_ms": 2826.7,
+    "decode_tok_s": 90.6,
+    "hit_rate": 0.941
+   },
+   {
+    "projection": null,
+    "time": 1790714463.1913157,
+    "duration_s": 5.2,
+    "finish": "length",
+    "prompt_tokens": 4201,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2365.6,
+    "decode_ms": 2868.4,
+    "decode_tok_s": 89.2,
+    "hit_rate": 0.96
+   }
+  ],
+  "requests_kept": 27,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 27,
+   "prompt_tokens": 693196,
+   "reused": 15892,
+   "output_tokens": 4972,
+   "prompt_ms": 324677.7,
+   "decode_ms": 59219.79999999999
+  },
+  "hardware": {
+   "gpu_util": 94,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 48,
+   "gpu_power": 197.275,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 822.3525390625,
+   "gpu_pcie_tx_mb": 183.15234375,
+   "cpu": 52.4,
+   "ram_used": 63860678656,
+   "ram_total": 177175990272,
+   "disk_read_mb": 1.1632307226246328,
+   "disk_write_mb": 0.2954830452969325,
+   "tok_s": 50.58164583911189,
+   "tok_s_mean": 50.57911214220715
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    100,
+    99,
+    99,
+    100,
+    100,
+    99,
+    90,
+    100,
+    0,
+    99,
+    99,
+    100,
+    96,
+    100,
+    99,
+    99,
+    99,
+    100,
+    99,
+    100,
+    99,
+    99,
+    96,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    99,
+    99,
+    100,
+    99,
+    99,
+    100,
+    100,
+    100,
+    95,
+    97,
+    93,
+    94
+   ],
+   "gpu_mem_used": [
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    49,
+    49,
+    48,
+    49,
+    49,
+    49,
+    48,
+    49,
+    46,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    48,
+    46,
+    45,
+    45,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    44,
+    42,
+    42,
+    42,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    41,
+    47,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    47,
+    48,
+    49,
+    49,
+    48
+   ],
+   "gpu_power": [
+    237.39,
+    277.05,
+    276.41,
+    278.18,
+    277.05,
+    276.73,
+    274.45,
+    265.8,
+    225.08,
+    215.47,
+    277.53,
+    277.2,
+    277.77,
+    276.7,
+    277.63,
+    275.8,
+    266.11,
+    275.21,
+    277.53,
+    275.92,
+    279.07,
+    278.53,
+    241.13,
+    171.76,
+    147.92,
+    134.65,
+    121.43,
+    121.4,
+    121.36,
+    121.31,
+    121.11,
+    121.03,
+    121.2,
+    121.14,
+    120.8,
+    121.02,
+    121.17,
+    121.22,
+    102.94,
+    60.93,
+    36.12,
+    35.26,
+    35.35,
+    35.31,
+    35.23,
+    35.08,
+    35.09,
+    35.22,
+    69.73,
+    277.07,
+    276.87,
+    276.87,
+    278.61,
+    277.22,
+    278.12,
+    247.41,
+    259.45,
+    271.65,
+    275.28,
+    197.28
+   ],
+   "gpu_pcie_rx_mb": [
+    13238.91,
+    26700.59,
+    116.78,
+    454.76,
+    16.55,
+    25179.15,
+    16555.31,
+    22367.24,
+    0.29,
+    6039.81,
+    15.31,
+    2050.78,
+    8.77,
+    51.02,
+    14.83,
+    11.11,
+    14.88,
+    25541.35,
+    21966.98,
+    22781.28,
+    20080.71,
+    86.07,
+    0.33,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.29,
+    0.33,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.43,
+    0.33,
+    0.33,
+    0.29,
+    0.33,
+    0.38,
+    0.43,
+    0.38,
+    0.43,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.43,
+    7494.78,
+    15213.11,
+    13.83,
+    86.31,
+    6.25,
+    4.48,
+    7.34,
+    1956.56,
+    3779.22,
+    10756.4,
+    0.33,
+    822.35
+   ],
+   "cpu": [
+    12.5,
+    14.4,
+    12.6,
+    8.8,
+    6.5,
+    7.5,
+    6.4,
+    6.2,
+    20.7,
+    7.5,
+    5.6,
+    6.7,
+    10.9,
+    6.4,
+    7.2,
+    16.4,
+    12.9,
+    5.7,
+    8.2,
+    6.5,
+    6.1,
+    6.7,
+    20.8,
+    2.1,
+    8.1,
+    1.0,
+    2.4,
+    4.3,
+    1.2,
+    15.4,
+    6.0,
+    0.7,
+    1.7,
+    1.9,
+    1.9,
+    1.8,
+    1.1,
+    2.0,
+    2.0,
+    1.3,
+    1.6,
+    1.5,
+    3.7,
+    3.2,
+    9.8,
+    1.4,
+    4.8,
+    1.6,
+    8.9,
+    6.6,
+    7.0,
+    5.9,
+    5.6,
+    7.0,
+    10.7,
+    49.0,
+    100.0,
+    99.6,
+    87.8,
+    52.4
+   ],
+   "ram_used": [
+    63688806400,
+    63692079104,
+    63653728256,
+    63611449344,
+    63604965376,
+    63573368832,
+    63636836352,
+    63624777728,
+    63754727424,
+    63698673664,
+    63690715136,
+    63681667072,
+    63675179008,
+    63536058368,
+    63521837056,
+    63517446144,
+    63523024896,
+    63518388224,
+    63517155328,
+    63521734656,
+    63516585984,
+    63515471872,
+    63598735360,
+    63575552000,
+    63746940928,
+    63743287296,
+    63744950272,
+    63746498560,
+    63748960256,
+    63781244928,
+    63777951744,
+    63778541568,
+    63778627584,
+    63784894464,
+    63788924928,
+    63789916160,
+    63785603072,
+    63778463744,
+    63786168320,
+    63781711872,
+    63785504768,
+    63775502336,
+    63798136832,
+    63833931776,
+    63825649664,
+    63827275776,
+    63826427904,
+    63833698304,
+    63620595712,
+    63618826240,
+    63602626560,
+    63602855936,
+    63603830784,
+    63609753600,
+    63610949632,
+    63683670016,
+    63692382208,
+    63679488000,
+    63682031616,
+    63860678656
+   ],
+   "disk_read_mb": [
+    51.95,
+    0.0,
+    0.0,
+    11.38,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.11,
+    4.58,
+    0.0,
+    0.0,
+    97.16,
+    0.0,
+    0.0,
+    0.0,
+    52.31,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.15,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    2.41,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    1.29,
+    3.18,
+    2.31,
+    2.73,
+    1.16
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    68.59,
+    62.69,
+    101.15,
+    0.0,
+    50.58
+   ]
+  },
+  "time": 1790714941.2661076
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-128k-1.json

@@ -0,0 +1,829 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "128k",
+ "target_tokens": 129792,
+ "run": 1,
+ "run_id": "183f9c0b2a25",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:43:56",
+ "prompt_tokens": 119256,
+ "completion_tokens": 256,
+ "ttft_s": 55.136,
+ "e2e_s": 58.347,
+ "api_prompt_tps": 2162.9,
+ "api_decode_tps": 79.4,
+ "engine_prompt_tps": 2185.1,
+ "engine_decode_tps": 79.3,
+ "log_lines": [
+  "strata serve: prompt 119256 tokens = 0 reused + 119256 read in 54577 ms (2185.1 tok/s), 256 generated in 3230 ms (79.3 tok/s), drafts accepted 151 of 208, 6 checkpoints",
+  "strata serve: decode expert cache hit rate: 95.5% (140181 hits / 146787 lookups)"
+ ],
+ "answer_head": "The last file shown is `include/strata/kernels/cpu/pool.hpp`. It declares the CPU expert pool (`ExpertPool`), which is the engine's mechanism for computing the routed MoE experts that do not fit in VRAM.\n\n**What it does:**\n\n1.  **Thread Management:** It creates a pool of worker threads pinned to phy",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714636.537164,
+    "duration_s": 57.8,
+    "finish": "length",
+    "prompt_tokens": 119256,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 54577.1,
+    "decode_ms": 3229.8,
+    "decode_tok_s": 79.3,
+    "hit_rate": 0.955
+   },
+   {
+    "projection": null,
+    "time": 1790714507.7469585,
+    "duration_s": 17.4,
+    "finish": "length",
+    "prompt_tokens": 31602,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14617.7,
+    "decode_ms": 2785.2,
+    "decode_tok_s": 91.9,
+    "hit_rate": 0.965
+   },
+   {
+    "projection": null,
+    "time": 1790714489.1532733,
+    "duration_s": 17.9,
+    "finish": "length",
+    "prompt_tokens": 31582,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14618.7,
+    "decode_ms": 3270.7,
+    "decode_tok_s": 78.3,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714469.1007972,
+    "duration_s": 19.4,
+    "finish": "length",
+    "prompt_tokens": 34932,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 16553.9,
+    "decode_ms": 2826.7,
+    "decode_tok_s": 90.6,
+    "hit_rate": 0.941
+   },
+   {
+    "projection": null,
+    "time": 1790714463.1913157,
+    "duration_s": 5.2,
+    "finish": "length",
+    "prompt_tokens": 4201,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2365.6,
+    "decode_ms": 2868.4,
+    "decode_tok_s": 89.2,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714458.6299415,
+    "duration_s": 4.0,
+    "finish": "stop",
+    "prompt_tokens": 4200,
+    "reused": 0,
+    "output_tokens": 142,
+    "engine_generated": 142,
+    "prompt_ms": 2370.0,
+    "decode_ms": 1660.0,
+    "decode_tok_s": 85.5,
+    "hit_rate": 0.896
+   },
+   {
+    "projection": null,
+    "time": 1790714453.0791314,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 3881,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2223.8,
+    "decode_ms": 2790.5,
+    "decode_tok_s": 91.7,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790714448.5537117,
+    "duration_s": 4.0,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1305.1,
+    "decode_ms": 2690.8,
+    "decode_tok_s": 95.1,
+    "hit_rate": 0.972
+   },
+   {
+    "projection": null,
+    "time": 1790714443.8943996,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1149,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1297.6,
+    "decode_ms": 2847.5,
+    "decode_tok_s": 89.9,
+    "hit_rate": 0.971
+   },
+   {
+    "projection": null,
+    "time": 1790714439.2770011,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1092,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1351.4,
+    "decode_ms": 2752.6,
+    "decode_tok_s": 93.0,
+    "hit_rate": 0.954
+   },
+   {
+    "projection": null,
+    "time": 1790714377.2599747,
+    "duration_s": 17.6,
+    "finish": "length",
+    "prompt_tokens": 31605,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14608.5,
+    "decode_ms": 3015.6,
+    "decode_tok_s": 84.9,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714359.1307795,
+    "duration_s": 17.5,
+    "finish": "length",
+    "prompt_tokens": 31589,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14609.7,
+    "decode_ms": 2833.5,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.959
+   }
+  ],
+  "requests_kept": 20,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 20,
+   "prompt_tokens": 346713,
+   "reused": 0,
+   "output_tokens": 4134,
+   "prompt_ms": 170012.5,
+   "decode_ms": 48004.89999999999
+  },
+  "hardware": {
+   "gpu_util": 0,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 45,
+   "gpu_power": 245.728,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.3330078125,
+   "gpu_pcie_tx_mb": 0.4765625,
+   "cpu": 55.3,
+   "ram_used": 63994736640,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.173699813344069,
+   "disk_write_mb": 0.07870292427625078,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0,
+    0,
+    0,
+    0,
+    100,
+    100,
+    99,
+    100,
+    99,
+    100,
+    99,
+    99,
+    100,
+    99,
+    99,
+    100,
+    99,
+    97,
+    96,
+    99,
+    99,
+    99,
+    100,
+    100,
+    97,
+    100,
+    100,
+    99,
+    97,
+    99,
+    100,
+    100,
+    99,
+    99,
+    98,
+    99,
+    99,
+    99,
+    98,
+    100,
+    100,
+    99,
+    99,
+    100,
+    100,
+    100,
+    99,
+    99,
+    99,
+    97,
+    100,
+    100,
+    100,
+    4,
+    100,
+    100,
+    89,
+    94,
+    95,
+    0
+   ],
+   "gpu_mem_used": [
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    38,
+    38,
+    38,
+    38,
+    45,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    47,
+    47,
+    47,
+    47,
+    46,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    46,
+    46,
+    46,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    46,
+    47,
+    47,
+    46,
+    47,
+    47,
+    45
+   ],
+   "gpu_power": [
+    34.95,
+    35.05,
+    34.94,
+    34.95,
+    146.71,
+    278.26,
+    278.02,
+    277.13,
+    278.38,
+    277.26,
+    277.39,
+    265.55,
+    278.37,
+    277.81,
+    278.33,
+    277.55,
+    276.64,
+    268.76,
+    278.4,
+    278.09,
+    279.44,
+    277.89,
+    278.06,
+    278.94,
+    266.27,
+    278.02,
+    278.54,
+    277.25,
+    277.61,
+    278.51,
+    277.09,
+    267.68,
+    277.75,
+    277.97,
+    277.81,
+    277.03,
+    277.2,
+    278.97,
+    266.74,
+    277.84,
+    277.04,
+    277.08,
+    277.19,
+    277.7,
+    278.99,
+    275.31,
+    278.3,
+    278.49,
+    277.62,
+    278.02,
+    276.32,
+    276.96,
+    277.41,
+    263.19,
+    267.16,
+    277.12,
+    240.24,
+    275.77,
+    278.35,
+    245.73
+   ],
+   "gpu_pcie_rx_mb": [
+    0.43,
+    0.38,
+    0.43,
+    0.62,
+    11.06,
+    10427.81,
+    2646.73,
+    383.9,
+    8262.3,
+    11.68,
+    24650.05,
+    893.64,
+    18094.54,
+    8958.05,
+    12.25,
+    6881.52,
+    17403.6,
+    4113.72,
+    25011.21,
+    21669.05,
+    15617.75,
+    11.78,
+    26451.63,
+    0.33,
+    5449.25,
+    2654.98,
+    14.88,
+    22.22,
+    15.59,
+    7.49,
+    1.29,
+    17076.97,
+    2352.29,
+    26573.8,
+    19.55,
+    15.45,
+    3977.3,
+    0.29,
+    16.07,
+    3622.44,
+    16839.27,
+    13.49,
+    13131.9,
+    19713.45,
+    26861.86,
+    17424.06,
+    12115.96,
+    5297.42,
+    26787.19,
+    12211.47,
+    5737.21,
+    18827.01,
+    8191.39,
+    15610.6,
+    18543.15,
+    81.97,
+    3906.87,
+    3778.6,
+    4665.28,
+    0.33
+   ],
+   "cpu": [
+    1.5,
+    6.0,
+    1.8,
+    7.9,
+    6.4,
+    6.6,
+    16.7,
+    12.8,
+    7.5,
+    6.4,
+    5.6,
+    6.5,
+    6.9,
+    6.6,
+    17.4,
+    5.9,
+    5.5,
+    14.4,
+    12.9,
+    6.6,
+    19.0,
+    17.6,
+    6.4,
+    7.6,
+    15.3,
+    10.6,
+    6.2,
+    15.5,
+    14.7,
+    6.1,
+    5.9,
+    17.4,
+    8.8,
+    6.2,
+    15.9,
+    21.5,
+    7.2,
+    7.4,
+    16.6,
+    7.1,
+    6.6,
+    8.9,
+    14.9,
+    6.1,
+    5.8,
+    10.7,
+    14.6,
+    6.8,
+    5.9,
+    19.6,
+    12.3,
+    6.2,
+    7.7,
+    6.1,
+    5.9,
+    6.3,
+    64.3,
+    100.0,
+    100.0,
+    55.3
+   ],
+   "ram_used": [
+    63599173632,
+    63581626368,
+    63584313344,
+    63628144640,
+    63383064576,
+    63385989120,
+    63410585600,
+    63420211200,
+    63412383744,
+    63414505472,
+    63524802560,
+    63511179264,
+    63497076736,
+    63495684096,
+    63489646592,
+    63487651840,
+    63472156672,
+    63594835968,
+    63578550272,
+    63580782592,
+    63599607808,
+    63609155584,
+    63609020416,
+    63658397696,
+    63611748352,
+    63599493120,
+    63591612416,
+    63610773504,
+    63607812096,
+    63605293056,
+    63719862272,
+    63720685568,
+    63714033664,
+    63716544512,
+    63707664384,
+    63700750336,
+    63700516864,
+    63791661056,
+    63808417792,
+    63809699840,
+    63813054464,
+    63809191936,
+    63809736704,
+    63785721856,
+    63788429312,
+    63899906048,
+    63903354880,
+    63885254656,
+    63887302656,
+    63919407104,
+    63917928448,
+    63921512448,
+    63919017984,
+    64030937088,
+    64033521664,
+    64022724608,
+    64014991360,
+    64020590592,
+    64011530240,
+    63994736640
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.59,
+    0.0,
+    0.0,
+    0.18,
+    0.0,
+    0.0,
+    0.31,
+    0.0,
+    0.0,
+    5.59,
+    188.01,
+    0.0,
+    0.0,
+    123.58,
+    109.11,
+    0.0,
+    61.11,
+    110.87,
+    0.0,
+    0.0,
+    173.38,
+    90.91,
+    0.0,
+    90.3,
+    151.87,
+    0.0,
+    0.0,
+    198.92,
+    33.71,
+    0.0,
+    100.92,
+    100.99,
+    0.0,
+    0.0,
+    190.6,
+    21.65,
+    0.0,
+    46.97,
+    157.79,
+    0.0,
+    0.0,
+    54.2,
+    150.86,
+    0.0,
+    0.0,
+    89.37,
+    16.55,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    1.57,
+    4.0,
+    4.88,
+    2.17
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    80.35,
+    77.04,
+    74.05,
+    0.0
+   ]
+  },
+  "time": 1790714694.882821
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-128k-2.json

@@ -0,0 +1,829 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "128k",
+ "target_tokens": 129792,
+ "run": 2,
+ "run_id": "815c67976126",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:44:54",
+ "prompt_tokens": 130360,
+ "completion_tokens": 256,
+ "ttft_s": 60.567,
+ "e2e_s": 63.777,
+ "api_prompt_tps": 2152.3,
+ "api_decode_tps": 79.4,
+ "engine_prompt_tps": 2173.6,
+ "engine_decode_tps": 79.4,
+ "log_lines": [
+  "strata serve: prompt 130360 tokens = 0 reused + 130360 read in 59974 ms (2173.6 tok/s), 256 generated in 3226 ms (79.4 tok/s), drafts accepted 158 of 225, 6 checkpoints",
+  "strata serve: decode expert cache hit rate: 94.5% (142001 hits / 150326 lookups)"
+ ],
+ "answer_head": "I'll analyze the last file in the repository listing.\n\n## What `include/strata/kernels/gdn.hpp` does\n\nThis header declares the **non-projection parts of the Gated Delta-Net (GDN) layer** \u2014 the \"mixer\" used in 36 of the model's 48 layers (the other 12 are QSA full-attention layers). It is the C++ cou",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714695.4286683,
+    "duration_s": 63.2,
+    "finish": "length",
+    "prompt_tokens": 130360,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 59973.9,
+    "decode_ms": 3226.1,
+    "decode_tok_s": 79.4,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714636.537164,
+    "duration_s": 57.8,
+    "finish": "length",
+    "prompt_tokens": 119256,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 54577.1,
+    "decode_ms": 3229.8,
+    "decode_tok_s": 79.3,
+    "hit_rate": 0.955
+   },
+   {
+    "projection": null,
+    "time": 1790714507.7469585,
+    "duration_s": 17.4,
+    "finish": "length",
+    "prompt_tokens": 31602,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14617.7,
+    "decode_ms": 2785.2,
+    "decode_tok_s": 91.9,
+    "hit_rate": 0.965
+   },
+   {
+    "projection": null,
+    "time": 1790714489.1532733,
+    "duration_s": 17.9,
+    "finish": "length",
+    "prompt_tokens": 31582,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14618.7,
+    "decode_ms": 3270.7,
+    "decode_tok_s": 78.3,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714469.1007972,
+    "duration_s": 19.4,
+    "finish": "length",
+    "prompt_tokens": 34932,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 16553.9,
+    "decode_ms": 2826.7,
+    "decode_tok_s": 90.6,
+    "hit_rate": 0.941
+   },
+   {
+    "projection": null,
+    "time": 1790714463.1913157,
+    "duration_s": 5.2,
+    "finish": "length",
+    "prompt_tokens": 4201,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2365.6,
+    "decode_ms": 2868.4,
+    "decode_tok_s": 89.2,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714458.6299415,
+    "duration_s": 4.0,
+    "finish": "stop",
+    "prompt_tokens": 4200,
+    "reused": 0,
+    "output_tokens": 142,
+    "engine_generated": 142,
+    "prompt_ms": 2370.0,
+    "decode_ms": 1660.0,
+    "decode_tok_s": 85.5,
+    "hit_rate": 0.896
+   },
+   {
+    "projection": null,
+    "time": 1790714453.0791314,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 3881,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2223.8,
+    "decode_ms": 2790.5,
+    "decode_tok_s": 91.7,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790714448.5537117,
+    "duration_s": 4.0,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1305.1,
+    "decode_ms": 2690.8,
+    "decode_tok_s": 95.1,
+    "hit_rate": 0.972
+   },
+   {
+    "projection": null,
+    "time": 1790714443.8943996,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1149,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1297.6,
+    "decode_ms": 2847.5,
+    "decode_tok_s": 89.9,
+    "hit_rate": 0.971
+   },
+   {
+    "projection": null,
+    "time": 1790714439.2770011,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1092,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1351.4,
+    "decode_ms": 2752.6,
+    "decode_tok_s": 93.0,
+    "hit_rate": 0.954
+   },
+   {
+    "projection": null,
+    "time": 1790714377.2599747,
+    "duration_s": 17.6,
+    "finish": "length",
+    "prompt_tokens": 31605,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14608.5,
+    "decode_ms": 3015.6,
+    "decode_tok_s": 84.9,
+    "hit_rate": 0.966
+   }
+  ],
+  "requests_kept": 21,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 21,
+   "prompt_tokens": 477073,
+   "reused": 0,
+   "output_tokens": 4390,
+   "prompt_ms": 229986.4,
+   "decode_ms": 51230.999999999985
+  },
+  "hardware": {
+   "gpu_util": 95,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 48,
+   "gpu_power": 278.59,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 1822.99609375,
+   "gpu_pcie_tx_mb": 282.4296875,
+   "cpu": 100.0,
+   "ram_used": 63968800768,
+   "ram_total": 177175990272,
+   "disk_read_mb": 3.7524610679737007,
+   "disk_write_mb": 0.06727519842980738,
+   "tok_s": 75.51795929153683,
+   "tok_s_mean": 78.52742680374247
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    99,
+    99,
+    100,
+    97,
+    99,
+    100,
+    3,
+    99,
+    100,
+    98,
+    100,
+    99,
+    100,
+    100,
+    99,
+    99,
+    98,
+    100,
+    100,
+    15,
+    99,
+    100,
+    100,
+    99,
+    100,
+    99,
+    100,
+    99,
+    100,
+    99,
+    98,
+    99,
+    100,
+    11,
+    99,
+    99,
+    100,
+    100,
+    99,
+    99,
+    99,
+    99,
+    100,
+    99,
+    100,
+    100,
+    100,
+    100,
+    99,
+    100,
+    99,
+    100,
+    99,
+    100,
+    100,
+    100,
+    49,
+    97,
+    93,
+    95
+   ],
+   "gpu_mem_used": [
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    47,
+    47,
+    48,
+    48,
+    48,
+    47,
+    46,
+    47,
+    47,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    47,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    46,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    47,
+    48,
+    48,
+    48
+   ],
+   "gpu_power": [
+    242.39,
+    277.2,
+    277.34,
+    277.65,
+    278.23,
+    277.3,
+    263.86,
+    266.5,
+    276.0,
+    277.29,
+    276.57,
+    277.42,
+    277.59,
+    268.12,
+    277.51,
+    276.99,
+    278.25,
+    276.6,
+    278.17,
+    261.67,
+    270.19,
+    277.42,
+    278.17,
+    277.12,
+    278.28,
+    277.75,
+    275.69,
+    267.25,
+    277.04,
+    278.86,
+    278.14,
+    276.63,
+    278.3,
+    265.62,
+    266.61,
+    278.07,
+    276.73,
+    277.66,
+    277.57,
+    279.3,
+    278.07,
+    267.14,
+    278.22,
+    278.47,
+    276.15,
+    278.64,
+    278.86,
+    277.69,
+    275.4,
+    267.11,
+    276.68,
+    277.32,
+    278.12,
+    277.19,
+    277.93,
+    278.16,
+    257.52,
+    253.15,
+    278.1,
+    278.59
+   ],
+   "gpu_pcie_rx_mb": [
+    5309.2,
+    958.97,
+    17058.94,
+    14.64,
+    4.05,
+    3010.03,
+    8.39,
+    15.12,
+    15999.51,
+    15.07,
+    10772.8,
+    7160.47,
+    71.72,
+    850.39,
+    14.26,
+    26414.87,
+    26332.33,
+    24917.6,
+    18346.6,
+    6700.37,
+    20833.01,
+    48.73,
+    15.64,
+    11585.24,
+    10728.5,
+    4552.75,
+    22.55,
+    13.78,
+    18.55,
+    13.88,
+    3975.96,
+    21618.6,
+    14728.83,
+    12107.04,
+    17.45,
+    2833.51,
+    11.78,
+    14105.51,
+    21939.33,
+    26894.86,
+    0.33,
+    21545.12,
+    4351.66,
+    13.54,
+    21.36,
+    50.16,
+    22.7,
+    14.26,
+    24800.11,
+    17.12,
+    405.07,
+    24.65,
+    15145.21,
+    16473.29,
+    23109.39,
+    24054.19,
+    24291.28,
+    1098.68,
+    2094.22,
+    1823.0
+   ],
+   "cpu": [
+    12.7,
+    5.7,
+    13.2,
+    17.8,
+    11.5,
+    8.3,
+    10.6,
+    6.0,
+    6.7,
+    8.3,
+    8.0,
+    5.8,
+    5.5,
+    10.0,
+    7.2,
+    6.7,
+    11.2,
+    15.8,
+    7.1,
+    9.4,
+    8.7,
+    6.2,
+    6.3,
+    10.1,
+    6.6,
+    8.2,
+    8.6,
+    6.9,
+    6.0,
+    6.4,
+    9.0,
+    16.5,
+    8.5,
+    8.3,
+    8.3,
+    6.3,
+    6.1,
+    9.9,
+    6.0,
+    7.2,
+    6.1,
+    8.2,
+    11.3,
+    7.1,
+    7.3,
+    16.3,
+    11.9,
+    6.5,
+    8.4,
+    17.1,
+    6.1,
+    6.2,
+    6.3,
+    6.4,
+    6.8,
+    7.5,
+    7.9,
+    100.0,
+    100.0,
+    100.0
+   ],
+   "ram_used": [
+    63257399296,
+    63250329600,
+    63299825664,
+    63294885888,
+    63299862528,
+    63298265088,
+    63372304384,
+    63374991360,
+    63367077888,
+    63366172672,
+    63358062592,
+    63355240448,
+    63368228864,
+    63471902720,
+    63463698432,
+    63469293568,
+    63458623488,
+    63534329856,
+    63535923200,
+    63653580800,
+    63646789632,
+    63642984448,
+    63647580160,
+    63642767360,
+    63650861056,
+    63672475648,
+    63778967552,
+    63777734656,
+    63777472512,
+    63771406336,
+    63775264768,
+    63864942592,
+    63851782144,
+    63977308160,
+    63971295232,
+    63971143680,
+    63971721216,
+    63963795456,
+    63959093248,
+    63937032192,
+    64053448704,
+    64054136832,
+    63885062144,
+    63887245312,
+    63892475904,
+    63940841472,
+    63943122944,
+    63941619712,
+    64051613696,
+    64039284736,
+    64037093376,
+    64025649152,
+    64001216512,
+    63998476288,
+    63988797440,
+    63985102848,
+    64024535040,
+    64006193152,
+    63985057792,
+    63968800768
+   ],
+   "disk_read_mb": [
+    82.45,
+    0.0,
+    0.0,
+    104.98,
+    0.0,
+    0.15,
+    86.65,
+    0.0,
+    0.0,
+    49.16,
+    30.63,
+    0.0,
+    0.0,
+    66.33,
+    0.0,
+    0.0,
+    49.25,
+    0.0,
+    0.0,
+    19.87,
+    53.0,
+    0.0,
+    0.0,
+    65.63,
+    0.0,
+    0.0,
+    49.27,
+    11.94,
+    0.0,
+    0.0,
+    53.35,
+    0.0,
+    0.0,
+    40.85,
+    14.45,
+    0.0,
+    0.0,
+    52.4,
+    0.0,
+    0.0,
+    0.0,
+    51.78,
+    0.0,
+    0.0,
+    14.03,
+    110.69,
+    0.0,
+    0.0,
+    0.19,
+    189.89,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.15,
+    2.91,
+    2.72,
+    3.75
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    80.44,
+    97.1,
+    75.52
+   ]
+  },
+  "time": 1790714759.167411
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-128k-3.json

@@ -0,0 +1,829 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "128k",
+ "target_tokens": 129792,
+ "run": 3,
+ "run_id": "317fbfabc24a",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:45:59",
+ "prompt_tokens": 129913,
+ "completion_tokens": 256,
+ "ttft_s": 60.534,
+ "e2e_s": 64.056,
+ "api_prompt_tps": 2146.1,
+ "api_decode_tps": 72.4,
+ "engine_prompt_tps": 2168.8,
+ "engine_decode_tps": 72.0,
+ "log_lines": [
+  "strata serve: prompt 129913 tokens = 0 reused + 129913 read in 59901 ms (2168.8 tok/s), 256 generated in 3556 ms (72.0 tok/s), drafts accepted 148 of 200, 6 checkpoints",
+  "strata serve: decode expert cache hit rate: 95.6% (138618 hits / 145073 lookups)"
+ ],
+ "answer_head": "The last file is `include/strata/kernels/gdn.hpp`. It declares the non-projection parts of the gated delta-net (GDN) layer used by 36 of the model's 48 layers. It defines `GdnShapes` (S=128, h_k=16, h_v=48) and declares `gdn_step`, which performs one in-place step of the delta-rule recurrence: it de",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714759.725435,
+    "duration_s": 63.5,
+    "finish": "length",
+    "prompt_tokens": 129913,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 59900.8,
+    "decode_ms": 3556.3,
+    "decode_tok_s": 72.0,
+    "hit_rate": 0.956
+   },
+   {
+    "projection": null,
+    "time": 1790714695.4286683,
+    "duration_s": 63.2,
+    "finish": "length",
+    "prompt_tokens": 130360,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 59973.9,
+    "decode_ms": 3226.1,
+    "decode_tok_s": 79.4,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714636.537164,
+    "duration_s": 57.8,
+    "finish": "length",
+    "prompt_tokens": 119256,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 54577.1,
+    "decode_ms": 3229.8,
+    "decode_tok_s": 79.3,
+    "hit_rate": 0.955
+   },
+   {
+    "projection": null,
+    "time": 1790714507.7469585,
+    "duration_s": 17.4,
+    "finish": "length",
+    "prompt_tokens": 31602,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14617.7,
+    "decode_ms": 2785.2,
+    "decode_tok_s": 91.9,
+    "hit_rate": 0.965
+   },
+   {
+    "projection": null,
+    "time": 1790714489.1532733,
+    "duration_s": 17.9,
+    "finish": "length",
+    "prompt_tokens": 31582,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14618.7,
+    "decode_ms": 3270.7,
+    "decode_tok_s": 78.3,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714469.1007972,
+    "duration_s": 19.4,
+    "finish": "length",
+    "prompt_tokens": 34932,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 16553.9,
+    "decode_ms": 2826.7,
+    "decode_tok_s": 90.6,
+    "hit_rate": 0.941
+   },
+   {
+    "projection": null,
+    "time": 1790714463.1913157,
+    "duration_s": 5.2,
+    "finish": "length",
+    "prompt_tokens": 4201,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2365.6,
+    "decode_ms": 2868.4,
+    "decode_tok_s": 89.2,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714458.6299415,
+    "duration_s": 4.0,
+    "finish": "stop",
+    "prompt_tokens": 4200,
+    "reused": 0,
+    "output_tokens": 142,
+    "engine_generated": 142,
+    "prompt_ms": 2370.0,
+    "decode_ms": 1660.0,
+    "decode_tok_s": 85.5,
+    "hit_rate": 0.896
+   },
+   {
+    "projection": null,
+    "time": 1790714453.0791314,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 3881,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2223.8,
+    "decode_ms": 2790.5,
+    "decode_tok_s": 91.7,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790714448.5537117,
+    "duration_s": 4.0,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1305.1,
+    "decode_ms": 2690.8,
+    "decode_tok_s": 95.1,
+    "hit_rate": 0.972
+   },
+   {
+    "projection": null,
+    "time": 1790714443.8943996,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1149,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1297.6,
+    "decode_ms": 2847.5,
+    "decode_tok_s": 89.9,
+    "hit_rate": 0.971
+   },
+   {
+    "projection": null,
+    "time": 1790714439.2770011,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1092,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1351.4,
+    "decode_ms": 2752.6,
+    "decode_tok_s": 93.0,
+    "hit_rate": 0.954
+   }
+  ],
+  "requests_kept": 22,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 22,
+   "prompt_tokens": 606986,
+   "reused": 0,
+   "output_tokens": 4646,
+   "prompt_ms": 289887.2,
+   "decode_ms": 54787.29999999999
+  },
+  "hardware": {
+   "gpu_util": 95,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 49,
+   "gpu_power": 273.544,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.28515625,
+   "gpu_pcie_tx_mb": 0.4765625,
+   "cpu": 86.8,
+   "ram_used": 64157536256,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.8283858363947703,
+   "disk_write_mb": 0.0,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    99,
+    99,
+    99,
+    99,
+    100,
+    100,
+    99,
+    99,
+    100,
+    97,
+    99,
+    100,
+    98,
+    100,
+    100,
+    100,
+    100,
+    99,
+    100,
+    99,
+    99,
+    99,
+    100,
+    100,
+    100,
+    100,
+    100,
+    99,
+    100,
+    100,
+    100,
+    99,
+    100,
+    97,
+    100,
+    100,
+    100,
+    99,
+    99,
+    99,
+    99,
+    100,
+    100,
+    100,
+    97,
+    99,
+    99,
+    100,
+    100,
+    100,
+    99,
+    100,
+    99,
+    100,
+    99,
+    99,
+    85,
+    91,
+    96,
+    95
+   ],
+   "gpu_mem_used": [
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    49,
+    48,
+    49,
+    49,
+    49,
+    49,
+    48,
+    49,
+    49,
+    49,
+    49,
+    48,
+    48,
+    49,
+    48,
+    49,
+    49,
+    49,
+    49,
+    48,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    48,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49
+   ],
+   "gpu_power": [
+    277.4,
+    277.63,
+    276.55,
+    277.24,
+    277.6,
+    277.7,
+    267.41,
+    277.71,
+    277.73,
+    277.11,
+    277.47,
+    278.48,
+    265.62,
+    277.65,
+    278.51,
+    277.16,
+    277.8,
+    278.42,
+    276.76,
+    264.34,
+    277.83,
+    278.79,
+    277.39,
+    278.33,
+    279.04,
+    277.09,
+    266.43,
+    277.15,
+    279.09,
+    277.12,
+    278.21,
+    279.11,
+    276.87,
+    266.5,
+    277.59,
+    276.92,
+    277.76,
+    277.37,
+    277.95,
+    278.05,
+    266.15,
+    281.14,
+    276.88,
+    277.27,
+    277.03,
+    278.78,
+    277.01,
+    276.66,
+    264.08,
+    276.78,
+    278.41,
+    277.85,
+    276.09,
+    278.51,
+    279.06,
+    278.13,
+    239.73,
+    262.71,
+    270.73,
+    273.54
+   ],
+   "gpu_pcie_rx_mb": [
+    11.44,
+    11737.44,
+    1061.11,
+    1303.05,
+    10.78,
+    20755.48,
+    892.69,
+    12802.27,
+    17290.83,
+    13.02,
+    669.76,
+    9859.75,
+    15.88,
+    26970.96,
+    27538.78,
+    8595.9,
+    10.82,
+    22491.22,
+    20098.16,
+    7703.26,
+    2655.31,
+    1810.74,
+    21.84,
+    656.7,
+    7.82,
+    17247.68,
+    12383.65,
+    14706.95,
+    18027.4,
+    14.64,
+    51.02,
+    23.94,
+    21140.67,
+    19852.11,
+    18272.88,
+    11345.43,
+    10748.2,
+    23.08,
+    3973.67,
+    14.35,
+    15.45,
+    12245.37,
+    13872.91,
+    2994.2,
+    10277.84,
+    51.35,
+    26547.62,
+    4842.76,
+    13126.71,
+    6178.52,
+    14.73,
+    12025.83,
+    22118.66,
+    18928.05,
+    18.36,
+    86.12,
+    284.24,
+    9240.67,
+    7805.97,
+    0.29
+   ],
+   "cpu": [
+    7.6,
+    6.7,
+    9.9,
+    8.5,
+    6.7,
+    6.2,
+    11.3,
+    5.7,
+    6.1,
+    11.0,
+    7.2,
+    6.5,
+    20.9,
+    10.8,
+    5.8,
+    8.9,
+    8.5,
+    6.7,
+    6.4,
+    11.1,
+    6.7,
+    5.6,
+    11.4,
+    8.9,
+    6.1,
+    6.0,
+    17.0,
+    11.4,
+    6.6,
+    10.0,
+    7.3,
+    6.2,
+    6.6,
+    10.7,
+    6.4,
+    6.2,
+    6.7,
+    9.0,
+    6.2,
+    7.0,
+    9.7,
+    17.6,
+    7.3,
+    7.5,
+    10.4,
+    6.7,
+    6.4,
+    6.7,
+    8.5,
+    6.3,
+    5.8,
+    8.2,
+    6.7,
+    6.2,
+    6.0,
+    13.2,
+    64.5,
+    99.9,
+    99.0,
+    86.8
+   ],
+   "ram_used": [
+    63275466752,
+    63276769280,
+    63271755776,
+    63272300544,
+    63263879168,
+    63359676416,
+    63357374464,
+    63357542400,
+    63358373888,
+    63359791104,
+    63361323008,
+    63363457024,
+    63494107136,
+    63494156288,
+    63489257472,
+    63490707456,
+    63488667648,
+    63458295808,
+    63462985728,
+    63580164096,
+    63581151232,
+    63584616448,
+    63609655296,
+    63608074240,
+    63610007552,
+    63607758848,
+    63749599232,
+    63745466368,
+    63718805504,
+    63722086400,
+    63728562176,
+    63731232768,
+    63729074176,
+    63842201600,
+    63847124992,
+    63850708992,
+    63853719552,
+    63846064128,
+    63846113280,
+    63848321024,
+    63970734080,
+    63991717888,
+    63989817344,
+    63985319936,
+    63983939584,
+    63975882752,
+    63976255488,
+    63973568512,
+    64091373568,
+    64083685376,
+    64078811136,
+    64100171776,
+    64087904256,
+    64069447680,
+    64067096576,
+    64095731712,
+    64224346112,
+    64185745408,
+    64170352640,
+    64157536256
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    71.01,
+    35.58,
+    0.0,
+    2.93,
+    89.18,
+    0.0,
+    0.0,
+    92.41,
+    0.0,
+    0.0,
+    76.71,
+    3.39,
+    0.0,
+    12.48,
+    48.82,
+    0.0,
+    0.0,
+    92.22,
+    0.0,
+    0.0,
+    44.7,
+    41.48,
+    0.0,
+    0.0,
+    83.51,
+    0.0,
+    0.0,
+    52.65,
+    20.67,
+    0.0,
+    0.0,
+    77.14,
+    0.0,
+    0.0,
+    21.58,
+    52.7,
+    0.0,
+    0.0,
+    74.56,
+    0.0,
+    0.0,
+    0.0,
+    65.27,
+    0.0,
+    0.0,
+    0.0,
+    48.73,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.89,
+    2.52,
+    3.25,
+    2.83
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    61.23,
+    76.01,
+    95.36,
+    0.0
+   ]
+  },
+  "time": 1790714823.7293081
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-1k-1.json

@@ -0,0 +1,815 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "1k",
+ "target_tokens": 1024,
+ "run": 1,
+ "run_id": "56b3dec5f002",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:40:39",
+ "prompt_tokens": 1092,
+ "completion_tokens": 256,
+ "ttft_s": 1.378,
+ "e2e_s": 4.115,
+ "api_prompt_tps": 792.3,
+ "api_decode_tps": 93.2,
+ "engine_prompt_tps": 808.1,
+ "engine_decode_tps": 93.0,
+ "log_lines": [
+  "strata serve: prompt 1092 tokens = 0 reused + 1092 read in 1351 ms (808.1 tok/s), 256 generated in 2753 ms (93.0 tok/s), drafts accepted 159 of 210, 1 checkpoints",
+  "strata serve: decode expert cache hit rate: 95.4% (137036 hits / 143638 lookups)"
+ ],
+ "answer_head": "Based on the content of `CMakeLists.txt`, here is an explanation of its function and a proposed improvement.\n\n### Explanation of `CMakeLists.txt`\n\nThis file serves as the top-level build configuration for the **Strata Engine** (version 0.1.26). Its primary responsibilities are:\n\n1.  **Project Defini",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714439.2770011,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1092,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1351.4,
+    "decode_ms": 2752.6,
+    "decode_tok_s": 93.0,
+    "hit_rate": 0.954
+   },
+   {
+    "projection": null,
+    "time": 1790714377.2599747,
+    "duration_s": 17.6,
+    "finish": "length",
+    "prompt_tokens": 31605,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14608.5,
+    "decode_ms": 3015.6,
+    "decode_tok_s": 84.9,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714359.1307795,
+    "duration_s": 17.5,
+    "finish": "length",
+    "prompt_tokens": 31589,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14609.7,
+    "decode_ms": 2833.5,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714338.0043592,
+    "duration_s": 20.5,
+    "finish": "length",
+    "prompt_tokens": 34942,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 17033.3,
+    "decode_ms": 3435.7,
+    "decode_tok_s": 74.5,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714334.3235514,
+    "duration_s": 3.0,
+    "finish": "stop",
+    "prompt_tokens": 4199,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 2369.7,
+    "decode_ms": 626.3,
+    "decode_tok_s": 78.2,
+    "hit_rate": 0.934
+   },
+   {
+    "projection": null,
+    "time": 1790714328.2723713,
+    "duration_s": 5.5,
+    "finish": "length",
+    "prompt_tokens": 4214,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2436.3,
+    "decode_ms": 3084.4,
+    "decode_tok_s": 83.0,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714324.3220718,
+    "duration_s": 3.4,
+    "finish": "stop",
+    "prompt_tokens": 3736,
+    "reused": 0,
+    "output_tokens": 56,
+    "engine_generated": 56,
+    "prompt_ms": 2700.1,
+    "decode_ms": 719.2,
+    "decode_tok_s": 77.9,
+    "hit_rate": 0.897
+   },
+   {
+    "projection": null,
+    "time": 1790714293.0977688,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1288.1,
+    "decode_ms": 2834.1,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714290.8242896,
+    "duration_s": 1.8,
+    "finish": "stop",
+    "prompt_tokens": 1147,
+    "reused": 0,
+    "output_tokens": 40,
+    "engine_generated": 40,
+    "prompt_ms": 1304.0,
+    "decode_ms": 456.3,
+    "decode_tok_s": 87.7,
+    "hit_rate": 0.916
+   },
+   {
+    "projection": null,
+    "time": 1790714285.3293037,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 1094,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1899.7,
+    "decode_ms": 3080.2,
+    "decode_tok_s": 83.1,
+    "hit_rate": 0.946
+   },
+   {
+    "projection": null,
+    "time": 1790713900.001974,
+    "duration_s": 0.7,
+    "finish": "stop",
+    "prompt_tokens": 22,
+    "reused": 0,
+    "output_tokens": 7,
+    "engine_generated": 7,
+    "prompt_ms": 482.2,
+    "decode_ms": 197.4,
+    "decode_tok_s": 35.5,
+    "hit_rate": 0.707
+   }
+  ],
+  "requests_kept": 11,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 11,
+   "prompt_tokens": 114775,
+   "reused": 0,
+   "output_tokens": 1944,
+   "prompt_ms": 60083.00000000001,
+   "decode_ms": 23035.299999999996
+  },
+  "hardware": {
+   "gpu_util": 94,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 45,
+   "gpu_power": 276.526,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.3330078125,
+   "gpu_pcie_tx_mb": 0.4765625,
+   "cpu": 87.0,
+   "ram_used": 63534440448,
+   "ram_total": 177175990272,
+   "disk_read_mb": 3.2598935177561885,
+   "disk_write_mb": 0.003747004043397918,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    99,
+    99,
+    99,
+    42,
+    99,
+    99,
+    100,
+    99,
+    99,
+    99,
+    94,
+    99,
+    95,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    100,
+    98,
+    94,
+    94
+   ],
+   "gpu_mem_used": [
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    44,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    42,
+    42,
+    41,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    39,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    45,
+    45,
+    45,
+    45
+   ],
+   "gpu_power": [
+    277.8,
+    276.98,
+    276.99,
+    264.47,
+    274.0,
+    276.95,
+    276.96,
+    277.85,
+    278.23,
+    277.83,
+    242.91,
+    278.73,
+    277.1,
+    247.26,
+    164.95,
+    142.09,
+    130.43,
+    119.16,
+    119.05,
+    119.02,
+    118.92,
+    118.89,
+    118.85,
+    118.74,
+    118.81,
+    118.95,
+    118.87,
+    118.88,
+    119.01,
+    118.97,
+    113.29,
+    37.07,
+    35.2,
+    35.1,
+    35.04,
+    34.98,
+    34.86,
+    34.88,
+    34.97,
+    34.79,
+    34.76,
+    34.94,
+    34.9,
+    34.89,
+    35.0,
+    34.87,
+    34.82,
+    34.93,
+    34.86,
+    34.92,
+    34.99,
+    34.83,
+    34.88,
+    34.97,
+    34.89,
+    34.96,
+    172.41,
+    265.41,
+    275.33,
+    276.53
+   ],
+   "gpu_pcie_rx_mb": [
+    20099.78,
+    18385.74,
+    52.36,
+    17363.88,
+    9102.54,
+    10.44,
+    87.17,
+    12174.99,
+    8.39,
+    78.63,
+    439.12,
+    1555.01,
+    511.84,
+    0.33,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.29,
+    0.29,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.57,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    9.97,
+    22329.52,
+    3689.24,
+    4264.07,
+    0.33
+   ],
+   "cpu": [
+    6.7,
+    5.9,
+    12.5,
+    10.7,
+    6.6,
+    10.7,
+    6.2,
+    7.2,
+    6.5,
+    5.9,
+    52.8,
+    100.0,
+    100.0,
+    46.0,
+    2.5,
+    9.7,
+    2.8,
+    11.4,
+    13.4,
+    2.8,
+    1.5,
+    1.3,
+    2.8,
+    1.6,
+    1.6,
+    1.6,
+    2.2,
+    1.1,
+    5.2,
+    2.3,
+    1.3,
+    11.2,
+    3.6,
+    1.3,
+    11.5,
+    1.1,
+    1.9,
+    1.2,
+    1.3,
+    1.3,
+    1.0,
+    0.6,
+    2.3,
+    0.8,
+    1.2,
+    6.3,
+    8.9,
+    0.6,
+    1.9,
+    2.2,
+    2.1,
+    1.1,
+    2.2,
+    4.7,
+    1.2,
+    6.8,
+    7.3,
+    83.2,
+    99.8,
+    87.0
+   ],
+   "ram_used": [
+    63341527040,
+    63336820736,
+    63366373376,
+    63501127680,
+    63474081792,
+    63398842368,
+    63397171200,
+    63396274176,
+    63387021312,
+    63380054016,
+    63501307904,
+    63504547840,
+    63499546624,
+    63490957312,
+    63502053376,
+    63600914432,
+    63612030976,
+    63779962880,
+    63785050112,
+    63794016256,
+    63795236864,
+    63794536448,
+    63786823680,
+    63781208064,
+    63760637952,
+    63756046336,
+    63757750272,
+    63751917568,
+    63784116224,
+    63784988672,
+    63783927808,
+    63781187584,
+    63772962816,
+    63781842944,
+    63808417792,
+    63797518336,
+    63793590272,
+    63788277760,
+    63788392448,
+    63791640576,
+    63791734784,
+    63793213440,
+    63793565696,
+    63794352128,
+    63793430528,
+    63799361536,
+    63590793216,
+    63591645184,
+    63583899648,
+    63571738624,
+    63579156480,
+    63572910080,
+    63581016064,
+    63625879552,
+    63626788864,
+    63427158016,
+    63429017600,
+    63532883968,
+    63535853568,
+    63534440448
+   ],
+   "disk_read_mb": [
+    0.2,
+    0.0,
+    0.0,
+    0.22,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    1.41,
+    2.29,
+    3.94,
+    1.13,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.1,
+    0.14,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.8,
+    3.91,
+    3.26
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    91.56,
+    92.69,
+    83.62,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    103.98,
+    98.22,
+    0.0
+   ]
+  },
+  "time": 1790714443.884817
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-1k-2.json

@@ -0,0 +1,829 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "1k",
+ "target_tokens": 1024,
+ "run": 2,
+ "run_id": "5da89525cd83",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:40:43",
+ "prompt_tokens": 1149,
+ "completion_tokens": 256,
+ "ttft_s": 1.323,
+ "e2e_s": 4.156,
+ "api_prompt_tps": 868.4,
+ "api_decode_tps": 90.0,
+ "engine_prompt_tps": 885.5,
+ "engine_decode_tps": 89.9,
+ "log_lines": [
+  "strata serve: prompt 1149 tokens = 0 reused + 1149 read in 1298 ms (885.5 tok/s), 256 generated in 2847 ms (89.9 tok/s), drafts accepted 156 of 215, 1 checkpoints",
+  "strata serve: decode expert cache hit rate: 97.1% (144603 hits / 148933 lookups)"
+ ],
+ "answer_head": "Based on the content provided, here is an explanation of the `CMakeLists.txt` file and a proposed improvement.\n\n### Explanation of `CMakeLists.txt`\n\nThis file is the top-level build configuration for the **Strata Engine** (specifically Phase 1, Step 1). Its primary responsibilities are:\n\n1.  **Proje",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714443.8943996,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1149,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1297.6,
+    "decode_ms": 2847.5,
+    "decode_tok_s": 89.9,
+    "hit_rate": 0.971
+   },
+   {
+    "projection": null,
+    "time": 1790714439.2770011,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1092,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1351.4,
+    "decode_ms": 2752.6,
+    "decode_tok_s": 93.0,
+    "hit_rate": 0.954
+   },
+   {
+    "projection": null,
+    "time": 1790714377.2599747,
+    "duration_s": 17.6,
+    "finish": "length",
+    "prompt_tokens": 31605,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14608.5,
+    "decode_ms": 3015.6,
+    "decode_tok_s": 84.9,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714359.1307795,
+    "duration_s": 17.5,
+    "finish": "length",
+    "prompt_tokens": 31589,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14609.7,
+    "decode_ms": 2833.5,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714338.0043592,
+    "duration_s": 20.5,
+    "finish": "length",
+    "prompt_tokens": 34942,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 17033.3,
+    "decode_ms": 3435.7,
+    "decode_tok_s": 74.5,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714334.3235514,
+    "duration_s": 3.0,
+    "finish": "stop",
+    "prompt_tokens": 4199,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 2369.7,
+    "decode_ms": 626.3,
+    "decode_tok_s": 78.2,
+    "hit_rate": 0.934
+   },
+   {
+    "projection": null,
+    "time": 1790714328.2723713,
+    "duration_s": 5.5,
+    "finish": "length",
+    "prompt_tokens": 4214,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2436.3,
+    "decode_ms": 3084.4,
+    "decode_tok_s": 83.0,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714324.3220718,
+    "duration_s": 3.4,
+    "finish": "stop",
+    "prompt_tokens": 3736,
+    "reused": 0,
+    "output_tokens": 56,
+    "engine_generated": 56,
+    "prompt_ms": 2700.1,
+    "decode_ms": 719.2,
+    "decode_tok_s": 77.9,
+    "hit_rate": 0.897
+   },
+   {
+    "projection": null,
+    "time": 1790714293.0977688,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1288.1,
+    "decode_ms": 2834.1,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714290.8242896,
+    "duration_s": 1.8,
+    "finish": "stop",
+    "prompt_tokens": 1147,
+    "reused": 0,
+    "output_tokens": 40,
+    "engine_generated": 40,
+    "prompt_ms": 1304.0,
+    "decode_ms": 456.3,
+    "decode_tok_s": 87.7,
+    "hit_rate": 0.916
+   },
+   {
+    "projection": null,
+    "time": 1790714285.3293037,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 1094,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1899.7,
+    "decode_ms": 3080.2,
+    "decode_tok_s": 83.1,
+    "hit_rate": 0.946
+   },
+   {
+    "projection": null,
+    "time": 1790713900.001974,
+    "duration_s": 0.7,
+    "finish": "stop",
+    "prompt_tokens": 22,
+    "reused": 0,
+    "output_tokens": 7,
+    "engine_generated": 7,
+    "prompt_ms": 482.2,
+    "decode_ms": 197.4,
+    "decode_tok_s": 35.5,
+    "hit_rate": 0.707
+   }
+  ],
+  "requests_kept": 12,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 12,
+   "prompt_tokens": 115924,
+   "reused": 0,
+   "output_tokens": 2200,
+   "prompt_ms": 61380.600000000006,
+   "decode_ms": 25882.799999999996
+  },
+  "hardware": {
+   "gpu_util": 95,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 45,
+   "gpu_power": 273.597,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 1249.6943359375,
+   "gpu_pcie_tx_mb": 212.6689453125,
+   "cpu": 97.9,
+   "ram_used": 63536766976,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.8181744035205307,
+   "disk_write_mb": 0.014930725316665065,
+   "tok_s": 76.9743768862725,
+   "tok_s_mean": 90.24600065341676
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    99,
+    99,
+    100,
+    99,
+    99,
+    99,
+    94,
+    99,
+    95,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    100,
+    98,
+    94,
+    94,
+    100,
+    21,
+    98,
+    95
+   ],
+   "gpu_mem_used": [
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    45,
+    42,
+    42,
+    41,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    39,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    45,
+    45,
+    45,
+    45,
+    45,
+    43,
+    45,
+    45
+   ],
+   "gpu_power": [
+    274.0,
+    276.95,
+    276.96,
+    277.85,
+    278.23,
+    277.83,
+    242.91,
+    278.73,
+    277.1,
+    247.26,
+    164.95,
+    142.09,
+    130.43,
+    119.16,
+    119.05,
+    119.02,
+    118.92,
+    118.89,
+    118.85,
+    118.74,
+    118.81,
+    118.95,
+    118.87,
+    118.88,
+    119.01,
+    118.97,
+    113.29,
+    37.07,
+    35.2,
+    35.1,
+    35.04,
+    34.98,
+    34.86,
+    34.88,
+    34.97,
+    34.79,
+    34.76,
+    34.94,
+    34.9,
+    34.89,
+    35.0,
+    34.87,
+    34.82,
+    34.93,
+    34.86,
+    34.92,
+    34.99,
+    34.83,
+    34.88,
+    34.97,
+    34.89,
+    34.96,
+    172.41,
+    265.41,
+    275.33,
+    276.53,
+    216.14,
+    263.1,
+    264.26,
+    273.6
+   ],
+   "gpu_pcie_rx_mb": [
+    9102.54,
+    10.44,
+    87.17,
+    12174.99,
+    8.39,
+    78.63,
+    439.12,
+    1555.01,
+    511.84,
+    0.33,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.29,
+    0.29,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.57,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    9.97,
+    22329.52,
+    3689.24,
+    4264.07,
+    0.33,
+    14955.09,
+    784.16,
+    3310.82,
+    1249.69
+   ],
+   "cpu": [
+    6.6,
+    10.7,
+    6.2,
+    7.2,
+    6.5,
+    5.9,
+    52.8,
+    100.0,
+    100.0,
+    46.0,
+    2.5,
+    9.7,
+    2.8,
+    11.4,
+    13.4,
+    2.8,
+    1.5,
+    1.3,
+    2.8,
+    1.6,
+    1.6,
+    1.6,
+    2.2,
+    1.1,
+    5.2,
+    2.3,
+    1.3,
+    11.2,
+    3.6,
+    1.3,
+    11.5,
+    1.1,
+    1.9,
+    1.2,
+    1.3,
+    1.3,
+    1.0,
+    0.6,
+    2.3,
+    0.8,
+    1.2,
+    6.3,
+    8.9,
+    0.6,
+    1.9,
+    2.2,
+    2.1,
+    1.1,
+    2.2,
+    4.7,
+    1.2,
+    6.8,
+    7.3,
+    83.2,
+    99.8,
+    87.0,
+    14.8,
+    49.9,
+    100.0,
+    97.9
+   ],
+   "ram_used": [
+    63474081792,
+    63398842368,
+    63397171200,
+    63396274176,
+    63387021312,
+    63380054016,
+    63501307904,
+    63504547840,
+    63499546624,
+    63490957312,
+    63502053376,
+    63600914432,
+    63612030976,
+    63779962880,
+    63785050112,
+    63794016256,
+    63795236864,
+    63794536448,
+    63786823680,
+    63781208064,
+    63760637952,
+    63756046336,
+    63757750272,
+    63751917568,
+    63784116224,
+    63784988672,
+    63783927808,
+    63781187584,
+    63772962816,
+    63781842944,
+    63808417792,
+    63797518336,
+    63793590272,
+    63788277760,
+    63788392448,
+    63791640576,
+    63791734784,
+    63793213440,
+    63793565696,
+    63794352128,
+    63793430528,
+    63799361536,
+    63590793216,
+    63591645184,
+    63583899648,
+    63571738624,
+    63579156480,
+    63572910080,
+    63581016064,
+    63625879552,
+    63626788864,
+    63427158016,
+    63429017600,
+    63532883968,
+    63535853568,
+    63534440448,
+    63490953216,
+    63544176640,
+    63529631744,
+    63536766976
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    1.41,
+    2.29,
+    3.94,
+    1.13,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.1,
+    0.14,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.8,
+    3.91,
+    3.26,
+    0.6,
+    0.0,
+    1.08,
+    2.82
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    91.56,
+    92.69,
+    83.62,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    103.98,
+    98.22,
+    0.0,
+    0.0,
+    112.83,
+    83.57,
+    76.97
+   ]
+  },
+  "time": 1790714448.544591
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-1k-3.json

@@ -0,0 +1,829 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "1k",
+ "target_tokens": 1024,
+ "run": 3,
+ "run_id": "3ad6050d1d4c",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:40:48",
+ "prompt_tokens": 1135,
+ "completion_tokens": 256,
+ "ttft_s": 1.33,
+ "e2e_s": 4.007,
+ "api_prompt_tps": 853.1,
+ "api_decode_tps": 95.3,
+ "engine_prompt_tps": 869.7,
+ "engine_decode_tps": 95.1,
+ "log_lines": [
+  "strata serve: prompt 1135 tokens = 0 reused + 1135 read in 1305 ms (869.7 tok/s), 256 generated in 2691 ms (95.1 tok/s), drafts accepted 159 of 217, 1 checkpoints",
+  "strata serve: decode expert cache hit rate: 97.2% (145229 hits / 149472 lookups)"
+ ],
+ "answer_head": "The `CMakeLists.txt` file defines the top-level build configuration for the **Strata Engine** project. Its primary responsibilities include:\n\n1.  **Project Definition**: Sets the project name (`strata`), version (`0.1.26`), and language standard (C++20).\n2.  **Target Management**: Conditionally adds",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714448.5537117,
+    "duration_s": 4.0,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1305.1,
+    "decode_ms": 2690.8,
+    "decode_tok_s": 95.1,
+    "hit_rate": 0.972
+   },
+   {
+    "projection": null,
+    "time": 1790714443.8943996,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1149,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1297.6,
+    "decode_ms": 2847.5,
+    "decode_tok_s": 89.9,
+    "hit_rate": 0.971
+   },
+   {
+    "projection": null,
+    "time": 1790714439.2770011,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1092,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1351.4,
+    "decode_ms": 2752.6,
+    "decode_tok_s": 93.0,
+    "hit_rate": 0.954
+   },
+   {
+    "projection": null,
+    "time": 1790714377.2599747,
+    "duration_s": 17.6,
+    "finish": "length",
+    "prompt_tokens": 31605,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14608.5,
+    "decode_ms": 3015.6,
+    "decode_tok_s": 84.9,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714359.1307795,
+    "duration_s": 17.5,
+    "finish": "length",
+    "prompt_tokens": 31589,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14609.7,
+    "decode_ms": 2833.5,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714338.0043592,
+    "duration_s": 20.5,
+    "finish": "length",
+    "prompt_tokens": 34942,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 17033.3,
+    "decode_ms": 3435.7,
+    "decode_tok_s": 74.5,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714334.3235514,
+    "duration_s": 3.0,
+    "finish": "stop",
+    "prompt_tokens": 4199,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 2369.7,
+    "decode_ms": 626.3,
+    "decode_tok_s": 78.2,
+    "hit_rate": 0.934
+   },
+   {
+    "projection": null,
+    "time": 1790714328.2723713,
+    "duration_s": 5.5,
+    "finish": "length",
+    "prompt_tokens": 4214,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2436.3,
+    "decode_ms": 3084.4,
+    "decode_tok_s": 83.0,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714324.3220718,
+    "duration_s": 3.4,
+    "finish": "stop",
+    "prompt_tokens": 3736,
+    "reused": 0,
+    "output_tokens": 56,
+    "engine_generated": 56,
+    "prompt_ms": 2700.1,
+    "decode_ms": 719.2,
+    "decode_tok_s": 77.9,
+    "hit_rate": 0.897
+   },
+   {
+    "projection": null,
+    "time": 1790714293.0977688,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1288.1,
+    "decode_ms": 2834.1,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714290.8242896,
+    "duration_s": 1.8,
+    "finish": "stop",
+    "prompt_tokens": 1147,
+    "reused": 0,
+    "output_tokens": 40,
+    "engine_generated": 40,
+    "prompt_ms": 1304.0,
+    "decode_ms": 456.3,
+    "decode_tok_s": 87.7,
+    "hit_rate": 0.916
+   },
+   {
+    "projection": null,
+    "time": 1790714285.3293037,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 1094,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1899.7,
+    "decode_ms": 3080.2,
+    "decode_tok_s": 83.1,
+    "hit_rate": 0.946
+   }
+  ],
+  "requests_kept": 13,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 13,
+   "prompt_tokens": 117059,
+   "reused": 0,
+   "output_tokens": 2456,
+   "prompt_ms": 62685.700000000004,
+   "decode_ms": 28573.599999999995
+  },
+  "hardware": {
+   "gpu_util": 21,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 43,
+   "gpu_power": 268.452,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.28515625,
+   "gpu_pcie_tx_mb": 0.4287109375,
+   "cpu": 63.3,
+   "ram_used": 63492075520,
+   "ram_total": 177175990272,
+   "disk_read_mb": 1.6601509757982635,
+   "disk_write_mb": 0.022485114796364742,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    99,
+    94,
+    99,
+    95,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    100,
+    98,
+    94,
+    94,
+    100,
+    21,
+    98,
+    95,
+    53,
+    100,
+    94,
+    96,
+    21
+   ],
+   "gpu_mem_used": [
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    45,
+    45,
+    45,
+    45,
+    42,
+    42,
+    41,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    39,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    45,
+    45,
+    45,
+    45,
+    45,
+    43,
+    45,
+    45,
+    44,
+    45,
+    45,
+    45,
+    43
+   ],
+   "gpu_power": [
+    277.83,
+    242.91,
+    278.73,
+    277.1,
+    247.26,
+    164.95,
+    142.09,
+    130.43,
+    119.16,
+    119.05,
+    119.02,
+    118.92,
+    118.89,
+    118.85,
+    118.74,
+    118.81,
+    118.95,
+    118.87,
+    118.88,
+    119.01,
+    118.97,
+    113.29,
+    37.07,
+    35.2,
+    35.1,
+    35.04,
+    34.98,
+    34.86,
+    34.88,
+    34.97,
+    34.79,
+    34.76,
+    34.94,
+    34.9,
+    34.89,
+    35.0,
+    34.87,
+    34.82,
+    34.93,
+    34.86,
+    34.92,
+    34.99,
+    34.83,
+    34.88,
+    34.97,
+    34.89,
+    34.96,
+    172.41,
+    265.41,
+    275.33,
+    276.53,
+    216.14,
+    263.1,
+    264.26,
+    273.6,
+    211.21,
+    274.12,
+    265.58,
+    276.31,
+    268.45
+   ],
+   "gpu_pcie_rx_mb": [
+    78.63,
+    439.12,
+    1555.01,
+    511.84,
+    0.33,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.29,
+    0.29,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.57,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    9.97,
+    22329.52,
+    3689.24,
+    4264.07,
+    0.33,
+    14955.09,
+    784.16,
+    3310.82,
+    1249.69,
+    15874.15,
+    18357.04,
+    4099.13,
+    485.18,
+    0.29
+   ],
+   "cpu": [
+    5.9,
+    52.8,
+    100.0,
+    100.0,
+    46.0,
+    2.5,
+    9.7,
+    2.8,
+    11.4,
+    13.4,
+    2.8,
+    1.5,
+    1.3,
+    2.8,
+    1.6,
+    1.6,
+    1.6,
+    2.2,
+    1.1,
+    5.2,
+    2.3,
+    1.3,
+    11.2,
+    3.6,
+    1.3,
+    11.5,
+    1.1,
+    1.9,
+    1.2,
+    1.3,
+    1.3,
+    1.0,
+    0.6,
+    2.3,
+    0.8,
+    1.2,
+    6.3,
+    8.9,
+    0.6,
+    1.9,
+    2.2,
+    2.1,
+    1.1,
+    2.2,
+    4.7,
+    1.2,
+    6.8,
+    7.3,
+    83.2,
+    99.8,
+    87.0,
+    14.8,
+    49.9,
+    100.0,
+    97.9,
+    33.3,
+    7.4,
+    99.9,
+    100.0,
+    63.3
+   ],
+   "ram_used": [
+    63380054016,
+    63501307904,
+    63504547840,
+    63499546624,
+    63490957312,
+    63502053376,
+    63600914432,
+    63612030976,
+    63779962880,
+    63785050112,
+    63794016256,
+    63795236864,
+    63794536448,
+    63786823680,
+    63781208064,
+    63760637952,
+    63756046336,
+    63757750272,
+    63751917568,
+    63784116224,
+    63784988672,
+    63783927808,
+    63781187584,
+    63772962816,
+    63781842944,
+    63808417792,
+    63797518336,
+    63793590272,
+    63788277760,
+    63788392448,
+    63791640576,
+    63791734784,
+    63793213440,
+    63793565696,
+    63794352128,
+    63793430528,
+    63799361536,
+    63590793216,
+    63591645184,
+    63583899648,
+    63571738624,
+    63579156480,
+    63572910080,
+    63581016064,
+    63625879552,
+    63626788864,
+    63427158016,
+    63429017600,
+    63532883968,
+    63535853568,
+    63534440448,
+    63490953216,
+    63544176640,
+    63529631744,
+    63536766976,
+    63443234816,
+    63502606336,
+    63508336640,
+    63509970944,
+    63492075520
+   ],
+   "disk_read_mb": [
+    0.0,
+    1.41,
+    2.29,
+    3.94,
+    1.13,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.1,
+    0.14,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.8,
+    3.91,
+    3.26,
+    0.6,
+    0.0,
+    1.08,
+    2.82,
+    1.34,
+    0.0,
+    1.55,
+    2.4,
+    1.66
+   ],
+   "tok_s": [
+    0.0,
+    91.56,
+    92.69,
+    83.62,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    103.98,
+    98.22,
+    0.0,
+    0.0,
+    112.83,
+    83.57,
+    76.97,
+    0.0,
+    0.0,
+    97.69,
+    87.72,
+    0.0
+   ]
+  },
+  "time": 1790714453.0546403
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-262k-1.json

@@ -0,0 +1,675 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "262k",
+ "target_tokens": 260864,
+ "run": 1,
+ "run_id": "459ef0e5c155",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:52:28",
+ "prompt_tokens": 232183,
+ "completion_tokens": 256,
+ "ttft_s": 131.32,
+ "e2e_s": 135.339,
+ "api_prompt_tps": 1768.1,
+ "api_decode_tps": 63.4,
+ "engine_prompt_tps": 1784.6,
+ "engine_decode_tps": 63.0,
+ "log_lines": [
+  "strata serve: prompt 232183 tokens = 0 reused + 232183 read in 130101 ms (1784.6 tok/s), 256 generated in 4061 ms (63.0 tok/s), drafts accepted 150 of 240, 6 checkpoints",
+  "strata serve: decode expert cache hit rate: 93.7% (149829 hits / 159942 lookups)"
+ ],
+ "answer_head": "The last file is `serve/server.py` (the listing is truncated at the very end, in the `main()` function where it assigns `svc.gpu_index`).\n\n## What it does\n\nIt is the HTTP API layer of Strata: a pure-Python bridge between OpenAI/Anthropic-style clients and the C++ engine. Concretely:\n\n- **Engine adap",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 262144,
+   "images": false,
+   "version": "0.1.26",
+   "context": 262144,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10451,
+   "expert_cache_mib": 17355,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 385,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790715149.1813676,
+    "duration_s": 134.2,
+    "finish": "length",
+    "prompt_tokens": 232183,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 130101.1,
+    "decode_ms": 4061.2,
+    "decode_tok_s": 63.0,
+    "hit_rate": 0.937
+   }
+  ],
+  "requests_kept": 1,
+  "totals": {
+   "since": 1790715131.4670517,
+   "requests": 1,
+   "prompt_tokens": 232183,
+   "reused": 0,
+   "output_tokens": 256,
+   "prompt_ms": 130101.1,
+   "decode_ms": 4061.2
+  },
+  "hardware": {
+   "gpu_util": 93,
+   "gpu_mem_used": 25512837120,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 50,
+   "gpu_power": 278.227,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 1220.75,
+   "gpu_pcie_tx_mb": 478.41015625,
+   "cpu": 100.0,
+   "ram_used": 65694810112,
+   "ram_total": 177175990272,
+   "disk_read_mb": 3.754966568441275,
+   "disk_write_mb": 0.0,
+   "tok_s": 74.58045953935934,
+   "tok_s_mean": 62.37130450720771
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    99,
+    100,
+    99,
+    100,
+    99,
+    100,
+    98,
+    100,
+    100,
+    100,
+    100,
+    100,
+    99,
+    99,
+    100,
+    99,
+    99,
+    100,
+    99,
+    100,
+    100,
+    98,
+    100,
+    100,
+    100,
+    97,
+    100,
+    100,
+    99,
+    99,
+    98,
+    97,
+    100,
+    100,
+    100,
+    99,
+    100,
+    100,
+    99,
+    99,
+    99,
+    100,
+    100,
+    100,
+    100,
+    100,
+    99,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    3,
+    100,
+    100,
+    93,
+    94,
+    96,
+    93
+   ],
+   "gpu_mem_used": [
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25412173824,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120
+   ],
+   "gpu_temp": [
+    49,
+    49,
+    49,
+    50,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    50,
+    49,
+    49,
+    49,
+    49,
+    50,
+    50,
+    49,
+    49,
+    50,
+    49,
+    50,
+    50,
+    50,
+    50,
+    50,
+    50,
+    50,
+    50,
+    50,
+    49,
+    50,
+    50,
+    50,
+    50,
+    50,
+    50,
+    50,
+    50,
+    49,
+    50,
+    50,
+    50,
+    51,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    50,
+    50,
+    47,
+    50,
+    50,
+    50
+   ],
+   "gpu_power": [
+    279.11,
+    266.12,
+    276.9,
+    277.46,
+    279.35,
+    277.6,
+    278.96,
+    278.2,
+    276.29,
+    279.45,
+    275.58,
+    268.64,
+    275.16,
+    277.98,
+    275.78,
+    278.17,
+    279.67,
+    276.04,
+    279.46,
+    275.1,
+    265.27,
+    279.22,
+    280.54,
+    276.07,
+    280.55,
+    278.32,
+    274.82,
+    275.85,
+    280.08,
+    279.93,
+    266.81,
+    277.79,
+    274.52,
+    274.38,
+    279.74,
+    277.72,
+    274.64,
+    279.65,
+    280.34,
+    278.34,
+    277.27,
+    274.99,
+    265.64,
+    277.06,
+    278.38,
+    279.64,
+    279.6,
+    273.86,
+    273.12,
+    273.61,
+    275.7,
+    275.96,
+    276.36,
+    266.12,
+    265.34,
+    275.53,
+    242.04,
+    256.98,
+    274.43,
+    278.23
+   ],
+   "gpu_pcie_rx_mb": [
+    28.08,
+    16676.85,
+    27.75,
+    7942.58,
+    15.78,
+    16.36,
+    19454.0,
+    15911.58,
+    26.8,
+    41.1,
+    28.75,
+    15.21,
+    24.79,
+    15.5,
+    12903.17,
+    15722.66,
+    13615.27,
+    5212.5,
+    21571.11,
+    9259.8,
+    29.85,
+    15.5,
+    9701.01,
+    153.35,
+    15.64,
+    30.23,
+    25023.46,
+    14.5,
+    29.71,
+    26217.94,
+    17128.85,
+    9320.26,
+    28.28,
+    28.94,
+    13.54,
+    87.12,
+    1377.34,
+    10125.25,
+    29.04,
+    30.52,
+    15905.0,
+    8753.63,
+    7850.26,
+    26.85,
+    27.04,
+    15.31,
+    3817.42,
+    27.99,
+    26.89,
+    26269.48,
+    26779.27,
+    20418.6,
+    6900.17,
+    6532.96,
+    1637.36,
+    26114.94,
+    1472.71,
+    1607.46,
+    3050.8,
+    1220.75
+   ],
+   "cpu": [
+    6.4,
+    10.7,
+    11.7,
+    16.5,
+    8.6,
+    6.3,
+    18.6,
+    6.0,
+    5.9,
+    5.7,
+    8.8,
+    12.7,
+    5.6,
+    5.9,
+    6.6,
+    12.9,
+    11.3,
+    13.4,
+    13.0,
+    9.8,
+    14.3,
+    23.4,
+    6.3,
+    5.9,
+    7.9,
+    13.4,
+    9.7,
+    6.5,
+    6.7,
+    5.9,
+    10.3,
+    17.8,
+    14.9,
+    7.1,
+    8.8,
+    5.8,
+    16.3,
+    7.3,
+    5.7,
+    7.3,
+    5.7,
+    8.0,
+    14.4,
+    7.4,
+    6.6,
+    5.9,
+    15.4,
+    12.5,
+    6.2,
+    6.9,
+    7.8,
+    7.2,
+    6.8,
+    7.1,
+    5.7,
+    6.0,
+    44.6,
+    99.9,
+    99.9,
+    100.0
+   ],
+   "ram_used": [
+    65760505856,
+    65825771520,
+    65805475840,
+    65899847680,
+    65884385280,
+    65881726976,
+    65877250048,
+    65865695232,
+    65856700416,
+    65843732480,
+    65918058496,
+    65911185408,
+    65901252608,
+    65896804352,
+    65888411648,
+    65880047616,
+    65859633152,
+    65903091712,
+    65906130944,
+    65929789440,
+    65996169216,
+    65697763328,
+    65680461824,
+    65660788736,
+    65639858176,
+    65624662016,
+    65614233600,
+    65604620288,
+    65594793984,
+    65589379072,
+    65666322432,
+    65661911040,
+    65706741760,
+    65701203968,
+    65697587200,
+    65691623424,
+    65677221888,
+    65665282048,
+    65657679872,
+    65628098560,
+    65621381120,
+    65695760384,
+    65678503936,
+    65672011776,
+    65665482752,
+    65658216448,
+    65673232384,
+    65655951360,
+    65646583808,
+    65641152512,
+    65659457536,
+    65656025088,
+    65651355648,
+    65716195328,
+    65703702528,
+    65693274112,
+    65702584320,
+    65687781376,
+    65690525696,
+    65694810112
+   ],
+   "disk_read_mb": [
+    0.0,
+    84.11,
+    102.3,
+    0.0,
+    0.0,
+    0.0,
+    175.14,
+    0.0,
+    0.0,
+    0.0,
+    43.72,
+    150.32,
+    0.0,
+    0.0,
+    0.0,
+    118.91,
+    69.66,
+    0.0,
+    0.0,
+    0.0,
+    128.82,
+    137.4,
+    0.0,
+    0.0,
+    0.0,
+    143.2,
+    89.93,
+    0.0,
+    0.0,
+    0.0,
+    81.49,
+    154.61,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    196.96,
+    31.0,
+    0.0,
+    0.0,
+    0.0,
+    22.35,
+    158.36,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    63.48,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    1.53,
+    4.38,
+    2.32,
+    3.75
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    41.84,
+    65.33,
+    69.08,
+    74.58
+   ]
+  },
+  "time": 1790715283.9323688
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-262k-2.json

@@ -0,0 +1,689 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "262k",
+ "target_tokens": 260864,
+ "run": 2,
+ "run_id": "fbabd65a593d",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:54:43",
+ "prompt_tokens": 259589,
+ "completion_tokens": 92,
+ "ttft_s": 152.099,
+ "e2e_s": 153.446,
+ "api_prompt_tps": 1706.7,
+ "api_decode_tps": 67.5,
+ "engine_prompt_tps": 1721.6,
+ "engine_decode_tps": 67.3,
+ "log_lines": [
+  "strata serve: prompt 259589 tokens = 0 reused + 259589 read in 150784 ms (1721.6 tok/s), 92 generated in 1367 ms (67.3 tok/s), drafts accepted 61 of 75, 6 checkpoints",
+  "strata serve: decode expert cache hit rate: 90.0% (43983 hits / 48863 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand the codebase and the last file in detail.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 262144,
+   "images": false,
+   "version": "0.1.26",
+   "context": 262144,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10451,
+   "expert_cache_mib": 17355,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 385,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790715285.1507986,
+    "duration_s": 152.2,
+    "finish": "stop",
+    "prompt_tokens": 259589,
+    "reused": 0,
+    "output_tokens": 92,
+    "engine_generated": 92,
+    "prompt_ms": 150783.8,
+    "decode_ms": 1366.8,
+    "decode_tok_s": 67.3,
+    "hit_rate": 0.9
+   },
+   {
+    "projection": null,
+    "time": 1790715149.1813676,
+    "duration_s": 134.2,
+    "finish": "length",
+    "prompt_tokens": 232183,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 130101.1,
+    "decode_ms": 4061.2,
+    "decode_tok_s": 63.0,
+    "hit_rate": 0.937
+   }
+  ],
+  "requests_kept": 2,
+  "totals": {
+   "since": 1790715131.4670517,
+   "requests": 2,
+   "prompt_tokens": 491772,
+   "reused": 0,
+   "output_tokens": 348,
+   "prompt_ms": 280884.9,
+   "decode_ms": 5428.0
+  },
+  "hardware": {
+   "gpu_util": 96,
+   "gpu_mem_used": 25512837120,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 52,
+   "gpu_power": 277.319,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 5709.8388671875,
+   "gpu_pcie_tx_mb": 1049.041015625,
+   "cpu": 99.9,
+   "ram_used": 65638817792,
+   "ram_total": 177175990272,
+   "disk_read_mb": 3.1622365794124767,
+   "disk_write_mb": 0.03737868297177868,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    99,
+    100,
+    100,
+    99,
+    100,
+    100,
+    100,
+    99,
+    100,
+    100,
+    99,
+    99,
+    97,
+    100,
+    100,
+    100,
+    99,
+    99,
+    97,
+    100,
+    100,
+    100,
+    100,
+    60,
+    100,
+    100,
+    99,
+    99,
+    100,
+    98,
+    99,
+    99,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    98,
+    100,
+    100,
+    99,
+    99,
+    100,
+    98,
+    100,
+    99,
+    100,
+    100,
+    100,
+    100,
+    100,
+    99,
+    100,
+    99,
+    96,
+    96
+   ],
+   "gpu_mem_used": [
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120
+   ],
+   "gpu_temp": [
+    51,
+    52,
+    52,
+    51,
+    51,
+    52,
+    51,
+    50,
+    51,
+    52,
+    51,
+    51,
+    51,
+    51,
+    51,
+    52,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    52,
+    50,
+    52,
+    51,
+    52,
+    52,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    51,
+    52,
+    52,
+    52,
+    52,
+    52,
+    52,
+    51,
+    51,
+    51,
+    52,
+    51,
+    52,
+    52,
+    52,
+    52,
+    52,
+    52,
+    52,
+    52,
+    52,
+    52,
+    52,
+    51,
+    52
+   ],
+   "gpu_power": [
+    280.1,
+    277.72,
+    269.31,
+    279.51,
+    276.6,
+    277.19,
+    280.46,
+    278.19,
+    275.14,
+    276.45,
+    280.49,
+    278.01,
+    265.6,
+    274.96,
+    274.76,
+    278.94,
+    279.21,
+    278.77,
+    277.22,
+    274.11,
+    273.93,
+    275.55,
+    279.88,
+    274.53,
+    278.22,
+    280.08,
+    279.56,
+    277.48,
+    275.9,
+    276.58,
+    275.74,
+    275.62,
+    274.27,
+    272.69,
+    272.53,
+    276.33,
+    279.54,
+    279.3,
+    279.1,
+    279.0,
+    278.24,
+    279.57,
+    276.45,
+    276.52,
+    276.57,
+    276.89,
+    273.45,
+    267.54,
+    275.91,
+    275.91,
+    277.91,
+    276.31,
+    276.26,
+    279.97,
+    276.73,
+    278.79,
+    274.29,
+    278.64,
+    241.5,
+    277.32
+   ],
+   "gpu_pcie_rx_mb": [
+    24.6,
+    11576.41,
+    14790.49,
+    31.66,
+    20927.76,
+    28.32,
+    23015.64,
+    14269.26,
+    27.42,
+    9814.79,
+    27090.45,
+    28.08,
+    29.28,
+    30.8,
+    22511.15,
+    22487.21,
+    30.47,
+    74.67,
+    20330.0,
+    20889.04,
+    27.89,
+    31.8,
+    55.55,
+    14868.74,
+    15.35,
+    20128.63,
+    27.13,
+    26.56,
+    53.21,
+    24118.71,
+    16327.71,
+    9496.97,
+    20100.59,
+    27.75,
+    15.97,
+    14788.58,
+    13.49,
+    13957.07,
+    26.13,
+    27.75,
+    26181.89,
+    48.68,
+    16340.3,
+    5056.67,
+    25172.95,
+    16.36,
+    87.55,
+    33.81,
+    31.28,
+    31.66,
+    35.0,
+    35.09,
+    8309.17,
+    6276.99,
+    31.14,
+    13669.87,
+    18489.17,
+    29.47,
+    2057.6,
+    5709.84
+   ],
+   "cpu": [
+    6.6,
+    7.6,
+    11.4,
+    12.8,
+    6.6,
+    6.0,
+    6.6,
+    11.6,
+    10.6,
+    6.8,
+    5.6,
+    6.1,
+    9.8,
+    25.0,
+    9.1,
+    5.9,
+    6.8,
+    6.1,
+    16.7,
+    7.0,
+    8.8,
+    6.9,
+    5.9,
+    12.8,
+    11.2,
+    5.6,
+    6.5,
+    14.2,
+    10.7,
+    16.4,
+    8.1,
+    7.0,
+    6.2,
+    5.9,
+    7.4,
+    13.8,
+    6.0,
+    5.7,
+    7.0,
+    6.1,
+    6.2,
+    15.3,
+    15.5,
+    12.5,
+    6.3,
+    6.3,
+    6.0,
+    14.9,
+    7.3,
+    9.6,
+    6.6,
+    6.1,
+    6.6,
+    8.6,
+    5.9,
+    6.2,
+    16.0,
+    9.9,
+    42.7,
+    99.9
+   ],
+   "ram_used": [
+    65829687296,
+    65809809408,
+    65878708224,
+    65862221824,
+    65854197760,
+    65848958976,
+    65840418816,
+    65838673920,
+    65828872192,
+    65821380608,
+    65813520384,
+    65804607488,
+    65870708736,
+    65884573696,
+    65869344768,
+    65860734976,
+    65835606016,
+    65825009664,
+    65818324992,
+    65817894912,
+    65821642752,
+    65816698880,
+    65812000768,
+    65862680576,
+    65843482624,
+    65833279488,
+    65827409920,
+    65838510080,
+    65830367232,
+    65803018240,
+    65796870144,
+    65792782336,
+    65785704448,
+    65781006336,
+    65859014656,
+    65839325184,
+    65832935424,
+    65818849280,
+    65813626880,
+    65805553664,
+    65796202496,
+    65790906368,
+    65792847872,
+    65822486528,
+    65813880832,
+    65796804608,
+    65786474496,
+    65854476288,
+    65841573888,
+    65829576704,
+    65816891392,
+    65810354176,
+    65800015872,
+    65793110016,
+    65783308288,
+    65776508928,
+    65580986368,
+    65575313408,
+    65644572672,
+    65638817792
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    99.27,
+    135.95,
+    0.0,
+    0.0,
+    0.0,
+    113.06,
+    91.94,
+    0.0,
+    0.0,
+    0.0,
+    46.52,
+    161.04,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    199.72,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    91.5,
+    66.56,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    191.98,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    13.73,
+    127.69,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    1.52,
+    125.36,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    171.54,
+    7.82,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    1.75,
+    3.16
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    69.44,
+    0.0
+   ]
+  },
+  "time": 1790715437.8967788
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-262k-3.json

@@ -0,0 +1,703 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "262k",
+ "target_tokens": 260864,
+ "run": 3,
+ "run_id": "40a721fd9a17",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:57:17",
+ "prompt_tokens": 261068,
+ "completion_tokens": 256,
+ "ttft_s": 152.605,
+ "e2e_s": 156.279,
+ "api_prompt_tps": 1710.7,
+ "api_decode_tps": 69.4,
+ "engine_prompt_tps": 1725.6,
+ "engine_decode_tps": 69.3,
+ "log_lines": [
+  "strata serve: prompt 261068 tokens = 0 reused + 261068 read in 151288 ms (1725.6 tok/s), 256 generated in 3693 ms (69.3 tok/s), drafts accepted 151 of 231, 6 checkpoints",
+  "strata serve: decode expert cache hit rate: 96.3% (153481 hits / 159304 lookups)"
+ ],
+ "answer_head": "I'll analyze the last file in the repository listing and then propose a concrete improvement.\n\n## What `serve/web/app.js` does\n\nThis file is the **entire client-side logic of the Strata web application** \u2014 a framework-free, single-file JavaScript app that talks only to the local Strata server. It po",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 262144,
+   "images": false,
+   "version": "0.1.26",
+   "context": 262144,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10451,
+   "expert_cache_mib": 17355,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 385,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790715439.1095636,
+    "duration_s": 155.1,
+    "finish": "length",
+    "prompt_tokens": 261068,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 151288.5,
+    "decode_ms": 3693.0,
+    "decode_tok_s": 69.3,
+    "hit_rate": 0.963
+   },
+   {
+    "projection": null,
+    "time": 1790715285.1507986,
+    "duration_s": 152.2,
+    "finish": "stop",
+    "prompt_tokens": 259589,
+    "reused": 0,
+    "output_tokens": 92,
+    "engine_generated": 92,
+    "prompt_ms": 150783.8,
+    "decode_ms": 1366.8,
+    "decode_tok_s": 67.3,
+    "hit_rate": 0.9
+   },
+   {
+    "projection": null,
+    "time": 1790715149.1813676,
+    "duration_s": 134.2,
+    "finish": "length",
+    "prompt_tokens": 232183,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 130101.1,
+    "decode_ms": 4061.2,
+    "decode_tok_s": 63.0,
+    "hit_rate": 0.937
+   }
+  ],
+  "requests_kept": 3,
+  "totals": {
+   "since": 1790715131.4670517,
+   "requests": 3,
+   "prompt_tokens": 752840,
+   "reused": 0,
+   "output_tokens": 604,
+   "prompt_ms": 432173.4,
+   "decode_ms": 9121.0
+  },
+  "hardware": {
+   "gpu_util": 92,
+   "gpu_mem_used": 25512837120,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 53,
+   "gpu_power": 277.576,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 3810.1669921875,
+   "gpu_pcie_tx_mb": 299.7392578125,
+   "cpu": 100.0,
+   "ram_used": 65843089408,
+   "ram_total": 177175990272,
+   "disk_read_mb": 4.922097648659462,
+   "disk_write_mb": 0.0,
+   "tok_s": 68.55526509763148,
+   "tok_s_mean": 69.76955378630859
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    99,
+    100,
+    99,
+    99,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    99,
+    100,
+    100,
+    100,
+    100,
+    100,
+    99,
+    100,
+    98,
+    99,
+    99,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    98,
+    100,
+    100,
+    100,
+    99,
+    99,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    88,
+    100,
+    100,
+    100,
+    99,
+    99,
+    99,
+    100,
+    100,
+    100,
+    100,
+    100,
+    100,
+    97,
+    95,
+    95,
+    92
+   ],
+   "gpu_mem_used": [
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120,
+    25512837120
+   ],
+   "gpu_temp": [
+    53,
+    52,
+    53,
+    52,
+    52,
+    52,
+    53,
+    53,
+    52,
+    52,
+    53,
+    53,
+    53,
+    52,
+    52,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    52,
+    52,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    52,
+    52,
+    52,
+    52,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    52,
+    53,
+    53,
+    52,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53,
+    53
+   ],
+   "gpu_power": [
+    279.28,
+    276.25,
+    280.52,
+    280.21,
+    276.32,
+    278.64,
+    280.39,
+    275.19,
+    275.3,
+    267.01,
+    278.7,
+    279.27,
+    280.07,
+    277.82,
+    274.89,
+    273.96,
+    277.23,
+    280.24,
+    279.4,
+    277.21,
+    266.21,
+    276.82,
+    277.27,
+    274.7,
+    272.68,
+    273.87,
+    273.33,
+    275.44,
+    276.56,
+    276.53,
+    276.52,
+    263.9,
+    276.06,
+    275.76,
+    276.06,
+    275.68,
+    275.74,
+    273.79,
+    276.62,
+    276.84,
+    276.88,
+    276.17,
+    275.71,
+    273.17,
+    265.54,
+    275.78,
+    273.31,
+    275.66,
+    275.72,
+    276.2,
+    275.38,
+    273.99,
+    273.72,
+    275.56,
+    276.07,
+    276.57,
+    245.75,
+    278.73,
+    277.86,
+    277.58
+   ],
+   "gpu_pcie_rx_mb": [
+    4135.46,
+    6890.58,
+    13.64,
+    3437.09,
+    30.71,
+    20378.88,
+    15.64,
+    30.99,
+    20391.65,
+    19570.21,
+    4926.97,
+    27.94,
+    28.08,
+    26564.6,
+    4245.0,
+    29.13,
+    28.94,
+    7148.27,
+    17857.03,
+    28.51,
+    11700.44,
+    28.28,
+    27.51,
+    26015.42,
+    14.3,
+    16.36,
+    11904.38,
+    29.71,
+    29.99,
+    33.33,
+    14.83,
+    30.04,
+    28.8,
+    28.18,
+    26.08,
+    17.07,
+    26755.14,
+    31.04,
+    30.14,
+    30.8,
+    29.04,
+    30.8,
+    33.24,
+    27187.54,
+    19414.33,
+    9223.17,
+    54.17,
+    12.68,
+    24.32,
+    33.43,
+    12868.02,
+    32.04,
+    29.66,
+    34.62,
+    20107.13,
+    84.11,
+    1655.15,
+    10931.06,
+    1614.47,
+    3810.17
+   ],
+   "cpu": [
+    12.6,
+    5.5,
+    6.4,
+    6.8,
+    13.7,
+    9.3,
+    10.4,
+    12.8,
+    12.2,
+    11.8,
+    13.0,
+    6.1,
+    6.4,
+    6.3,
+    8.4,
+    15.9,
+    9.1,
+    6.5,
+    7.0,
+    5.8,
+    13.8,
+    18.7,
+    8.2,
+    6.0,
+    7.7,
+    8.1,
+    14.9,
+    5.8,
+    6.5,
+    6.5,
+    5.9,
+    9.5,
+    9.8,
+    6.4,
+    6.2,
+    13.8,
+    11.5,
+    9.4,
+    9.4,
+    6.0,
+    7.2,
+    5.8,
+    6.6,
+    6.5,
+    15.9,
+    8.9,
+    7.4,
+    6.0,
+    6.1,
+    9.0,
+    13.9,
+    6.6,
+    8.0,
+    6.2,
+    6.0,
+    7.2,
+    56.6,
+    100.0,
+    100.0,
+    100.0
+   ],
+   "ram_used": [
+    65659777024,
+    65650483200,
+    65649508352,
+    65644490752,
+    65632600064,
+    65621151744,
+    65675010048,
+    65661538304,
+    65681547264,
+    65747542016,
+    65735454720,
+    65723609088,
+    65715372032,
+    65702187008,
+    65677402112,
+    65653993472,
+    65667379200,
+    65663696896,
+    65657884672,
+    65648963584,
+    65722257408,
+    65791442944,
+    65798762496,
+    65787301888,
+    65781350400,
+    65659375616,
+    65646817280,
+    65635090432,
+    65627262976,
+    65615314944,
+    65606516736,
+    65678639104,
+    65669410816,
+    65663270912,
+    65656647680,
+    65739636736,
+    65798643712,
+    65792847872,
+    65780387840,
+    65772281856,
+    65774555136,
+    65768087552,
+    65765552128,
+    65838772224,
+    65833267200,
+    65850757120,
+    65824538624,
+    65812451328,
+    65801834496,
+    65796722688,
+    65808052224,
+    65799196672,
+    65796362240,
+    65791713280,
+    65787756544,
+    65798152192,
+    65876860928,
+    65866027008,
+    65852674048,
+    65843089408
+   ],
+   "disk_read_mb": [
+    116.54,
+    0.0,
+    0.0,
+    0.0,
+    150.82,
+    68.05,
+    0.0,
+    0.0,
+    0.0,
+    84.44,
+    137.56,
+    0.0,
+    0.0,
+    0.0,
+    19.83,
+    193.88,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    143.92,
+    24.87,
+    0.0,
+    0.0,
+    0.0,
+    25.89,
+    153.9,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    69.98,
+    56.36,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    48.96,
+    64.15,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    206.65,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    2.78,
+    4.65,
+    4.44,
+    4.92
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    74.98,
+    63.75,
+    72.35,
+    68.56
+   ]
+  },
+  "time": 1790715594.6843412
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-32k-1.json

@@ -0,0 +1,829 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 1,
+ "run_id": "563f1c703ebe",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:41:08",
+ "prompt_tokens": 34932,
+ "completion_tokens": 256,
+ "ttft_s": 16.748,
+ "e2e_s": 19.56,
+ "api_prompt_tps": 2085.7,
+ "api_decode_tps": 90.7,
+ "engine_prompt_tps": 2110.2,
+ "engine_decode_tps": 90.6,
+ "log_lines": [
+  "strata serve: prompt 34932 tokens = 0 reused + 34932 read in 16554 ms (2110.2 tok/s), 256 generated in 2827 ms (90.6 tok/s), drafts accepted 167 of 216, 3 checkpoints",
+  "strata serve: decode expert cache hit rate: 94.1% (133137 hits / 141416 lookups)"
+ ],
+ "answer_head": "I'll analyze the last file in the repository listing and propose an improvement.\n\n## What `docs/DETAILS.md` does\n\nThis is the comprehensive technical documentation file for the Strata project. It serves as the single source of truth for:\n\n1. **Performance benchmarks** \u2014 Detailed tables of prompt-pro",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714469.1007972,
+    "duration_s": 19.4,
+    "finish": "length",
+    "prompt_tokens": 34932,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 16553.9,
+    "decode_ms": 2826.7,
+    "decode_tok_s": 90.6,
+    "hit_rate": 0.941
+   },
+   {
+    "projection": null,
+    "time": 1790714463.1913157,
+    "duration_s": 5.2,
+    "finish": "length",
+    "prompt_tokens": 4201,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2365.6,
+    "decode_ms": 2868.4,
+    "decode_tok_s": 89.2,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714458.6299415,
+    "duration_s": 4.0,
+    "finish": "stop",
+    "prompt_tokens": 4200,
+    "reused": 0,
+    "output_tokens": 142,
+    "engine_generated": 142,
+    "prompt_ms": 2370.0,
+    "decode_ms": 1660.0,
+    "decode_tok_s": 85.5,
+    "hit_rate": 0.896
+   },
+   {
+    "projection": null,
+    "time": 1790714453.0791314,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 3881,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2223.8,
+    "decode_ms": 2790.5,
+    "decode_tok_s": 91.7,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790714448.5537117,
+    "duration_s": 4.0,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1305.1,
+    "decode_ms": 2690.8,
+    "decode_tok_s": 95.1,
+    "hit_rate": 0.972
+   },
+   {
+    "projection": null,
+    "time": 1790714443.8943996,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1149,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1297.6,
+    "decode_ms": 2847.5,
+    "decode_tok_s": 89.9,
+    "hit_rate": 0.971
+   },
+   {
+    "projection": null,
+    "time": 1790714439.2770011,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1092,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1351.4,
+    "decode_ms": 2752.6,
+    "decode_tok_s": 93.0,
+    "hit_rate": 0.954
+   },
+   {
+    "projection": null,
+    "time": 1790714377.2599747,
+    "duration_s": 17.6,
+    "finish": "length",
+    "prompt_tokens": 31605,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14608.5,
+    "decode_ms": 3015.6,
+    "decode_tok_s": 84.9,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714359.1307795,
+    "duration_s": 17.5,
+    "finish": "length",
+    "prompt_tokens": 31589,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14609.7,
+    "decode_ms": 2833.5,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714338.0043592,
+    "duration_s": 20.5,
+    "finish": "length",
+    "prompt_tokens": 34942,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 17033.3,
+    "decode_ms": 3435.7,
+    "decode_tok_s": 74.5,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714334.3235514,
+    "duration_s": 3.0,
+    "finish": "stop",
+    "prompt_tokens": 4199,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 2369.7,
+    "decode_ms": 626.3,
+    "decode_tok_s": 78.2,
+    "hit_rate": 0.934
+   },
+   {
+    "projection": null,
+    "time": 1790714328.2723713,
+    "duration_s": 5.5,
+    "finish": "length",
+    "prompt_tokens": 4214,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2436.3,
+    "decode_ms": 3084.4,
+    "decode_tok_s": 83.0,
+    "hit_rate": 0.959
+   }
+  ],
+  "requests_kept": 17,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 17,
+   "prompt_tokens": 164273,
+   "reused": 0,
+   "output_tokens": 3366,
+   "prompt_ms": 86199.0,
+   "decode_ms": 38719.19999999999
+  },
+  "hardware": {
+   "gpu_util": 97,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 46,
+   "gpu_power": 278.81,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 5630.111328125,
+   "gpu_pcie_tx_mb": 76.2939453125,
+   "cpu": 100.0,
+   "ram_used": 63789760512,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.3587450688100042,
+   "disk_write_mb": 0.1268945131954749,
+   "tok_s": 96.52070218780436,
+   "tok_s_mean": 91.25774344883004
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    100,
+    98,
+    94,
+    94,
+    100,
+    21,
+    98,
+    95,
+    53,
+    100,
+    94,
+    96,
+    21,
+    99,
+    99,
+    92,
+    97,
+    97,
+    0,
+    98,
+    99,
+    98,
+    0,
+    99,
+    100,
+    99,
+    96,
+    96,
+    0,
+    100,
+    99,
+    100,
+    100,
+    99,
+    100,
+    100,
+    99,
+    100,
+    99,
+    100,
+    99,
+    99,
+    99,
+    100,
+    98,
+    99,
+    97
+   ],
+   "gpu_mem_used": [
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    38,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    45,
+    45,
+    45,
+    45,
+    45,
+    43,
+    45,
+    45,
+    44,
+    45,
+    45,
+    45,
+    43,
+    45,
+    45,
+    45,
+    45,
+    46,
+    42,
+    46,
+    45,
+    46,
+    42,
+    45,
+    45,
+    45,
+    46,
+    46,
+    42,
+    46,
+    46,
+    46,
+    46,
+    46,
+    45,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46
+   ],
+   "gpu_power": [
+    34.89,
+    35.0,
+    34.87,
+    34.82,
+    34.93,
+    34.86,
+    34.92,
+    34.99,
+    34.83,
+    34.88,
+    34.97,
+    34.89,
+    34.96,
+    172.41,
+    265.41,
+    275.33,
+    276.53,
+    216.14,
+    263.1,
+    264.26,
+    273.6,
+    211.21,
+    274.12,
+    265.58,
+    276.31,
+    268.45,
+    217.39,
+    278.78,
+    252.44,
+    278.12,
+    278.79,
+    215.59,
+    278.37,
+    251.8,
+    276.73,
+    219.4,
+    271.43,
+    276.62,
+    251.66,
+    277.46,
+    278.06,
+    194.1,
+    265.98,
+    277.8,
+    274.38,
+    278.56,
+    277.99,
+    277.15,
+    264.89,
+    276.96,
+    277.37,
+    277.24,
+    276.83,
+    276.94,
+    278.15,
+    264.71,
+    273.73,
+    243.99,
+    277.71,
+    278.81
+   ],
+   "gpu_pcie_rx_mb": [
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    9.97,
+    22329.52,
+    3689.24,
+    4264.07,
+    0.33,
+    14955.09,
+    784.16,
+    3310.82,
+    1249.69,
+    15874.15,
+    18357.04,
+    4099.13,
+    485.18,
+    0.29,
+    11.78,
+    84.59,
+    865.12,
+    8675.62,
+    0.33,
+    3342.49,
+    34.05,
+    7461.5,
+    4551.6,
+    6961.73,
+    24513.15,
+    26324.56,
+    679.21,
+    426.48,
+    0.33,
+    10.87,
+    10442.35,
+    23162.22,
+    22324.94,
+    24878.6,
+    26870.2,
+    14619.68,
+    11419.68,
+    893.4,
+    13515.52,
+    20953.08,
+    7.87,
+    14.59,
+    23972.7,
+    9329.51,
+    82.11,
+    7527.59,
+    4538.68,
+    5630.11
+   ],
+   "cpu": [
+    0.8,
+    1.2,
+    6.3,
+    8.9,
+    0.6,
+    1.9,
+    2.2,
+    2.1,
+    1.1,
+    2.2,
+    4.7,
+    1.2,
+    6.8,
+    7.3,
+    83.2,
+    99.8,
+    87.0,
+    14.8,
+    49.9,
+    100.0,
+    97.9,
+    33.3,
+    7.4,
+    99.9,
+    100.0,
+    63.3,
+    5.7,
+    6.4,
+    81.2,
+    100.0,
+    93.8,
+    14.8,
+    15.8,
+    37.5,
+    100.0,
+    32.6,
+    6.8,
+    6.6,
+    96.7,
+    100.0,
+    83.5,
+    5.1,
+    7.7,
+    6.8,
+    5.8,
+    11.1,
+    14.6,
+    9.1,
+    8.4,
+    7.3,
+    7.4,
+    7.0,
+    7.1,
+    6.5,
+    5.9,
+    6.0,
+    6.9,
+    72.2,
+    100.0,
+    100.0
+   ],
+   "ram_used": [
+    63794352128,
+    63793430528,
+    63799361536,
+    63590793216,
+    63591645184,
+    63583899648,
+    63571738624,
+    63579156480,
+    63572910080,
+    63581016064,
+    63625879552,
+    63626788864,
+    63427158016,
+    63429017600,
+    63532883968,
+    63535853568,
+    63534440448,
+    63490953216,
+    63544176640,
+    63529631744,
+    63536766976,
+    63443234816,
+    63502606336,
+    63508336640,
+    63509970944,
+    63492075520,
+    63440269312,
+    63431364608,
+    63482470400,
+    63473512448,
+    63471906816,
+    63401848832,
+    63554486272,
+    63453114368,
+    63441051648,
+    63399321600,
+    63389306880,
+    63427080192,
+    63418408960,
+    63417094144,
+    63416299520,
+    63358083072,
+    63351615488,
+    63342723072,
+    63336611840,
+    63371808768,
+    63426170880,
+    63504416768,
+    63594704896,
+    63611879424,
+    63612047360,
+    63599980544,
+    63573508096,
+    63570501632,
+    63568433152,
+    63702056960,
+    63700324352,
+    63800950784,
+    63801184256,
+    63789760512
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.8,
+    3.91,
+    3.26,
+    0.6,
+    0.0,
+    1.08,
+    2.82,
+    1.34,
+    0.0,
+    1.55,
+    2.4,
+    1.66,
+    0.64,
+    0.0,
+    2.25,
+    2.98,
+    2.66,
+    0.64,
+    0.0,
+    0.21,
+    2.1,
+    1.17,
+    0.0,
+    0.0,
+    1.29,
+    3.73,
+    2.96,
+    0.82,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.0,
+    0.0,
+    0.23,
+    0.0,
+    0.0,
+    0.22,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.7,
+    0.31,
+    2.36
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    103.98,
+    98.22,
+    0.0,
+    0.0,
+    112.83,
+    83.57,
+    76.97,
+    0.0,
+    0.0,
+    97.69,
+    87.72,
+    0.0,
+    0.0,
+    0.0,
+    102.14,
+    97.9,
+    0.0,
+    0.0,
+    0.0,
+    89.96,
+    98.26,
+    0.0,
+    0.0,
+    0.0,
+    93.47,
+    116.21,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    97.18,
+    97.71,
+    96.52
+   ]
+  },
+  "time": 1790714488.9946597
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-32k-2.json

@@ -0,0 +1,829 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 2,
+ "run_id": "283cb867e188",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:41:28",
+ "prompt_tokens": 31582,
+ "completion_tokens": 256,
+ "ttft_s": 14.807,
+ "e2e_s": 18.058,
+ "api_prompt_tps": 2132.9,
+ "api_decode_tps": 78.4,
+ "engine_prompt_tps": 2160.4,
+ "engine_decode_tps": 78.3,
+ "log_lines": [
+  "strata serve: prompt 31582 tokens = 0 reused + 31582 read in 14619 ms (2160.4 tok/s), 256 generated in 3271 ms (78.3 tok/s), drafts accepted 161 of 240, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 96.0% (151459 hits / 157708 lookups)"
+ ],
+ "answer_head": "The last file in the provided list is `docs/DETAILS.md`.\n\n### What it does\nThis file serves as the **comprehensive technical manual and reference guide** for the Strata engine. While the `README.md` focuses on quick installation and high-level marketing, `DETAILS.md` provides the deep technical spec",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714489.1532733,
+    "duration_s": 17.9,
+    "finish": "length",
+    "prompt_tokens": 31582,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14618.7,
+    "decode_ms": 3270.7,
+    "decode_tok_s": 78.3,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714469.1007972,
+    "duration_s": 19.4,
+    "finish": "length",
+    "prompt_tokens": 34932,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 16553.9,
+    "decode_ms": 2826.7,
+    "decode_tok_s": 90.6,
+    "hit_rate": 0.941
+   },
+   {
+    "projection": null,
+    "time": 1790714463.1913157,
+    "duration_s": 5.2,
+    "finish": "length",
+    "prompt_tokens": 4201,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2365.6,
+    "decode_ms": 2868.4,
+    "decode_tok_s": 89.2,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714458.6299415,
+    "duration_s": 4.0,
+    "finish": "stop",
+    "prompt_tokens": 4200,
+    "reused": 0,
+    "output_tokens": 142,
+    "engine_generated": 142,
+    "prompt_ms": 2370.0,
+    "decode_ms": 1660.0,
+    "decode_tok_s": 85.5,
+    "hit_rate": 0.896
+   },
+   {
+    "projection": null,
+    "time": 1790714453.0791314,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 3881,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2223.8,
+    "decode_ms": 2790.5,
+    "decode_tok_s": 91.7,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790714448.5537117,
+    "duration_s": 4.0,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1305.1,
+    "decode_ms": 2690.8,
+    "decode_tok_s": 95.1,
+    "hit_rate": 0.972
+   },
+   {
+    "projection": null,
+    "time": 1790714443.8943996,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1149,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1297.6,
+    "decode_ms": 2847.5,
+    "decode_tok_s": 89.9,
+    "hit_rate": 0.971
+   },
+   {
+    "projection": null,
+    "time": 1790714439.2770011,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1092,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1351.4,
+    "decode_ms": 2752.6,
+    "decode_tok_s": 93.0,
+    "hit_rate": 0.954
+   },
+   {
+    "projection": null,
+    "time": 1790714377.2599747,
+    "duration_s": 17.6,
+    "finish": "length",
+    "prompt_tokens": 31605,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14608.5,
+    "decode_ms": 3015.6,
+    "decode_tok_s": 84.9,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714359.1307795,
+    "duration_s": 17.5,
+    "finish": "length",
+    "prompt_tokens": 31589,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14609.7,
+    "decode_ms": 2833.5,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714338.0043592,
+    "duration_s": 20.5,
+    "finish": "length",
+    "prompt_tokens": 34942,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 17033.3,
+    "decode_ms": 3435.7,
+    "decode_tok_s": 74.5,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714334.3235514,
+    "duration_s": 3.0,
+    "finish": "stop",
+    "prompt_tokens": 4199,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 2369.7,
+    "decode_ms": 626.3,
+    "decode_tok_s": 78.2,
+    "hit_rate": 0.934
+   }
+  ],
+  "requests_kept": 18,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 18,
+   "prompt_tokens": 195855,
+   "reused": 0,
+   "output_tokens": 3622,
+   "prompt_ms": 100817.7,
+   "decode_ms": 41989.89999999999
+  },
+  "hardware": {
+   "gpu_util": 94,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 47,
+   "gpu_power": 276.188,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.380859375,
+   "gpu_pcie_tx_mb": 0.4287109375,
+   "cpu": 84.0,
+   "ram_used": 63683891200,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.12697518787369,
+   "disk_write_mb": 0.05242544477153462,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    21,
+    98,
+    95,
+    53,
+    100,
+    94,
+    96,
+    21,
+    99,
+    99,
+    92,
+    97,
+    97,
+    0,
+    98,
+    99,
+    98,
+    0,
+    99,
+    100,
+    99,
+    96,
+    96,
+    0,
+    100,
+    99,
+    100,
+    100,
+    99,
+    100,
+    100,
+    99,
+    100,
+    99,
+    100,
+    99,
+    99,
+    99,
+    100,
+    98,
+    99,
+    97,
+    0,
+    99,
+    99,
+    99,
+    99,
+    99,
+    99,
+    98,
+    100,
+    99,
+    100,
+    99,
+    99,
+    99,
+    73,
+    98,
+    97,
+    94
+   ],
+   "gpu_mem_used": [
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    43,
+    45,
+    45,
+    44,
+    45,
+    45,
+    45,
+    43,
+    45,
+    45,
+    45,
+    45,
+    46,
+    42,
+    46,
+    45,
+    46,
+    42,
+    45,
+    45,
+    45,
+    46,
+    46,
+    42,
+    46,
+    46,
+    46,
+    46,
+    46,
+    45,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    43,
+    46,
+    47,
+    46,
+    46,
+    46,
+    47,
+    46,
+    47,
+    46,
+    47,
+    46,
+    46,
+    46,
+    44,
+    46,
+    46,
+    47
+   ],
+   "gpu_power": [
+    263.1,
+    264.26,
+    273.6,
+    211.21,
+    274.12,
+    265.58,
+    276.31,
+    268.45,
+    217.39,
+    278.78,
+    252.44,
+    278.12,
+    278.79,
+    215.59,
+    278.37,
+    251.8,
+    276.73,
+    219.4,
+    271.43,
+    276.62,
+    251.66,
+    277.46,
+    278.06,
+    194.1,
+    265.98,
+    277.8,
+    274.38,
+    278.56,
+    277.99,
+    277.15,
+    264.89,
+    276.96,
+    277.37,
+    277.24,
+    276.83,
+    276.94,
+    278.15,
+    264.71,
+    273.73,
+    243.99,
+    277.71,
+    278.81,
+    203.42,
+    258.01,
+    277.13,
+    277.38,
+    277.05,
+    276.55,
+    275.55,
+    267.16,
+    280.52,
+    277.94,
+    278.68,
+    277.45,
+    276.52,
+    278.47,
+    255.06,
+    250.07,
+    273.21,
+    276.19
+   ],
+   "gpu_pcie_rx_mb": [
+    784.16,
+    3310.82,
+    1249.69,
+    15874.15,
+    18357.04,
+    4099.13,
+    485.18,
+    0.29,
+    11.78,
+    84.59,
+    865.12,
+    8675.62,
+    0.33,
+    3342.49,
+    34.05,
+    7461.5,
+    4551.6,
+    6961.73,
+    24513.15,
+    26324.56,
+    679.21,
+    426.48,
+    0.33,
+    10.87,
+    10442.35,
+    23162.22,
+    22324.94,
+    24878.6,
+    26870.2,
+    14619.68,
+    11419.68,
+    893.4,
+    13515.52,
+    20953.08,
+    7.87,
+    14.59,
+    23972.7,
+    9329.51,
+    82.11,
+    7527.59,
+    4538.68,
+    5630.11,
+    6.67,
+    11788.08,
+    773.91,
+    87.02,
+    13.59,
+    4.53,
+    50.21,
+    7.15,
+    6744.72,
+    14.45,
+    115.11,
+    2575.49,
+    26114.03,
+    3364.04,
+    3375.15,
+    957.73,
+    2766.08,
+    0.38
+   ],
+   "cpu": [
+    49.9,
+    100.0,
+    97.9,
+    33.3,
+    7.4,
+    99.9,
+    100.0,
+    63.3,
+    5.7,
+    6.4,
+    81.2,
+    100.0,
+    93.8,
+    14.8,
+    15.8,
+    37.5,
+    100.0,
+    32.6,
+    6.8,
+    6.6,
+    96.7,
+    100.0,
+    83.5,
+    5.1,
+    7.7,
+    6.8,
+    5.8,
+    11.1,
+    14.6,
+    9.1,
+    8.4,
+    7.3,
+    7.4,
+    7.0,
+    7.1,
+    6.5,
+    5.9,
+    6.0,
+    6.9,
+    72.2,
+    100.0,
+    100.0,
+    19.3,
+    8.8,
+    6.1,
+    7.1,
+    6.6,
+    7.1,
+    6.0,
+    6.8,
+    5.6,
+    5.5,
+    6.6,
+    7.8,
+    6.1,
+    6.2,
+    41.1,
+    99.9,
+    99.4,
+    84.0
+   ],
+   "ram_used": [
+    63544176640,
+    63529631744,
+    63536766976,
+    63443234816,
+    63502606336,
+    63508336640,
+    63509970944,
+    63492075520,
+    63440269312,
+    63431364608,
+    63482470400,
+    63473512448,
+    63471906816,
+    63401848832,
+    63554486272,
+    63453114368,
+    63441051648,
+    63399321600,
+    63389306880,
+    63427080192,
+    63418408960,
+    63417094144,
+    63416299520,
+    63358083072,
+    63351615488,
+    63342723072,
+    63336611840,
+    63371808768,
+    63426170880,
+    63504416768,
+    63594704896,
+    63611879424,
+    63612047360,
+    63599980544,
+    63573508096,
+    63570501632,
+    63568433152,
+    63702056960,
+    63700324352,
+    63800950784,
+    63801184256,
+    63789760512,
+    63567470592,
+    63559606272,
+    63544537088,
+    63536496640,
+    63529500672,
+    63515496448,
+    63496245248,
+    63530811392,
+    63529254912,
+    63530737664,
+    63536521216,
+    63543078912,
+    63537020928,
+    63539232768,
+    63659114496,
+    63677952000,
+    63679279104,
+    63683891200
+   ],
+   "disk_read_mb": [
+    0.0,
+    1.08,
+    2.82,
+    1.34,
+    0.0,
+    1.55,
+    2.4,
+    1.66,
+    0.64,
+    0.0,
+    2.25,
+    2.98,
+    2.66,
+    0.64,
+    0.0,
+    0.21,
+    2.1,
+    1.17,
+    0.0,
+    0.0,
+    1.29,
+    3.73,
+    2.96,
+    0.82,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.0,
+    0.0,
+    0.23,
+    0.0,
+    0.0,
+    0.22,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.7,
+    0.31,
+    2.36,
+    0.74,
+    0.0,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.0,
+    0.34,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.15,
+    1.22,
+    1.87,
+    2.13
+   ],
+   "tok_s": [
+    112.83,
+    83.57,
+    76.97,
+    0.0,
+    0.0,
+    97.69,
+    87.72,
+    0.0,
+    0.0,
+    0.0,
+    102.14,
+    97.9,
+    0.0,
+    0.0,
+    0.0,
+    89.96,
+    98.26,
+    0.0,
+    0.0,
+    0.0,
+    93.47,
+    116.21,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    97.18,
+    97.71,
+    96.52,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    80.68,
+    75.74,
+    112.93,
+    0.0
+   ]
+  },
+  "time": 1790714507.5571826
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-32k-3.json

@@ -0,0 +1,829 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 3,
+ "run_id": "1aba3d88be7a",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:41:47",
+ "prompt_tokens": 31602,
+ "completion_tokens": 256,
+ "ttft_s": 14.833,
+ "e2e_s": 17.604,
+ "api_prompt_tps": 2130.5,
+ "api_decode_tps": 92.0,
+ "engine_prompt_tps": 2161.9,
+ "engine_decode_tps": 91.9,
+ "log_lines": [
+  "strata serve: prompt 31602 tokens = 0 reused + 31602 read in 14618 ms (2161.9 tok/s), 256 generated in 2785 ms (91.9 tok/s), drafts accepted 160 of 207, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 96.5% (137828 hits / 142765 lookups)"
+ ],
+ "answer_head": "The last file in the provided list is `docs/DETAILS.md`.\n\n### What it does\nThis file serves as the comprehensive technical documentation and user manual for the \"Strata\" engine. It bridges the gap between the high-level marketing pitch in `README.md` and the raw source code. Specifically, it:\n\n1.  *",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714507.7469585,
+    "duration_s": 17.4,
+    "finish": "length",
+    "prompt_tokens": 31602,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14617.7,
+    "decode_ms": 2785.2,
+    "decode_tok_s": 91.9,
+    "hit_rate": 0.965
+   },
+   {
+    "projection": null,
+    "time": 1790714489.1532733,
+    "duration_s": 17.9,
+    "finish": "length",
+    "prompt_tokens": 31582,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14618.7,
+    "decode_ms": 3270.7,
+    "decode_tok_s": 78.3,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714469.1007972,
+    "duration_s": 19.4,
+    "finish": "length",
+    "prompt_tokens": 34932,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 16553.9,
+    "decode_ms": 2826.7,
+    "decode_tok_s": 90.6,
+    "hit_rate": 0.941
+   },
+   {
+    "projection": null,
+    "time": 1790714463.1913157,
+    "duration_s": 5.2,
+    "finish": "length",
+    "prompt_tokens": 4201,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2365.6,
+    "decode_ms": 2868.4,
+    "decode_tok_s": 89.2,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714458.6299415,
+    "duration_s": 4.0,
+    "finish": "stop",
+    "prompt_tokens": 4200,
+    "reused": 0,
+    "output_tokens": 142,
+    "engine_generated": 142,
+    "prompt_ms": 2370.0,
+    "decode_ms": 1660.0,
+    "decode_tok_s": 85.5,
+    "hit_rate": 0.896
+   },
+   {
+    "projection": null,
+    "time": 1790714453.0791314,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 3881,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2223.8,
+    "decode_ms": 2790.5,
+    "decode_tok_s": 91.7,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790714448.5537117,
+    "duration_s": 4.0,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1305.1,
+    "decode_ms": 2690.8,
+    "decode_tok_s": 95.1,
+    "hit_rate": 0.972
+   },
+   {
+    "projection": null,
+    "time": 1790714443.8943996,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1149,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1297.6,
+    "decode_ms": 2847.5,
+    "decode_tok_s": 89.9,
+    "hit_rate": 0.971
+   },
+   {
+    "projection": null,
+    "time": 1790714439.2770011,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1092,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1351.4,
+    "decode_ms": 2752.6,
+    "decode_tok_s": 93.0,
+    "hit_rate": 0.954
+   },
+   {
+    "projection": null,
+    "time": 1790714377.2599747,
+    "duration_s": 17.6,
+    "finish": "length",
+    "prompt_tokens": 31605,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14608.5,
+    "decode_ms": 3015.6,
+    "decode_tok_s": 84.9,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714359.1307795,
+    "duration_s": 17.5,
+    "finish": "length",
+    "prompt_tokens": 31589,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14609.7,
+    "decode_ms": 2833.5,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714338.0043592,
+    "duration_s": 20.5,
+    "finish": "length",
+    "prompt_tokens": 34942,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 17033.3,
+    "decode_ms": 3435.7,
+    "decode_tok_s": 74.5,
+    "hit_rate": 0.945
+   }
+  ],
+  "requests_kept": 19,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 19,
+   "prompt_tokens": 227457,
+   "reused": 0,
+   "output_tokens": 3878,
+   "prompt_ms": 115435.4,
+   "decode_ms": 44775.099999999984
+  },
+  "hardware": {
+   "gpu_util": 94,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 47,
+   "gpu_power": 277.644,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 3986.787109375,
+   "gpu_pcie_tx_mb": 397.443359375,
+   "cpu": 99.9,
+   "ram_used": 63539171328,
+   "ram_total": 177175990272,
+   "disk_read_mb": 3.5686025775501298,
+   "disk_write_mb": 0.3811491758221081,
+   "tok_s": 96.24470767975828,
+   "tok_s_mean": 94.09593644569512
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0,
+    99,
+    100,
+    99,
+    96,
+    96,
+    0,
+    100,
+    99,
+    100,
+    100,
+    99,
+    100,
+    100,
+    99,
+    100,
+    99,
+    100,
+    99,
+    99,
+    99,
+    100,
+    98,
+    99,
+    97,
+    0,
+    99,
+    99,
+    99,
+    99,
+    99,
+    99,
+    98,
+    100,
+    99,
+    100,
+    99,
+    99,
+    99,
+    73,
+    98,
+    97,
+    94,
+    100,
+    100,
+    99,
+    99,
+    99,
+    99,
+    99,
+    100,
+    99,
+    100,
+    100,
+    100,
+    99,
+    99,
+    97,
+    97,
+    94
+   ],
+   "gpu_mem_used": [
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    42,
+    45,
+    45,
+    45,
+    46,
+    46,
+    42,
+    46,
+    46,
+    46,
+    46,
+    46,
+    45,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    43,
+    46,
+    47,
+    46,
+    46,
+    46,
+    47,
+    46,
+    47,
+    46,
+    47,
+    46,
+    46,
+    46,
+    44,
+    46,
+    46,
+    47,
+    46,
+    46,
+    47,
+    46,
+    46,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    47,
+    46,
+    47,
+    47
+   ],
+   "gpu_power": [
+    219.4,
+    271.43,
+    276.62,
+    251.66,
+    277.46,
+    278.06,
+    194.1,
+    265.98,
+    277.8,
+    274.38,
+    278.56,
+    277.99,
+    277.15,
+    264.89,
+    276.96,
+    277.37,
+    277.24,
+    276.83,
+    276.94,
+    278.15,
+    264.71,
+    273.73,
+    243.99,
+    277.71,
+    278.81,
+    203.42,
+    258.01,
+    277.13,
+    277.38,
+    277.05,
+    276.55,
+    275.55,
+    267.16,
+    280.52,
+    277.94,
+    278.68,
+    277.45,
+    276.52,
+    278.47,
+    255.06,
+    250.07,
+    273.21,
+    276.19,
+    200.05,
+    276.81,
+    276.45,
+    275.4,
+    277.81,
+    277.45,
+    277.14,
+    266.34,
+    281.85,
+    279.13,
+    278.52,
+    278.21,
+    279.47,
+    278.62,
+    244.74,
+    278.38,
+    277.64
+   ],
+   "gpu_pcie_rx_mb": [
+    6961.73,
+    24513.15,
+    26324.56,
+    679.21,
+    426.48,
+    0.33,
+    10.87,
+    10442.35,
+    23162.22,
+    22324.94,
+    24878.6,
+    26870.2,
+    14619.68,
+    11419.68,
+    893.4,
+    13515.52,
+    20953.08,
+    7.87,
+    14.59,
+    23972.7,
+    9329.51,
+    82.11,
+    7527.59,
+    4538.68,
+    5630.11,
+    6.67,
+    11788.08,
+    773.91,
+    87.02,
+    13.59,
+    4.53,
+    50.21,
+    7.15,
+    6744.72,
+    14.45,
+    115.11,
+    2575.49,
+    26114.03,
+    3364.04,
+    3375.15,
+    957.73,
+    2766.08,
+    0.38,
+    16.21,
+    26374.96,
+    15489.62,
+    25149.82,
+    11.68,
+    14.21,
+    14.21,
+    5900.76,
+    15.35,
+    3692.53,
+    87.59,
+    8.2,
+    25347.8,
+    80.63,
+    5132.53,
+    22.03,
+    3986.79
+   ],
+   "cpu": [
+    32.6,
+    6.8,
+    6.6,
+    96.7,
+    100.0,
+    83.5,
+    5.1,
+    7.7,
+    6.8,
+    5.8,
+    11.1,
+    14.6,
+    9.1,
+    8.4,
+    7.3,
+    7.4,
+    7.0,
+    7.1,
+    6.5,
+    5.9,
+    6.0,
+    6.9,
+    72.2,
+    100.0,
+    100.0,
+    19.3,
+    8.8,
+    6.1,
+    7.1,
+    6.6,
+    7.1,
+    6.0,
+    6.8,
+    5.6,
+    5.5,
+    6.6,
+    7.8,
+    6.1,
+    6.2,
+    41.1,
+    99.9,
+    99.4,
+    84.0,
+    4.7,
+    8.8,
+    7.4,
+    6.1,
+    6.1,
+    5.7,
+    6.3,
+    7.3,
+    6.8,
+    5.8,
+    8.4,
+    15.2,
+    18.1,
+    6.8,
+    56.4,
+    100.0,
+    99.9
+   ],
+   "ram_used": [
+    63399321600,
+    63389306880,
+    63427080192,
+    63418408960,
+    63417094144,
+    63416299520,
+    63358083072,
+    63351615488,
+    63342723072,
+    63336611840,
+    63371808768,
+    63426170880,
+    63504416768,
+    63594704896,
+    63611879424,
+    63612047360,
+    63599980544,
+    63573508096,
+    63570501632,
+    63568433152,
+    63702056960,
+    63700324352,
+    63800950784,
+    63801184256,
+    63789760512,
+    63567470592,
+    63559606272,
+    63544537088,
+    63536496640,
+    63529500672,
+    63515496448,
+    63496245248,
+    63530811392,
+    63529254912,
+    63530737664,
+    63536521216,
+    63543078912,
+    63537020928,
+    63539232768,
+    63659114496,
+    63677952000,
+    63679279104,
+    63683891200,
+    63526555648,
+    63547674624,
+    63533957120,
+    63515795456,
+    63502536704,
+    63493373952,
+    63485927424,
+    63556431872,
+    63548153856,
+    63549214720,
+    63593517056,
+    63649341440,
+    63479009280,
+    63476772864,
+    63577780224,
+    63546908672,
+    63539171328
+   ],
+   "disk_read_mb": [
+    1.17,
+    0.0,
+    0.0,
+    1.29,
+    3.73,
+    2.96,
+    0.82,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.0,
+    0.0,
+    0.23,
+    0.0,
+    0.0,
+    0.22,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.7,
+    0.31,
+    2.36,
+    0.74,
+    0.0,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.0,
+    0.34,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.15,
+    1.22,
+    1.87,
+    2.13,
+    0.67,
+    0.0,
+    0.0,
+    0.27,
+    0.0,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.45,
+    0.74,
+    3.57
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    93.47,
+    116.21,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    97.18,
+    97.71,
+    96.52,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    80.68,
+    75.74,
+    112.93,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    108.5,
+    87.33,
+    96.24
+   ]
+  },
+  "time": 1790714525.664817
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-4k-1.json

@@ -0,0 +1,829 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "4k",
+ "target_tokens": 4096,
+ "run": 1,
+ "run_id": "6c6e9cb15bc9",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:40:53",
+ "prompt_tokens": 3881,
+ "completion_tokens": 256,
+ "ttft_s": 2.267,
+ "e2e_s": 5.041,
+ "api_prompt_tps": 1711.6,
+ "api_decode_tps": 91.9,
+ "engine_prompt_tps": 1745.2,
+ "engine_decode_tps": 91.7,
+ "log_lines": [
+  "strata serve: prompt 3881 tokens = 0 reused + 3881 read in 2224 ms (1745.2 tok/s), 256 generated in 2791 ms (91.7 tok/s), drafts accepted 155 of 215, 1 checkpoints",
+  "strata serve: decode expert cache hit rate: 97.3% (146401 hits / 150495 lookups)"
+ ],
+ "answer_head": "The `CMakeLists.txt` file is the top-level build configuration for the **Strata Engine**. It orchestrates the compilation of the project, handling complex dependencies between CPU-only components, GPU kernels (CUDA/HIP), and testing infrastructure.\n\n### What the file does:\n\n1.  **Project Configurati",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714453.0791314,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 3881,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2223.8,
+    "decode_ms": 2790.5,
+    "decode_tok_s": 91.7,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790714448.5537117,
+    "duration_s": 4.0,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1305.1,
+    "decode_ms": 2690.8,
+    "decode_tok_s": 95.1,
+    "hit_rate": 0.972
+   },
+   {
+    "projection": null,
+    "time": 1790714443.8943996,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1149,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1297.6,
+    "decode_ms": 2847.5,
+    "decode_tok_s": 89.9,
+    "hit_rate": 0.971
+   },
+   {
+    "projection": null,
+    "time": 1790714439.2770011,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1092,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1351.4,
+    "decode_ms": 2752.6,
+    "decode_tok_s": 93.0,
+    "hit_rate": 0.954
+   },
+   {
+    "projection": null,
+    "time": 1790714377.2599747,
+    "duration_s": 17.6,
+    "finish": "length",
+    "prompt_tokens": 31605,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14608.5,
+    "decode_ms": 3015.6,
+    "decode_tok_s": 84.9,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714359.1307795,
+    "duration_s": 17.5,
+    "finish": "length",
+    "prompt_tokens": 31589,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14609.7,
+    "decode_ms": 2833.5,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714338.0043592,
+    "duration_s": 20.5,
+    "finish": "length",
+    "prompt_tokens": 34942,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 17033.3,
+    "decode_ms": 3435.7,
+    "decode_tok_s": 74.5,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714334.3235514,
+    "duration_s": 3.0,
+    "finish": "stop",
+    "prompt_tokens": 4199,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 2369.7,
+    "decode_ms": 626.3,
+    "decode_tok_s": 78.2,
+    "hit_rate": 0.934
+   },
+   {
+    "projection": null,
+    "time": 1790714328.2723713,
+    "duration_s": 5.5,
+    "finish": "length",
+    "prompt_tokens": 4214,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2436.3,
+    "decode_ms": 3084.4,
+    "decode_tok_s": 83.0,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714324.3220718,
+    "duration_s": 3.4,
+    "finish": "stop",
+    "prompt_tokens": 3736,
+    "reused": 0,
+    "output_tokens": 56,
+    "engine_generated": 56,
+    "prompt_ms": 2700.1,
+    "decode_ms": 719.2,
+    "decode_tok_s": 77.9,
+    "hit_rate": 0.897
+   },
+   {
+    "projection": null,
+    "time": 1790714293.0977688,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1288.1,
+    "decode_ms": 2834.1,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714290.8242896,
+    "duration_s": 1.8,
+    "finish": "stop",
+    "prompt_tokens": 1147,
+    "reused": 0,
+    "output_tokens": 40,
+    "engine_generated": 40,
+    "prompt_ms": 1304.0,
+    "decode_ms": 456.3,
+    "decode_tok_s": 87.7,
+    "hit_rate": 0.916
+   }
+  ],
+  "requests_kept": 14,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 14,
+   "prompt_tokens": 120940,
+   "reused": 0,
+   "output_tokens": 2712,
+   "prompt_ms": 64909.50000000001,
+   "decode_ms": 31364.099999999995
+  },
+  "hardware": {
+   "gpu_util": 97,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 46,
+   "gpu_power": 278.792,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.3330078125,
+   "gpu_pcie_tx_mb": 0.5244140625,
+   "cpu": 93.8,
+   "ram_used": 63471906816,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.6609869128828647,
+   "disk_write_mb": 0.0037478688913843163,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    100,
+    98,
+    94,
+    94,
+    100,
+    21,
+    98,
+    95,
+    53,
+    100,
+    94,
+    96,
+    21,
+    99,
+    99,
+    92,
+    97,
+    97
+   ],
+   "gpu_mem_used": [
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    42,
+    41,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    39,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    45,
+    45,
+    45,
+    45,
+    45,
+    43,
+    45,
+    45,
+    44,
+    45,
+    45,
+    45,
+    43,
+    45,
+    45,
+    45,
+    45,
+    46
+   ],
+   "gpu_power": [
+    164.95,
+    142.09,
+    130.43,
+    119.16,
+    119.05,
+    119.02,
+    118.92,
+    118.89,
+    118.85,
+    118.74,
+    118.81,
+    118.95,
+    118.87,
+    118.88,
+    119.01,
+    118.97,
+    113.29,
+    37.07,
+    35.2,
+    35.1,
+    35.04,
+    34.98,
+    34.86,
+    34.88,
+    34.97,
+    34.79,
+    34.76,
+    34.94,
+    34.9,
+    34.89,
+    35.0,
+    34.87,
+    34.82,
+    34.93,
+    34.86,
+    34.92,
+    34.99,
+    34.83,
+    34.88,
+    34.97,
+    34.89,
+    34.96,
+    172.41,
+    265.41,
+    275.33,
+    276.53,
+    216.14,
+    263.1,
+    264.26,
+    273.6,
+    211.21,
+    274.12,
+    265.58,
+    276.31,
+    268.45,
+    217.39,
+    278.78,
+    252.44,
+    278.12,
+    278.79
+   ],
+   "gpu_pcie_rx_mb": [
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.29,
+    0.29,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.57,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    9.97,
+    22329.52,
+    3689.24,
+    4264.07,
+    0.33,
+    14955.09,
+    784.16,
+    3310.82,
+    1249.69,
+    15874.15,
+    18357.04,
+    4099.13,
+    485.18,
+    0.29,
+    11.78,
+    84.59,
+    865.12,
+    8675.62,
+    0.33
+   ],
+   "cpu": [
+    2.5,
+    9.7,
+    2.8,
+    11.4,
+    13.4,
+    2.8,
+    1.5,
+    1.3,
+    2.8,
+    1.6,
+    1.6,
+    1.6,
+    2.2,
+    1.1,
+    5.2,
+    2.3,
+    1.3,
+    11.2,
+    3.6,
+    1.3,
+    11.5,
+    1.1,
+    1.9,
+    1.2,
+    1.3,
+    1.3,
+    1.0,
+    0.6,
+    2.3,
+    0.8,
+    1.2,
+    6.3,
+    8.9,
+    0.6,
+    1.9,
+    2.2,
+    2.1,
+    1.1,
+    2.2,
+    4.7,
+    1.2,
+    6.8,
+    7.3,
+    83.2,
+    99.8,
+    87.0,
+    14.8,
+    49.9,
+    100.0,
+    97.9,
+    33.3,
+    7.4,
+    99.9,
+    100.0,
+    63.3,
+    5.7,
+    6.4,
+    81.2,
+    100.0,
+    93.8
+   ],
+   "ram_used": [
+    63502053376,
+    63600914432,
+    63612030976,
+    63779962880,
+    63785050112,
+    63794016256,
+    63795236864,
+    63794536448,
+    63786823680,
+    63781208064,
+    63760637952,
+    63756046336,
+    63757750272,
+    63751917568,
+    63784116224,
+    63784988672,
+    63783927808,
+    63781187584,
+    63772962816,
+    63781842944,
+    63808417792,
+    63797518336,
+    63793590272,
+    63788277760,
+    63788392448,
+    63791640576,
+    63791734784,
+    63793213440,
+    63793565696,
+    63794352128,
+    63793430528,
+    63799361536,
+    63590793216,
+    63591645184,
+    63583899648,
+    63571738624,
+    63579156480,
+    63572910080,
+    63581016064,
+    63625879552,
+    63626788864,
+    63427158016,
+    63429017600,
+    63532883968,
+    63535853568,
+    63534440448,
+    63490953216,
+    63544176640,
+    63529631744,
+    63536766976,
+    63443234816,
+    63502606336,
+    63508336640,
+    63509970944,
+    63492075520,
+    63440269312,
+    63431364608,
+    63482470400,
+    63473512448,
+    63471906816
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.1,
+    0.14,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.8,
+    3.91,
+    3.26,
+    0.6,
+    0.0,
+    1.08,
+    2.82,
+    1.34,
+    0.0,
+    1.55,
+    2.4,
+    1.66,
+    0.64,
+    0.0,
+    2.25,
+    2.98,
+    2.66
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    103.98,
+    98.22,
+    0.0,
+    0.0,
+    112.83,
+    83.57,
+    76.97,
+    0.0,
+    0.0,
+    97.69,
+    87.72,
+    0.0,
+    0.0,
+    0.0,
+    102.14,
+    97.9,
+    0.0
+   ]
+  },
+  "time": 1790714458.598403
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-4k-2.json

@@ -0,0 +1,829 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "4k",
+ "target_tokens": 4096,
+ "run": 2,
+ "run_id": "1df8c923675b",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:40:58",
+ "prompt_tokens": 4200,
+ "completion_tokens": 142,
+ "ttft_s": 2.418,
+ "e2e_s": 4.063,
+ "api_prompt_tps": 1736.7,
+ "api_decode_tps": 85.7,
+ "engine_prompt_tps": 1772.2,
+ "engine_decode_tps": 85.5,
+ "log_lines": [
+  "strata serve: prompt 4200 tokens = 0 reused + 4200 read in 2370 ms (1772.2 tok/s), 142 generated in 1660 ms (85.5 tok/s), drafts accepted 96 of 114, 1 checkpoints",
+  "strata serve: decode expert cache hit rate: 89.6% (64163 hits / 71608 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand the context better.",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714458.6299415,
+    "duration_s": 4.0,
+    "finish": "stop",
+    "prompt_tokens": 4200,
+    "reused": 0,
+    "output_tokens": 142,
+    "engine_generated": 142,
+    "prompt_ms": 2370.0,
+    "decode_ms": 1660.0,
+    "decode_tok_s": 85.5,
+    "hit_rate": 0.896
+   },
+   {
+    "projection": null,
+    "time": 1790714453.0791314,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 3881,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2223.8,
+    "decode_ms": 2790.5,
+    "decode_tok_s": 91.7,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790714448.5537117,
+    "duration_s": 4.0,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1305.1,
+    "decode_ms": 2690.8,
+    "decode_tok_s": 95.1,
+    "hit_rate": 0.972
+   },
+   {
+    "projection": null,
+    "time": 1790714443.8943996,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1149,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1297.6,
+    "decode_ms": 2847.5,
+    "decode_tok_s": 89.9,
+    "hit_rate": 0.971
+   },
+   {
+    "projection": null,
+    "time": 1790714439.2770011,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1092,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1351.4,
+    "decode_ms": 2752.6,
+    "decode_tok_s": 93.0,
+    "hit_rate": 0.954
+   },
+   {
+    "projection": null,
+    "time": 1790714377.2599747,
+    "duration_s": 17.6,
+    "finish": "length",
+    "prompt_tokens": 31605,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14608.5,
+    "decode_ms": 3015.6,
+    "decode_tok_s": 84.9,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714359.1307795,
+    "duration_s": 17.5,
+    "finish": "length",
+    "prompt_tokens": 31589,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14609.7,
+    "decode_ms": 2833.5,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714338.0043592,
+    "duration_s": 20.5,
+    "finish": "length",
+    "prompt_tokens": 34942,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 17033.3,
+    "decode_ms": 3435.7,
+    "decode_tok_s": 74.5,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714334.3235514,
+    "duration_s": 3.0,
+    "finish": "stop",
+    "prompt_tokens": 4199,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 2369.7,
+    "decode_ms": 626.3,
+    "decode_tok_s": 78.2,
+    "hit_rate": 0.934
+   },
+   {
+    "projection": null,
+    "time": 1790714328.2723713,
+    "duration_s": 5.5,
+    "finish": "length",
+    "prompt_tokens": 4214,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2436.3,
+    "decode_ms": 3084.4,
+    "decode_tok_s": 83.0,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714324.3220718,
+    "duration_s": 3.4,
+    "finish": "stop",
+    "prompt_tokens": 3736,
+    "reused": 0,
+    "output_tokens": 56,
+    "engine_generated": 56,
+    "prompt_ms": 2700.1,
+    "decode_ms": 719.2,
+    "decode_tok_s": 77.9,
+    "hit_rate": 0.897
+   },
+   {
+    "projection": null,
+    "time": 1790714293.0977688,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1288.1,
+    "decode_ms": 2834.1,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.966
+   }
+  ],
+  "requests_kept": 15,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 15,
+   "prompt_tokens": 125140,
+   "reused": 0,
+   "output_tokens": 2854,
+   "prompt_ms": 67279.5,
+   "decode_ms": 33024.09999999999
+  },
+  "hardware": {
+   "gpu_util": 98,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 46,
+   "gpu_power": 276.727,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 4551.6005859375,
+   "gpu_pcie_tx_mb": 1681.041015625,
+   "cpu": 100.0,
+   "ram_used": 63441051648,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.097570021071815,
+   "disk_write_mb": 0.10077293695540748,
+   "tok_s": 98.25687730909294,
+   "tok_s_mean": 85.16861474502441
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    100,
+    98,
+    94,
+    94,
+    100,
+    21,
+    98,
+    95,
+    53,
+    100,
+    94,
+    96,
+    21,
+    99,
+    99,
+    92,
+    97,
+    97,
+    0,
+    98,
+    99,
+    98
+   ],
+   "gpu_mem_used": [
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    39,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    45,
+    45,
+    45,
+    45,
+    45,
+    43,
+    45,
+    45,
+    44,
+    45,
+    45,
+    45,
+    43,
+    45,
+    45,
+    45,
+    45,
+    46,
+    42,
+    46,
+    45,
+    46
+   ],
+   "gpu_power": [
+    119.05,
+    119.02,
+    118.92,
+    118.89,
+    118.85,
+    118.74,
+    118.81,
+    118.95,
+    118.87,
+    118.88,
+    119.01,
+    118.97,
+    113.29,
+    37.07,
+    35.2,
+    35.1,
+    35.04,
+    34.98,
+    34.86,
+    34.88,
+    34.97,
+    34.79,
+    34.76,
+    34.94,
+    34.9,
+    34.89,
+    35.0,
+    34.87,
+    34.82,
+    34.93,
+    34.86,
+    34.92,
+    34.99,
+    34.83,
+    34.88,
+    34.97,
+    34.89,
+    34.96,
+    172.41,
+    265.41,
+    275.33,
+    276.53,
+    216.14,
+    263.1,
+    264.26,
+    273.6,
+    211.21,
+    274.12,
+    265.58,
+    276.31,
+    268.45,
+    217.39,
+    278.78,
+    252.44,
+    278.12,
+    278.79,
+    215.59,
+    278.37,
+    251.8,
+    276.73
+   ],
+   "gpu_pcie_rx_mb": [
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.33,
+    0.29,
+    0.29,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.57,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    9.97,
+    22329.52,
+    3689.24,
+    4264.07,
+    0.33,
+    14955.09,
+    784.16,
+    3310.82,
+    1249.69,
+    15874.15,
+    18357.04,
+    4099.13,
+    485.18,
+    0.29,
+    11.78,
+    84.59,
+    865.12,
+    8675.62,
+    0.33,
+    3342.49,
+    34.05,
+    7461.5,
+    4551.6
+   ],
+   "cpu": [
+    13.4,
+    2.8,
+    1.5,
+    1.3,
+    2.8,
+    1.6,
+    1.6,
+    1.6,
+    2.2,
+    1.1,
+    5.2,
+    2.3,
+    1.3,
+    11.2,
+    3.6,
+    1.3,
+    11.5,
+    1.1,
+    1.9,
+    1.2,
+    1.3,
+    1.3,
+    1.0,
+    0.6,
+    2.3,
+    0.8,
+    1.2,
+    6.3,
+    8.9,
+    0.6,
+    1.9,
+    2.2,
+    2.1,
+    1.1,
+    2.2,
+    4.7,
+    1.2,
+    6.8,
+    7.3,
+    83.2,
+    99.8,
+    87.0,
+    14.8,
+    49.9,
+    100.0,
+    97.9,
+    33.3,
+    7.4,
+    99.9,
+    100.0,
+    63.3,
+    5.7,
+    6.4,
+    81.2,
+    100.0,
+    93.8,
+    14.8,
+    15.8,
+    37.5,
+    100.0
+   ],
+   "ram_used": [
+    63785050112,
+    63794016256,
+    63795236864,
+    63794536448,
+    63786823680,
+    63781208064,
+    63760637952,
+    63756046336,
+    63757750272,
+    63751917568,
+    63784116224,
+    63784988672,
+    63783927808,
+    63781187584,
+    63772962816,
+    63781842944,
+    63808417792,
+    63797518336,
+    63793590272,
+    63788277760,
+    63788392448,
+    63791640576,
+    63791734784,
+    63793213440,
+    63793565696,
+    63794352128,
+    63793430528,
+    63799361536,
+    63590793216,
+    63591645184,
+    63583899648,
+    63571738624,
+    63579156480,
+    63572910080,
+    63581016064,
+    63625879552,
+    63626788864,
+    63427158016,
+    63429017600,
+    63532883968,
+    63535853568,
+    63534440448,
+    63490953216,
+    63544176640,
+    63529631744,
+    63536766976,
+    63443234816,
+    63502606336,
+    63508336640,
+    63509970944,
+    63492075520,
+    63440269312,
+    63431364608,
+    63482470400,
+    63473512448,
+    63471906816,
+    63401848832,
+    63554486272,
+    63453114368,
+    63441051648
+   ],
+   "disk_read_mb": [
+    0.1,
+    0.14,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.8,
+    3.91,
+    3.26,
+    0.6,
+    0.0,
+    1.08,
+    2.82,
+    1.34,
+    0.0,
+    1.55,
+    2.4,
+    1.66,
+    0.64,
+    0.0,
+    2.25,
+    2.98,
+    2.66,
+    0.64,
+    0.0,
+    0.21,
+    2.1
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    103.98,
+    98.22,
+    0.0,
+    0.0,
+    112.83,
+    83.57,
+    76.97,
+    0.0,
+    0.0,
+    97.69,
+    87.72,
+    0.0,
+    0.0,
+    0.0,
+    102.14,
+    97.9,
+    0.0,
+    0.0,
+    0.0,
+    89.96,
+    98.26
+   ]
+  },
+  "time": 1790714463.1652913
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-1x3090/speed-4k-3.json

@@ -0,0 +1,829 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-1x3090",
+ "mode": "speed",
+ "ctx": "4k",
+ "target_tokens": 4096,
+ "run": 3,
+ "run_id": "777f84f22df4",
+ "corpus_repeated": false,
+ "time": "2026-09-29T20:41:03",
+ "prompt_tokens": 4201,
+ "completion_tokens": 256,
+ "ttft_s": 2.409,
+ "e2e_s": 5.262,
+ "api_prompt_tps": 1744.1,
+ "api_decode_tps": 89.4,
+ "engine_prompt_tps": 1775.8,
+ "engine_decode_tps": 89.2,
+ "log_lines": [
+  "strata serve: prompt 4201 tokens = 0 reused + 4201 read in 2366 ms (1775.8 tok/s), 256 generated in 2868 ms (89.2 tok/s), drafts accepted 161 of 222, 1 checkpoints",
+  "strata serve: decode expert cache hit rate: 96.0% (144002 hits / 149996 lookups)"
+ ],
+ "answer_head": "The `CMakeLists.txt` file is the top-level build configuration for the **Strata Engine**. Its primary responsibilities are:\n\n1.  **Project Definition**: Sets up the project name (`strata`), version (`0.1.26`), and C++20 standard.\n2.  **Backend Selection**: Manages mutually exclusive GPU backends (CU",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 131072,
+   "images": false,
+   "version": "0.1.26",
+   "context": 131072,
+   "kv": "int8",
+   "kv_resident": 32768,
+   "expert_slots": 10584,
+   "expert_cache_mib": 17583,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 389,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790714463.1913157,
+    "duration_s": 5.2,
+    "finish": "length",
+    "prompt_tokens": 4201,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2365.6,
+    "decode_ms": 2868.4,
+    "decode_tok_s": 89.2,
+    "hit_rate": 0.96
+   },
+   {
+    "projection": null,
+    "time": 1790714458.6299415,
+    "duration_s": 4.0,
+    "finish": "stop",
+    "prompt_tokens": 4200,
+    "reused": 0,
+    "output_tokens": 142,
+    "engine_generated": 142,
+    "prompt_ms": 2370.0,
+    "decode_ms": 1660.0,
+    "decode_tok_s": 85.5,
+    "hit_rate": 0.896
+   },
+   {
+    "projection": null,
+    "time": 1790714453.0791314,
+    "duration_s": 5.0,
+    "finish": "length",
+    "prompt_tokens": 3881,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2223.8,
+    "decode_ms": 2790.5,
+    "decode_tok_s": 91.7,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790714448.5537117,
+    "duration_s": 4.0,
+    "finish": "length",
+    "prompt_tokens": 1135,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1305.1,
+    "decode_ms": 2690.8,
+    "decode_tok_s": 95.1,
+    "hit_rate": 0.972
+   },
+   {
+    "projection": null,
+    "time": 1790714443.8943996,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1149,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1297.6,
+    "decode_ms": 2847.5,
+    "decode_tok_s": 89.9,
+    "hit_rate": 0.971
+   },
+   {
+    "projection": null,
+    "time": 1790714439.2770011,
+    "duration_s": 4.1,
+    "finish": "length",
+    "prompt_tokens": 1092,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 1351.4,
+    "decode_ms": 2752.6,
+    "decode_tok_s": 93.0,
+    "hit_rate": 0.954
+   },
+   {
+    "projection": null,
+    "time": 1790714377.2599747,
+    "duration_s": 17.6,
+    "finish": "length",
+    "prompt_tokens": 31605,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14608.5,
+    "decode_ms": 3015.6,
+    "decode_tok_s": 84.9,
+    "hit_rate": 0.966
+   },
+   {
+    "projection": null,
+    "time": 1790714359.1307795,
+    "duration_s": 17.5,
+    "finish": "length",
+    "prompt_tokens": 31589,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 14609.7,
+    "decode_ms": 2833.5,
+    "decode_tok_s": 90.3,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714338.0043592,
+    "duration_s": 20.5,
+    "finish": "length",
+    "prompt_tokens": 34942,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 17033.3,
+    "decode_ms": 3435.7,
+    "decode_tok_s": 74.5,
+    "hit_rate": 0.945
+   },
+   {
+    "projection": null,
+    "time": 1790714334.3235514,
+    "duration_s": 3.0,
+    "finish": "stop",
+    "prompt_tokens": 4199,
+    "reused": 0,
+    "output_tokens": 49,
+    "engine_generated": 49,
+    "prompt_ms": 2369.7,
+    "decode_ms": 626.3,
+    "decode_tok_s": 78.2,
+    "hit_rate": 0.934
+   },
+   {
+    "projection": null,
+    "time": 1790714328.2723713,
+    "duration_s": 5.5,
+    "finish": "length",
+    "prompt_tokens": 4214,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 2436.3,
+    "decode_ms": 3084.4,
+    "decode_tok_s": 83.0,
+    "hit_rate": 0.959
+   },
+   {
+    "projection": null,
+    "time": 1790714324.3220718,
+    "duration_s": 3.4,
+    "finish": "stop",
+    "prompt_tokens": 3736,
+    "reused": 0,
+    "output_tokens": 56,
+    "engine_generated": 56,
+    "prompt_ms": 2700.1,
+    "decode_ms": 719.2,
+    "decode_tok_s": 77.9,
+    "hit_rate": 0.897
+   }
+  ],
+  "requests_kept": 16,
+  "totals": {
+   "since": 1790713882.6043305,
+   "requests": 16,
+   "prompt_tokens": 129341,
+   "reused": 0,
+   "output_tokens": 3110,
+   "prompt_ms": 69645.1,
+   "decode_ms": 35892.49999999999
+  },
+  "hardware": {
+   "gpu_util": 96,
+   "gpu_mem_used": 25517031424,
+   "gpu_mem_total": 25769803776,
+   "gpu_temp": 46,
+   "gpu_power": 278.057,
+   "gpu_power_limit": 280.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.3330078125,
+   "gpu_pcie_tx_mb": 0.4287109375,
+   "cpu": 83.5,
+   "ram_used": 63416299520,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.956676951508761,
+   "disk_write_mb": 0.16488439273306146,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090",
+   "gpu_count": 1,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    100,
+    98,
+    94,
+    94,
+    100,
+    21,
+    98,
+    95,
+    53,
+    100,
+    94,
+    96,
+    21,
+    99,
+    99,
+    92,
+    97,
+    97,
+    0,
+    98,
+    99,
+    98,
+    0,
+    99,
+    100,
+    99,
+    96,
+    96
+   ],
+   "gpu_mem_used": [
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25514934272,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424,
+    25517031424
+   ],
+   "gpu_temp": [
+    40,
+    40,
+    40,
+    40,
+    40,
+    40,
+    39,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    38,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    37,
+    45,
+    45,
+    45,
+    45,
+    45,
+    43,
+    45,
+    45,
+    44,
+    45,
+    45,
+    45,
+    43,
+    45,
+    45,
+    45,
+    45,
+    46,
+    42,
+    46,
+    45,
+    46,
+    42,
+    45,
+    45,
+    45,
+    46,
+    46
+   ],
+   "gpu_power": [
+    118.81,
+    118.95,
+    118.87,
+    118.88,
+    119.01,
+    118.97,
+    113.29,
+    37.07,
+    35.2,
+    35.1,
+    35.04,
+    34.98,
+    34.86,
+    34.88,
+    34.97,
+    34.79,
+    34.76,
+    34.94,
+    34.9,
+    34.89,
+    35.0,
+    34.87,
+    34.82,
+    34.93,
+    34.86,
+    34.92,
+    34.99,
+    34.83,
+    34.88,
+    34.97,
+    34.89,
+    34.96,
+    172.41,
+    265.41,
+    275.33,
+    276.53,
+    216.14,
+    263.1,
+    264.26,
+    273.6,
+    211.21,
+    274.12,
+    265.58,
+    276.31,
+    268.45,
+    217.39,
+    278.78,
+    252.44,
+    278.12,
+    278.79,
+    215.59,
+    278.37,
+    251.8,
+    276.73,
+    219.4,
+    271.43,
+    276.62,
+    251.66,
+    277.46,
+    278.06
+   ],
+   "gpu_pcie_rx_mb": [
+    0.29,
+    0.29,
+    0.29,
+    0.33,
+    0.33,
+    0.33,
+    0.57,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.43,
+    0.38,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    0.43,
+    0.38,
+    0.38,
+    9.97,
+    22329.52,
+    3689.24,
+    4264.07,
+    0.33,
+    14955.09,
+    784.16,
+    3310.82,
+    1249.69,
+    15874.15,
+    18357.04,
+    4099.13,
+    485.18,
+    0.29,
+    11.78,
+    84.59,
+    865.12,
+    8675.62,
+    0.33,
+    3342.49,
+    34.05,
+    7461.5,
+    4551.6,
+    6961.73,
+    24513.15,
+    26324.56,
+    679.21,
+    426.48,
+    0.33
+   ],
+   "cpu": [
+    1.6,
+    1.6,
+    2.2,
+    1.1,
+    5.2,
+    2.3,
+    1.3,
+    11.2,
+    3.6,
+    1.3,
+    11.5,
+    1.1,
+    1.9,
+    1.2,
+    1.3,
+    1.3,
+    1.0,
+    0.6,
+    2.3,
+    0.8,
+    1.2,
+    6.3,
+    8.9,
+    0.6,
+    1.9,
+    2.2,
+    2.1,
+    1.1,
+    2.2,
+    4.7,
+    1.2,
+    6.8,
+    7.3,
+    83.2,
+    99.8,
+    87.0,
+    14.8,
+    49.9,
+    100.0,
+    97.9,
+    33.3,
+    7.4,
+    99.9,
+    100.0,
+    63.3,
+    5.7,
+    6.4,
+    81.2,
+    100.0,
+    93.8,
+    14.8,
+    15.8,
+    37.5,
+    100.0,
+    32.6,
+    6.8,
+    6.6,
+    96.7,
+    100.0,
+    83.5
+   ],
+   "ram_used": [
+    63760637952,
+    63756046336,
+    63757750272,
+    63751917568,
+    63784116224,
+    63784988672,
+    63783927808,
+    63781187584,
+    63772962816,
+    63781842944,
+    63808417792,
+    63797518336,
+    63793590272,
+    63788277760,
+    63788392448,
+    63791640576,
+    63791734784,
+    63793213440,
+    63793565696,
+    63794352128,
+    63793430528,
+    63799361536,
+    63590793216,
+    63591645184,
+    63583899648,
+    63571738624,
+    63579156480,
+    63572910080,
+    63581016064,
+    63625879552,
+    63626788864,
+    63427158016,
+    63429017600,
+    63532883968,
+    63535853568,
+    63534440448,
+    63490953216,
+    63544176640,
+    63529631744,
+    63536766976,
+    63443234816,
+    63502606336,
+    63508336640,
+    63509970944,
+    63492075520,
+    63440269312,
+    63431364608,
+    63482470400,
+    63473512448,
+    63471906816,
+    63401848832,
+    63554486272,
+    63453114368,
+    63441051648,
+    63399321600,
+    63389306880,
+    63427080192,
+    63418408960,
+    63417094144,
+    63416299520
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.35,
+    0.0,
+    0.8,
+    3.91,
+    3.26,
+    0.6,
+    0.0,
+    1.08,
+    2.82,
+    1.34,
+    0.0,
+    1.55,
+    2.4,
+    1.66,
+    0.64,
+    0.0,
+    2.25,
+    2.98,
+    2.66,
+    0.64,
+    0.0,
+    0.21,
+    2.1,
+    1.17,
+    0.0,
+    0.0,
+    1.29,
+    3.73,
+    2.96
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    103.98,
+    98.22,
+    0.0,
+    0.0,
+    112.83,
+    83.57,
+    76.97,
+    0.0,
+    0.0,
+    97.69,
+    87.72,
+    0.0,
+    0.0,
+    0.0,
+    102.14,
+    97.9,
+    0.0,
+    0.0,
+    0.0,
+    89.96,
+    98.26,
+    0.0,
+    0.0,
+    0.0,
+    93.47,
+    116.21,
+    0.0
+   ]
+  },
+  "time": 1790714468.9308503
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-2x3090-auto-needle/needle-16k-1.json

@@ -0,0 +1,501 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-2x3090-auto-needle",
+ "mode": "needle",
+ "ctx": "16k",
+ "target_tokens": 16384,
+ "run": 1,
+ "run_id": "8f89dfed9af3",
+ "corpus_repeated": false,
+ "time": "2026-09-29T22:00:26",
+ "prompt_tokens": 17344,
+ "completion_tokens": 8,
+ "ttft_s": 15.582,
+ "e2e_s": 15.663,
+ "api_prompt_tps": 1113.1,
+ "api_decode_tps": 86.5,
+ "engine_prompt_tps": 1119.9,
+ "engine_decode_tps": 83.8,
+ "log_lines": [
+  "strata serve: prompt 17344 tokens = 0 reused + 17344 read in 15487 ms (1119.9 tok/s), 8 generated in 96 ms (83.8 tok/s), drafts accepted 6 of 6, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 97.3% (4205 hits / 4320 lookups)"
+ ],
+ "answer_head": "05449BFC",
+ "needle_found": true,
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 11883,
+   "expert_cache_mib": 17564,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 645,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790719226.530526,
+    "duration_s": 15.6,
+    "finish": "stop",
+    "prompt_tokens": 17344,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 15486.8,
+    "decode_ms": 95.5,
+    "decode_tok_s": 83.8,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790719217.9333243,
+    "duration_s": 8.0,
+    "finish": "stop",
+    "prompt_tokens": 7493,
+    "reused": 0,
+    "output_tokens": 9,
+    "engine_generated": 9,
+    "prompt_ms": 7861.5,
+    "decode_ms": 153.7,
+    "decode_tok_s": 58.6,
+    "hit_rate": 0.94
+   }
+  ],
+  "requests_kept": 2,
+  "totals": {
+   "since": 1790719202.7795203,
+   "requests": 2,
+   "prompt_tokens": 24837,
+   "reused": 0,
+   "output_tokens": 17,
+   "prompt_ms": 23348.3,
+   "decode_ms": 249.2
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 0,
+     "mem_used": 25171001344,
+     "mem_total": 25769803776,
+     "temp": 47,
+     "power": 148.05
+    },
+    {
+     "index": 1,
+     "util": 40,
+     "mem_used": 24185339904,
+     "mem_total": 25769803776,
+     "temp": 54,
+     "power": 237.219
+    }
+   ],
+   "gpu_util": 20.0,
+   "gpu_mem_used": 49356341248,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 54,
+   "gpu_power": 385.269,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 1.7626953125,
+   "gpu_pcie_tx_mb": 0.9052734375,
+   "cpu": 9.1,
+   "ram_used": 62596341760,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.0,
+   "disk_write_mb": 0.0,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.5,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    2.0,
+    73.5,
+    23.5,
+    31.0,
+    56.5,
+    23.5,
+    20.5,
+    0.5,
+    70.5,
+    22.5,
+    26.5,
+    56.0,
+    29.0,
+    77.0,
+    11.0,
+    28.5,
+    59.5,
+    28.5,
+    19.5,
+    17.0,
+    27.0,
+    20.0
+   ],
+   "gpu_mem_used": [
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49274552320,
+    49291329536,
+    49291329536,
+    49291329536,
+    49291329536,
+    49291329536,
+    49291329536,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248
+   ],
+   "gpu_temp": [
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    51,
+    53,
+    52,
+    52,
+    52,
+    53,
+    49,
+    51,
+    52,
+    52,
+    53,
+    53,
+    53,
+    52,
+    53,
+    51,
+    53,
+    53,
+    53,
+    53,
+    54
+   ],
+   "gpu_power": [
+    257.56,
+    245.14,
+    244.59,
+    244.41,
+    244.51,
+    244.45,
+    244.36,
+    244.36,
+    244.26,
+    244.16,
+    244.27,
+    244.18,
+    244.09,
+    243.92,
+    243.88,
+    245.24,
+    443.06,
+    409.13,
+    432.67,
+    433.1,
+    400.92,
+    389.08,
+    278.03,
+    383.36,
+    392.74,
+    440.94,
+    403.62,
+    421.29,
+    427.09,
+    407.67,
+    422.11,
+    423.13,
+    417.76,
+    440.15,
+    377.19,
+    403.8,
+    385.27
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.62,
+    0.57,
+    0.62,
+    0.67,
+    0.57,
+    0.62,
+    0.62,
+    0.67,
+    0.62,
+    0.62,
+    0.67,
+    0.62,
+    0.57,
+    0.62,
+    2169.46,
+    6179.19,
+    3838.3,
+    4927.82,
+    4008.2,
+    7314.58,
+    0.95,
+    1338.19,
+    7892.51,
+    3024.72,
+    5741.74,
+    2921.96,
+    5454.83,
+    5667.49,
+    3768.87,
+    6803.13,
+    8164.64,
+    2711.1,
+    5725.29,
+    6417.89,
+    0.67,
+    1.76
+   ],
+   "cpu": [
+    0.0,
+    15.8,
+    15.7,
+    6.4,
+    8.0,
+    8.1,
+    4.7,
+    6.7,
+    0.9,
+    1.5,
+    5.3,
+    12.5,
+    1.6,
+    3.2,
+    8.4,
+    14.2,
+    13.2,
+    10.5,
+    10.5,
+    13.4,
+    9.0,
+    12.2,
+    23.9,
+    8.1,
+    16.5,
+    13.0,
+    13.0,
+    9.0,
+    9.7,
+    12.8,
+    8.6,
+    12.9,
+    10.4,
+    8.6,
+    9.0,
+    9.4,
+    9.1
+   ],
+   "ram_used": [
+    61827047424,
+    61871300608,
+    61941383168,
+    61960540160,
+    61978361856,
+    61977542656,
+    61984432128,
+    61992636416,
+    61992636416,
+    62000824320,
+    62002450432,
+    62005186560,
+    62006108160,
+    62014894080,
+    62023741440,
+    62230065152,
+    62246010880,
+    62250319872,
+    62234144768,
+    62227582976,
+    62226890752,
+    62392664064,
+    62490071040,
+    62486704128,
+    62515294208,
+    62517342208,
+    62467452928,
+    62510080000,
+    62495842304,
+    62471684096,
+    62463819776,
+    62459879424,
+    62443171840,
+    62374465536,
+    62303469568,
+    62390607872,
+    62596341760
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    28.33,
+    101.36,
+    41.6,
+    10.77,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    1.06,
+    0.0,
+    14.72,
+    0.0,
+    56.04,
+    0.0,
+    33.79,
+    66.18,
+    16.29,
+    83.86,
+    33.38,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790719242.619913
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-2x3090-auto-needle/needle-28k-1.json

@@ -0,0 +1,722 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-2x3090-auto-needle",
+ "mode": "needle",
+ "ctx": "28k",
+ "target_tokens": 28672,
+ "run": 1,
+ "run_id": "93a9a0e32114",
+ "corpus_repeated": false,
+ "time": "2026-09-29T22:00:42",
+ "prompt_tokens": 29825,
+ "completion_tokens": 9,
+ "ttft_s": 26.19,
+ "e2e_s": 26.262,
+ "api_prompt_tps": 1138.8,
+ "api_decode_tps": 110.5,
+ "engine_prompt_tps": 1146.7,
+ "engine_decode_tps": 103.9,
+ "log_lines": [
+  "strata serve: prompt 29825 tokens = 0 reused + 29825 read in 26009 ms (1146.7 tok/s), 9 generated in 87 ms (103.9 tok/s), drafts accepted 6 of 6, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 98.2% (4241 hits / 4320 lookups)"
+ ],
+ "answer_head": "98C00B03",
+ "needle_found": true,
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 11883,
+   "expert_cache_mib": 17564,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 645,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790719242.7790127,
+    "duration_s": 26.1,
+    "finish": "stop",
+    "prompt_tokens": 29825,
+    "reused": 0,
+    "output_tokens": 9,
+    "engine_generated": 9,
+    "prompt_ms": 26008.5,
+    "decode_ms": 86.6,
+    "decode_tok_s": 103.9,
+    "hit_rate": 0.982
+   },
+   {
+    "projection": null,
+    "time": 1790719226.530526,
+    "duration_s": 15.6,
+    "finish": "stop",
+    "prompt_tokens": 17344,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 15486.8,
+    "decode_ms": 95.5,
+    "decode_tok_s": 83.8,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790719217.9333243,
+    "duration_s": 8.0,
+    "finish": "stop",
+    "prompt_tokens": 7493,
+    "reused": 0,
+    "output_tokens": 9,
+    "engine_generated": 9,
+    "prompt_ms": 7861.5,
+    "decode_ms": 153.7,
+    "decode_tok_s": 58.6,
+    "hit_rate": 0.94
+   }
+  ],
+  "requests_kept": 3,
+  "totals": {
+   "since": 1790719202.7795203,
+   "requests": 3,
+   "prompt_tokens": 54662,
+   "reused": 0,
+   "output_tokens": 26,
+   "prompt_ms": 49356.8,
+   "decode_ms": 335.79999999999995
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 6,
+     "mem_used": 25171001344,
+     "mem_total": 25769803776,
+     "temp": 47,
+     "power": 141.832
+    },
+    {
+     "index": 1,
+     "util": 20,
+     "mem_used": 24185339904,
+     "mem_total": 25769803776,
+     "temp": 52,
+     "power": 243.782
+    }
+   ],
+   "gpu_util": 13.0,
+   "gpu_mem_used": 49356341248,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 52,
+   "gpu_power": 385.61400000000003,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.666015625,
+   "gpu_pcie_tx_mb": 0.857421875,
+   "cpu": 16.6,
+   "ram_used": 62951010304,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.08651659039933747,
+   "disk_write_mb": 0.0,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    2.0,
+    73.5,
+    23.5,
+    31.0,
+    56.5,
+    23.5,
+    20.5,
+    0.5,
+    70.5,
+    22.5,
+    26.5,
+    56.0,
+    29.0,
+    77.0,
+    11.0,
+    28.5,
+    59.5,
+    28.5,
+    19.5,
+    17.0,
+    27.0,
+    20.0,
+    0.0,
+    60.5,
+    25.5,
+    71.0,
+    67.0,
+    26.5,
+    26.5,
+    85.0,
+    26.0,
+    73.5,
+    49.5,
+    25.0,
+    48.0,
+    32.0,
+    76.5,
+    9.0,
+    27.5,
+    47.5,
+    30.0,
+    62.5,
+    15.5,
+    31.5,
+    29.0,
+    39.5,
+    13.0
+   ],
+   "gpu_mem_used": [
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49274552320,
+    49291329536,
+    49291329536,
+    49291329536,
+    49291329536,
+    49291329536,
+    49291329536,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248
+   ],
+   "gpu_temp": [
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    51,
+    53,
+    52,
+    52,
+    52,
+    53,
+    49,
+    51,
+    52,
+    52,
+    53,
+    53,
+    53,
+    52,
+    53,
+    51,
+    53,
+    53,
+    53,
+    53,
+    54,
+    50,
+    51,
+    53,
+    53,
+    52,
+    54,
+    52,
+    52,
+    53,
+    53,
+    53,
+    53,
+    52,
+    53,
+    54,
+    52,
+    53,
+    52,
+    53,
+    52,
+    52,
+    54,
+    53,
+    54,
+    52
+   ],
+   "gpu_power": [
+    244.59,
+    244.41,
+    244.51,
+    244.45,
+    244.36,
+    244.36,
+    244.26,
+    244.16,
+    244.27,
+    244.18,
+    244.09,
+    243.92,
+    243.88,
+    245.24,
+    443.06,
+    409.13,
+    432.67,
+    433.1,
+    400.92,
+    389.08,
+    278.03,
+    383.36,
+    392.74,
+    440.94,
+    403.62,
+    421.29,
+    427.09,
+    407.67,
+    422.11,
+    423.13,
+    417.76,
+    440.15,
+    377.19,
+    403.8,
+    385.27,
+    299.57,
+    398.78,
+    401.71,
+    449.87,
+    430.87,
+    402.83,
+    439.43,
+    401.46,
+    455.62,
+    425.18,
+    404.39,
+    417.35,
+    406.87,
+    423.67,
+    422.86,
+    404.6,
+    421.5,
+    402.4,
+    422.71,
+    432.67,
+    398.05,
+    455.72,
+    434.46,
+    403.83,
+    385.61
+   ],
+   "gpu_pcie_rx_mb": [
+    0.57,
+    0.62,
+    0.67,
+    0.57,
+    0.62,
+    0.62,
+    0.67,
+    0.62,
+    0.62,
+    0.67,
+    0.62,
+    0.57,
+    0.62,
+    2169.46,
+    6179.19,
+    3838.3,
+    4927.82,
+    4008.2,
+    7314.58,
+    0.95,
+    1338.19,
+    7892.51,
+    3024.72,
+    5741.74,
+    2921.96,
+    5454.83,
+    5667.49,
+    3768.87,
+    6803.13,
+    8164.64,
+    2711.1,
+    5725.29,
+    6417.89,
+    0.67,
+    1.76,
+    1343.92,
+    7796.67,
+    10809.8,
+    5598.92,
+    3013.99,
+    91.89,
+    4693.79,
+    5621.38,
+    5381.87,
+    7487.25,
+    2228.12,
+    7344.77,
+    3943.44,
+    0.81,
+    5476.19,
+    2437.4,
+    6693.74,
+    3836.63,
+    7843.59,
+    5638.65,
+    2853.11,
+    8215.43,
+    8500.96,
+    4445.79,
+    0.67
+   ],
+   "cpu": [
+    15.7,
+    6.4,
+    8.0,
+    8.1,
+    4.7,
+    6.7,
+    0.9,
+    1.5,
+    5.3,
+    12.5,
+    1.6,
+    3.2,
+    8.4,
+    14.2,
+    13.2,
+    10.5,
+    10.5,
+    13.4,
+    9.0,
+    12.2,
+    23.9,
+    8.1,
+    16.5,
+    13.0,
+    13.0,
+    9.0,
+    9.7,
+    12.8,
+    8.6,
+    12.9,
+    10.4,
+    8.6,
+    9.0,
+    9.4,
+    9.1,
+    16.9,
+    17.1,
+    22.7,
+    7.5,
+    10.8,
+    8.9,
+    7.3,
+    7.3,
+    8.3,
+    7.7,
+    9.8,
+    6.9,
+    12.4,
+    6.4,
+    12.2,
+    20.6,
+    19.9,
+    11.0,
+    8.1,
+    10.3,
+    9.8,
+    7.6,
+    8.1,
+    7.3,
+    16.6
+   ],
+   "ram_used": [
+    61941383168,
+    61960540160,
+    61978361856,
+    61977542656,
+    61984432128,
+    61992636416,
+    61992636416,
+    62000824320,
+    62002450432,
+    62005186560,
+    62006108160,
+    62014894080,
+    62023741440,
+    62230065152,
+    62246010880,
+    62250319872,
+    62234144768,
+    62227582976,
+    62226890752,
+    62392664064,
+    62490071040,
+    62486704128,
+    62515294208,
+    62517342208,
+    62467452928,
+    62510080000,
+    62495842304,
+    62471684096,
+    62463819776,
+    62459879424,
+    62443171840,
+    62374465536,
+    62303469568,
+    62390607872,
+    62596341760,
+    62489825280,
+    62562496512,
+    62577270784,
+    62577836032,
+    62576951296,
+    62564986880,
+    62553812992,
+    62539886592,
+    62533107712,
+    62521810944,
+    62503256064,
+    62503677952,
+    62490673152,
+    62492131328,
+    62495432704,
+    62503706624,
+    62769029120,
+    62766465024,
+    62765707264,
+    62747688960,
+    62726733824,
+    62717767680,
+    62722011136,
+    62728314880,
+    62951010304
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    28.33,
+    101.36,
+    41.6,
+    10.77,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    1.06,
+    0.0,
+    14.72,
+    0.0,
+    56.04,
+    0.0,
+    33.79,
+    66.18,
+    16.29,
+    83.86,
+    33.38,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.73,
+    0.0,
+    0.0,
+    0.0,
+    0.22,
+    0.0,
+    0.0,
+    0.14,
+    0.0,
+    0.53,
+    47.85,
+    0.0,
+    71.37,
+    0.0,
+    78.98,
+    55.42,
+    22.25,
+    53.69,
+    0.0,
+    54.32,
+    39.07,
+    0.0,
+    0.0,
+    0.0,
+    0.09
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790719269.3852785
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-2x3090-auto-needle/needle-8k-1.json

@@ -0,0 +1,352 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-2x3090-auto-needle",
+ "mode": "needle",
+ "ctx": "8k",
+ "target_tokens": 8192,
+ "run": 1,
+ "run_id": "d1d745f05911",
+ "corpus_repeated": false,
+ "time": "2026-09-29T22:00:17",
+ "prompt_tokens": 7493,
+ "completion_tokens": 9,
+ "ttft_s": 7.948,
+ "e2e_s": 8.057,
+ "api_prompt_tps": 942.8,
+ "api_decode_tps": 73.4,
+ "engine_prompt_tps": 953.1,
+ "engine_decode_tps": 58.6,
+ "log_lines": [
+  "strata serve: prompt 7493 tokens = 0 reused + 7493 read in 7861 ms (953.1 tok/s), 9 generated in 154 ms (58.6 tok/s), drafts accepted 6 of 6, 1 checkpoints",
+  "strata serve: decode expert cache hit rate: 94.0% (4062 hits / 4320 lookups)"
+ ],
+ "answer_head": "69231569",
+ "needle_found": true,
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 11883,
+   "expert_cache_mib": 17564,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 645,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790719217.9333243,
+    "duration_s": 8.0,
+    "finish": "stop",
+    "prompt_tokens": 7493,
+    "reused": 0,
+    "output_tokens": 9,
+    "engine_generated": 9,
+    "prompt_ms": 7861.5,
+    "decode_ms": 153.7,
+    "decode_tok_s": 58.6,
+    "hit_rate": 0.94
+   }
+  ],
+  "requests_kept": 1,
+  "totals": {
+   "since": 1790719202.7795203,
+   "requests": 1,
+   "prompt_tokens": 7493,
+   "reused": 0,
+   "output_tokens": 9,
+   "prompt_ms": 7861.5,
+   "decode_ms": 153.7
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 0,
+     "mem_used": 25139544064,
+     "mem_total": 25769803776,
+     "temp": 46,
+     "power": 145.526
+    },
+    {
+     "index": 1,
+     "util": 41,
+     "mem_used": 24151785472,
+     "mem_total": 25769803776,
+     "temp": 53,
+     "power": 243.553
+    }
+   ],
+   "gpu_util": 20.5,
+   "gpu_mem_used": 49291329536,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 53,
+   "gpu_power": 389.079,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.9521484375,
+   "gpu_pcie_tx_mb": 4154.1572265625,
+   "cpu": 12.2,
+   "ram_used": 62392664064,
+   "ram_total": 177175990272,
+   "disk_read_mb": 0.0036010722333241606,
+   "disk_write_mb": 0.0,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.5,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    2.0,
+    73.5,
+    23.5,
+    31.0,
+    56.5,
+    23.5,
+    20.5
+   ],
+   "gpu_mem_used": [
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49274552320,
+    49291329536,
+    49291329536,
+    49291329536,
+    49291329536,
+    49291329536,
+    49291329536
+   ],
+   "gpu_temp": [
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    48,
+    51,
+    53,
+    52,
+    52,
+    52,
+    53
+   ],
+   "gpu_power": [
+    257.56,
+    245.14,
+    244.59,
+    244.41,
+    244.51,
+    244.45,
+    244.36,
+    244.36,
+    244.26,
+    244.16,
+    244.27,
+    244.18,
+    244.09,
+    243.92,
+    243.88,
+    245.24,
+    443.06,
+    409.13,
+    432.67,
+    433.1,
+    400.92,
+    389.08
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.62,
+    0.57,
+    0.62,
+    0.67,
+    0.57,
+    0.62,
+    0.62,
+    0.67,
+    0.62,
+    0.62,
+    0.67,
+    0.62,
+    0.57,
+    0.62,
+    2169.46,
+    6179.19,
+    3838.3,
+    4927.82,
+    4008.2,
+    7314.58,
+    0.95
+   ],
+   "cpu": [
+    0.0,
+    15.8,
+    15.7,
+    6.4,
+    8.0,
+    8.1,
+    4.7,
+    6.7,
+    0.9,
+    1.5,
+    5.3,
+    12.5,
+    1.6,
+    3.2,
+    8.4,
+    14.2,
+    13.2,
+    10.5,
+    10.5,
+    13.4,
+    9.0,
+    12.2
+   ],
+   "ram_used": [
+    61827047424,
+    61871300608,
+    61941383168,
+    61960540160,
+    61978361856,
+    61977542656,
+    61984432128,
+    61992636416,
+    61992636416,
+    62000824320,
+    62002450432,
+    62005186560,
+    62006108160,
+    62014894080,
+    62023741440,
+    62230065152,
+    62246010880,
+    62250319872,
+    62234144768,
+    62227582976,
+    62226890752,
+    62392664064
+   ],
+   "disk_read_mb": [
+    null,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    28.33,
+    101.36,
+    41.6,
+    10.77,
+    0.0,
+    0.01,
+    0.0,
+    0.0
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0
+   ]
+  },
+  "time": 1790719226.4537745
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-2x3090-auto-two-turn/two-turn-16k-1.json

@@ -0,0 +1,755 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-2x3090-auto-two-turn",
+ "mode": "two-turn",
+ "ctx": "16k",
+ "target_tokens": 16384,
+ "run": 1,
+ "run_id": "dedda4f9d6be",
+ "corpus_repeated": false,
+ "time": "2026-09-29T22:01:09",
+ "prompt_tokens": 15635,
+ "completion_tokens": 44,
+ "ttft_s": 13.63,
+ "e2e_s": 14.13,
+ "api_prompt_tps": 1147.1,
+ "api_decode_tps": 86.1,
+ "engine_prompt_tps": 1154.6,
+ "engine_decode_tps": 85.6,
+ "log_lines": [
+  "strata serve: prompt 15635 tokens = 0 reused + 15635 read in 13541 ms (1154.6 tok/s), 44 generated in 514 ms (85.6 tok/s), drafts accepted 30 of 36, 1 checkpoints",
+  "strata serve: decode expert cache hit rate: 97.9% (24433 hits / 24957 lookups)"
+ ],
+ "answer_head": "I'll start by exploring the repository structure to understand the context better.",
+ "turn2_ttft_s": 0.577,
+ "turn2_prompt_tokens": 15672,
+ "turn2_log_lines": [
+  "strata serve: prompt 15672 tokens = 15628 reused + 44 read in 490 ms (89.8 tok/s), 48 generated in 567 ms (84.6 tok/s), drafts accepted 28 of 43, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 98.6% (31236 hits / 31679 lookups)"
+ ],
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 11883,
+   "expert_cache_mib": 17564,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 645,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790719284.681533,
+    "duration_s": 1.1,
+    "finish": "stop",
+    "prompt_tokens": 15672,
+    "reused": 15628,
+    "output_tokens": 48,
+    "engine_generated": 48,
+    "prompt_ms": 490.1,
+    "decode_ms": 567.1,
+    "decode_tok_s": 84.6,
+    "hit_rate": 0.986
+   },
+   {
+    "projection": null,
+    "time": 1790719270.0522854,
+    "duration_s": 14.1,
+    "finish": "stop",
+    "prompt_tokens": 15635,
+    "reused": 0,
+    "output_tokens": 44,
+    "engine_generated": 44,
+    "prompt_ms": 13541.4,
+    "decode_ms": 513.7,
+    "decode_tok_s": 85.7,
+    "hit_rate": 0.979
+   },
+   {
+    "projection": null,
+    "time": 1790719242.7790127,
+    "duration_s": 26.1,
+    "finish": "stop",
+    "prompt_tokens": 29825,
+    "reused": 0,
+    "output_tokens": 9,
+    "engine_generated": 9,
+    "prompt_ms": 26008.5,
+    "decode_ms": 86.6,
+    "decode_tok_s": 103.9,
+    "hit_rate": 0.982
+   },
+   {
+    "projection": null,
+    "time": 1790719226.530526,
+    "duration_s": 15.6,
+    "finish": "stop",
+    "prompt_tokens": 17344,
+    "reused": 0,
+    "output_tokens": 8,
+    "engine_generated": 8,
+    "prompt_ms": 15486.8,
+    "decode_ms": 95.5,
+    "decode_tok_s": 83.8,
+    "hit_rate": 0.973
+   },
+   {
+    "projection": null,
+    "time": 1790719217.9333243,
+    "duration_s": 8.0,
+    "finish": "stop",
+    "prompt_tokens": 7493,
+    "reused": 0,
+    "output_tokens": 9,
+    "engine_generated": 9,
+    "prompt_ms": 7861.5,
+    "decode_ms": 153.7,
+    "decode_tok_s": 58.6,
+    "hit_rate": 0.94
+   }
+  ],
+  "requests_kept": 5,
+  "totals": {
+   "since": 1790719202.7795203,
+   "requests": 5,
+   "prompt_tokens": 85969,
+   "reused": 15628,
+   "output_tokens": 118,
+   "prompt_ms": 63388.3,
+   "decode_ms": 1416.6
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 15,
+     "mem_used": 25191972864,
+     "mem_total": 25769803776,
+     "temp": 48,
+     "power": 148.771
+    },
+    {
+     "index": 1,
+     "util": 0,
+     "mem_used": 24206311424,
+     "mem_total": 25769803776,
+     "temp": 51,
+     "power": 169.566
+    }
+   ],
+   "gpu_util": 7.5,
+   "gpu_mem_used": 49398284288,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 51,
+   "gpu_power": 318.337,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 26.1298828125,
+   "gpu_pcie_tx_mb": 56.265625,
+   "cpu": 39.7,
+   "ram_used": 63088369664,
+   "ram_total": 177175990272,
+   "disk_read_mb": 1.2843527404984159,
+   "disk_write_mb": 0.0035875774874257423,
+   "tok_s": 52.0,
+   "tok_s_mean": 89.37171592337687
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    23.5,
+    31.0,
+    56.5,
+    23.5,
+    20.5,
+    0.5,
+    70.5,
+    22.5,
+    26.5,
+    56.0,
+    29.0,
+    77.0,
+    11.0,
+    28.5,
+    59.5,
+    28.5,
+    19.5,
+    17.0,
+    27.0,
+    20.0,
+    0.0,
+    60.5,
+    25.5,
+    71.0,
+    67.0,
+    26.5,
+    26.5,
+    85.0,
+    26.0,
+    73.5,
+    49.5,
+    25.0,
+    48.0,
+    32.0,
+    76.5,
+    9.0,
+    27.5,
+    47.5,
+    30.0,
+    62.5,
+    15.5,
+    31.5,
+    29.0,
+    39.5,
+    13.0,
+    0.0,
+    69.5,
+    24.0,
+    30.5,
+    61.5,
+    27.0,
+    62.5,
+    17.0,
+    26.5,
+    53.5,
+    23.5,
+    29.5,
+    29.0,
+    41.0,
+    7.5
+   ],
+   "gpu_mem_used": [
+    49291329536,
+    49291329536,
+    49291329536,
+    49291329536,
+    49291329536,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49356341248,
+    49398284288,
+    49398284288
+   ],
+   "gpu_temp": [
+    53,
+    52,
+    52,
+    52,
+    53,
+    49,
+    51,
+    52,
+    52,
+    53,
+    53,
+    53,
+    52,
+    53,
+    51,
+    53,
+    53,
+    53,
+    53,
+    54,
+    50,
+    51,
+    53,
+    53,
+    52,
+    54,
+    52,
+    52,
+    53,
+    53,
+    53,
+    53,
+    52,
+    53,
+    54,
+    52,
+    53,
+    52,
+    53,
+    52,
+    52,
+    54,
+    53,
+    54,
+    52,
+    50,
+    51,
+    53,
+    54,
+    52,
+    54,
+    53,
+    53,
+    54,
+    53,
+    53,
+    54,
+    55,
+    53,
+    51
+   ],
+   "gpu_power": [
+    409.13,
+    432.67,
+    433.1,
+    400.92,
+    389.08,
+    278.03,
+    383.36,
+    392.74,
+    440.94,
+    403.62,
+    421.29,
+    427.09,
+    407.67,
+    422.11,
+    423.13,
+    417.76,
+    440.15,
+    377.19,
+    403.8,
+    385.27,
+    299.57,
+    398.78,
+    401.71,
+    449.87,
+    430.87,
+    402.83,
+    439.43,
+    401.46,
+    455.62,
+    425.18,
+    404.39,
+    417.35,
+    406.87,
+    423.67,
+    422.86,
+    404.6,
+    421.5,
+    402.4,
+    422.71,
+    432.67,
+    398.05,
+    455.72,
+    434.46,
+    403.83,
+    385.61,
+    282.41,
+    360.22,
+    446.82,
+    449.66,
+    410.55,
+    427.9,
+    441.96,
+    389.46,
+    451.44,
+    439.2,
+    402.13,
+    441.82,
+    408.98,
+    365.37,
+    318.34
+   ],
+   "gpu_pcie_rx_mb": [
+    3838.3,
+    4927.82,
+    4008.2,
+    7314.58,
+    0.95,
+    1338.19,
+    7892.51,
+    3024.72,
+    5741.74,
+    2921.96,
+    5454.83,
+    5667.49,
+    3768.87,
+    6803.13,
+    8164.64,
+    2711.1,
+    5725.29,
+    6417.89,
+    0.67,
+    1.76,
+    1343.92,
+    7796.67,
+    10809.8,
+    5598.92,
+    3013.99,
+    91.89,
+    4693.79,
+    5621.38,
+    5381.87,
+    7487.25,
+    2228.12,
+    7344.77,
+    3943.44,
+    0.81,
+    5476.19,
+    2437.4,
+    6693.74,
+    3836.63,
+    7843.59,
+    5638.65,
+    2853.11,
+    8215.43,
+    8500.96,
+    4445.79,
+    0.67,
+    0.62,
+    4431.82,
+    4655.12,
+    7505.13,
+    4915.86,
+    7990.69,
+    6352.23,
+    3326.13,
+    3696.82,
+    7077.12,
+    4858.06,
+    5613.99,
+    7732.68,
+    0.67,
+    26.13
+   ],
+   "cpu": [
+    10.5,
+    10.5,
+    13.4,
+    9.0,
+    12.2,
+    23.9,
+    8.1,
+    16.5,
+    13.0,
+    13.0,
+    9.0,
+    9.7,
+    12.8,
+    8.6,
+    12.9,
+    10.4,
+    8.6,
+    9.0,
+    9.4,
+    9.1,
+    16.9,
+    17.1,
+    22.7,
+    7.5,
+    10.8,
+    8.9,
+    7.3,
+    7.3,
+    8.3,
+    7.7,
+    9.8,
+    6.9,
+    12.4,
+    6.4,
+    12.2,
+    20.6,
+    19.9,
+    11.0,
+    8.1,
+    10.3,
+    9.8,
+    7.6,
+    8.1,
+    7.3,
+    16.6,
+    7.1,
+    8.3,
+    7.7,
+    7.5,
+    19.7,
+    13.3,
+    9.8,
+    12.2,
+    12.2,
+    11.3,
+    11.1,
+    11.3,
+    11.7,
+    49.6,
+    39.7
+   ],
+   "ram_used": [
+    62250319872,
+    62234144768,
+    62227582976,
+    62226890752,
+    62392664064,
+    62490071040,
+    62486704128,
+    62515294208,
+    62517342208,
+    62467452928,
+    62510080000,
+    62495842304,
+    62471684096,
+    62463819776,
+    62459879424,
+    62443171840,
+    62374465536,
+    62303469568,
+    62390607872,
+    62596341760,
+    62489825280,
+    62562496512,
+    62577270784,
+    62577836032,
+    62576951296,
+    62564986880,
+    62553812992,
+    62539886592,
+    62533107712,
+    62521810944,
+    62503256064,
+    62503677952,
+    62490673152,
+    62492131328,
+    62495432704,
+    62503706624,
+    62769029120,
+    62766465024,
+    62765707264,
+    62747688960,
+    62726733824,
+    62717767680,
+    62722011136,
+    62728314880,
+    62951010304,
+    62691995648,
+    62677225472,
+    62661165056,
+    62640271360,
+    62636351488,
+    62633791488,
+    62731657216,
+    62747160576,
+    62744502272,
+    62746267648,
+    62732316672,
+    62762123264,
+    62711451648,
+    62861885440,
+    63088369664
+   ],
+   "disk_read_mb": [
+    10.77,
+    0.0,
+    0.01,
+    0.0,
+    0.0,
+    1.06,
+    0.0,
+    14.72,
+    0.0,
+    56.04,
+    0.0,
+    33.79,
+    66.18,
+    16.29,
+    83.86,
+    33.38,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.73,
+    0.0,
+    0.0,
+    0.0,
+    0.22,
+    0.0,
+    0.0,
+    0.14,
+    0.0,
+    0.53,
+    47.85,
+    0.0,
+    71.37,
+    0.0,
+    78.98,
+    55.42,
+    22.25,
+    53.69,
+    0.0,
+    54.32,
+    39.07,
+    0.0,
+    0.0,
+    0.0,
+    0.09,
+    0.58,
+    0.02,
+    0.01,
+    0.0,
+    0.03,
+    0.0,
+    2.6,
+    6.44,
+    7.35,
+    8.08,
+    4.14,
+    2.09,
+    0.01,
+    2.47,
+    1.28
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    52.0
+   ]
+  },
+  "time": 1790719286.2452984
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-2x3090-auto/speed-32k-1.json

@@ -0,0 +1,612 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-2x3090-auto",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 1,
+ "run_id": "cfa75b59eb61",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:39:52",
+ "prompt_tokens": 28314,
+ "completion_tokens": 256,
+ "ttft_s": 26.573,
+ "e2e_s": 29.595,
+ "api_prompt_tps": 1065.5,
+ "api_decode_tps": 84.4,
+ "engine_prompt_tps": 1071.8,
+ "engine_decode_tps": 83.7,
+ "log_lines": [
+  "strata serve: prompt 28314 tokens = 0 reused + 28314 read in 26417 ms (1071.8 tok/s), 256 generated in 3058 ms (83.7 tok/s), drafts accepted 147 of 251, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 98.9% (171443 hits / 173268 lookups)"
+ ],
+ "answer_head": "The last file in the repository listing is `docs/DETAILS.md`.\n\n### What `docs/DETAILS.md` does\n\nIt is the comprehensive technical documentation and reference manual for the Strata engine. While the `README.md` focuses on quick installation and high-level usage, `DETAILS.md` provides the deep technic",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 11883,
+   "expert_cache_mib": 17564,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 645,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790717992.6885722,
+    "duration_s": 29.5,
+    "finish": "length",
+    "prompt_tokens": 28314,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 26416.6,
+    "decode_ms": 3057.9,
+    "decode_tok_s": 83.7,
+    "hit_rate": 0.989
+   }
+  ],
+  "requests_kept": 1,
+  "totals": {
+   "since": 1790717968.2500567,
+   "requests": 1,
+   "prompt_tokens": 28314,
+   "reused": 0,
+   "output_tokens": 256,
+   "prompt_ms": 26416.6,
+   "decode_ms": 3057.9
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 44,
+     "mem_used": 25196167168,
+     "mem_total": 25769803776,
+     "temp": 52,
+     "power": 244.795
+    },
+    {
+     "index": 1,
+     "util": 0,
+     "mem_used": 24208408576,
+     "mem_total": 25769803776,
+     "temp": 52,
+     "power": 218.413
+    }
+   ],
+   "gpu_util": 22.0,
+   "gpu_mem_used": 49404575744,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 52,
+   "gpu_power": 463.20799999999997,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 0.666015625,
+   "gpu_pcie_tx_mb": 0.857421875,
+   "cpu": 60.1,
+   "ram_used": 62328995840,
+   "ram_total": 177175990272,
+   "disk_read_mb": 3.420667733965059,
+   "disk_write_mb": 0.02523147959721329,
+   "tok_s": 0.0,
+   "tok_s_mean": 0.0
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    1.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    49.5,
+    21.5,
+    58.5,
+    25.5,
+    31.5,
+    24.0,
+    25.0,
+    35.0,
+    27.0,
+    64.5,
+    29.0,
+    24.5,
+    24.5,
+    27.0,
+    16.0,
+    26.0,
+    59.5,
+    24.5,
+    28.5,
+    48.0,
+    26.5,
+    29.0,
+    31.0,
+    3.0,
+    46.0,
+    46.0,
+    22.0
+   ],
+   "gpu_mem_used": [
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49205346304,
+    49278746624,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49383604224,
+    49404575744,
+    49404575744,
+    49404575744
+   ],
+   "gpu_temp": [
+    50,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    49,
+    50,
+    49,
+    46,
+    46,
+    46,
+    46,
+    46,
+    46,
+    48,
+    52,
+    53,
+    53,
+    53,
+    53,
+    54,
+    54,
+    53,
+    54,
+    53,
+    53,
+    54,
+    54,
+    54,
+    52,
+    55,
+    53,
+    54,
+    55,
+    54,
+    54,
+    54,
+    55,
+    53,
+    55,
+    55,
+    52
+   ],
+   "gpu_power": [
+    260.95,
+    248.93,
+    247.54,
+    246.62,
+    246.82,
+    246.65,
+    246.9,
+    246.83,
+    246.8,
+    246.68,
+    246.72,
+    246.75,
+    246.63,
+    246.53,
+    246.57,
+    246.83,
+    236.89,
+    159.99,
+    79.32,
+    65.35,
+    64.71,
+    64.62,
+    64.91,
+    91.65,
+    314.3,
+    428.65,
+    367.67,
+    455.67,
+    441.66,
+    390.44,
+    440.16,
+    375.92,
+    453.54,
+    370.98,
+    401.8,
+    438.74,
+    379.04,
+    440.69,
+    353.57,
+    419.67,
+    435.26,
+    406.67,
+    447.91,
+    390.07,
+    459.5,
+    448.73,
+    405.61,
+    374.45,
+    407.02,
+    468.77,
+    463.21
+   ],
+   "gpu_pcie_rx_mb": [
+    0.67,
+    0.62,
+    0.62,
+    0.67,
+    0.57,
+    7.87,
+    0.62,
+    0.62,
+    0.67,
+    0.62,
+    0.62,
+    0.57,
+    0.62,
+    0.62,
+    1.38,
+    0.57,
+    0.76,
+    0.86,
+    0.91,
+    0.81,
+    0.86,
+    0.86,
+    0.81,
+    539.92,
+    4704.14,
+    5432.94,
+    3021.19,
+    4790.88,
+    4142.71,
+    2301.69,
+    8161.88,
+    3244.83,
+    5099.68,
+    7126.09,
+    2825.64,
+    6742.29,
+    2064.18,
+    5534.93,
+    4369.31,
+    7614.9,
+    4834.13,
+    2332.54,
+    7141.88,
+    4942.8,
+    3514.62,
+    5720.52,
+    6970.74,
+    14.21,
+    15.78,
+    30.52,
+    0.67
+   ],
+   "cpu": [
+    0.0,
+    14.3,
+    2.7,
+    0.9,
+    1.7,
+    1.5,
+    2.1,
+    1.3,
+    1.5,
+    1.3,
+    1.2,
+    2.2,
+    1.2,
+    0.9,
+    1.2,
+    19.1,
+    3.1,
+    3.2,
+    2.2,
+    1.9,
+    2.8,
+    2.0,
+    4.5,
+    12.1,
+    9.2,
+    7.1,
+    7.5,
+    5.5,
+    13.3,
+    15.7,
+    18.9,
+    15.5,
+    7.4,
+    9.3,
+    7.1,
+    10.1,
+    9.1,
+    6.1,
+    9.7,
+    7.4,
+    9.1,
+    9.2,
+    13.5,
+    15.4,
+    6.6,
+    8.7,
+    6.0,
+    28.4,
+    94.8,
+    99.5,
+    60.1
+   ],
+   "ram_used": [
+    61824409600,
+    61846106112,
+    61853241344,
+    61854605312,
+    61855711232,
+    61855694848,
+    61853442048,
+    61858058240,
+    61861154816,
+    61861928960,
+    61862432768,
+    61867282432,
+    61861072896,
+    61856456704,
+    61880406016,
+    61890236416,
+    61890191360,
+    61887963136,
+    61887766528,
+    61882646528,
+    61870043136,
+    61879148544,
+    61934235648,
+    61984825344,
+    62089637888,
+    62090121216,
+    62098157568,
+    62098403328,
+    62088568832,
+    62106021888,
+    61750501376,
+    61766746112,
+    61776080896,
+    61778165760,
+    61779779584,
+    61908017152,
+    61907206144,
+    61910032384,
+    62031020032,
+    62033805312,
+    62032531456,
+    61967249408,
+    61974036480,
+    61990858752,
+    61985890304,
+    61979660288,
+    61980233728,
+    62277226496,
+    62302523392,
+    62302191616,
+    62328995840
+   ],
+   "disk_read_mb": [
+    null,
+    0.01,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    103.38,
+    66.8,
+    0.0,
+    24.38,
+    0.0,
+    55.65,
+    33.54,
+    0.11,
+    67.43,
+    0.0,
+    83.76,
+    1.78,
+    79.2,
+    71.21,
+    0.0,
+    79.44,
+    0.0,
+    77.02,
+    53.81,
+    0.7,
+    42.73,
+    0.0,
+    0.0,
+    0.0,
+    1.02,
+    5.65,
+    4.98,
+    3.42
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    48.0,
+    86.94,
+    102.06,
+    0.0
+   ]
+  },
+  "time": 1790718022.6741135
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-2x3090-auto/speed-32k-2.json

@@ -0,0 +1,707 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-2x3090-auto",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 2,
+ "run_id": "c27c8adc0b65",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:40:22",
+ "prompt_tokens": 31633,
+ "completion_tokens": 256,
+ "ttft_s": 26.904,
+ "e2e_s": 29.536,
+ "api_prompt_tps": 1175.8,
+ "api_decode_tps": 96.9,
+ "engine_prompt_tps": 1185.2,
+ "engine_decode_tps": 96.7,
+ "log_lines": [
+  "strata serve: prompt 31633 tokens = 0 reused + 31633 read in 26691 ms (1185.2 tok/s), 256 generated in 2648 ms (96.7 tok/s), drafts accepted 160 of 214, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 99.1% (147426 hits / 148799 lookups)"
+ ],
+ "answer_head": "I cannot analyze the last file because the provided repository content is truncated. The `docs/DETAILS.md` file cuts off mid-sentence in the \"Using it\" section (`...if y`), and no subsequent files are shown.\n\nHowever, I can analyze the **`chat.py`** file, which is the last *complete* file provided i",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 11883,
+   "expert_cache_mib": 17564,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 645,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790718022.860633,
+    "duration_s": 29.4,
+    "finish": "length",
+    "prompt_tokens": 31633,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 26690.6,
+    "decode_ms": 2648.4,
+    "decode_tok_s": 96.7,
+    "hit_rate": 0.991
+   },
+   {
+    "projection": null,
+    "time": 1790717992.6885722,
+    "duration_s": 29.5,
+    "finish": "length",
+    "prompt_tokens": 28314,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 26416.6,
+    "decode_ms": 3057.9,
+    "decode_tok_s": 83.7,
+    "hit_rate": 0.989
+   }
+  ],
+  "requests_kept": 2,
+  "totals": {
+   "since": 1790717968.2500567,
+   "requests": 2,
+   "prompt_tokens": 59947,
+   "reused": 0,
+   "output_tokens": 512,
+   "prompt_ms": 53107.2,
+   "decode_ms": 5706.3
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 37,
+     "mem_used": 25196167168,
+     "mem_total": 25769803776,
+     "temp": 51,
+     "power": 209.254
+    },
+    {
+     "index": 1,
+     "util": 55,
+     "mem_used": 24208408576,
+     "mem_total": 25769803776,
+     "temp": 55,
+     "power": 248.664
+    }
+   ],
+   "gpu_util": 46.0,
+   "gpu_mem_used": 49404575744,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 55,
+   "gpu_power": 457.918,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 30.46875,
+   "gpu_pcie_tx_mb": 164.2216796875,
+   "cpu": 97.1,
+   "ram_used": 62326792192,
+   "ram_total": 177175990272,
+   "disk_read_mb": 4.657722192080744,
+   "disk_write_mb": 0.11106875996500236,
+   "tok_s": 102.58306863780025,
+   "tok_s_mean": 98.12574884260572
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    49.5,
+    21.5,
+    58.5,
+    25.5,
+    31.5,
+    24.0,
+    25.0,
+    35.0,
+    27.0,
+    64.5,
+    29.0,
+    24.5,
+    24.5,
+    27.0,
+    16.0,
+    26.0,
+    59.5,
+    24.5,
+    28.5,
+    48.0,
+    26.5,
+    29.0,
+    31.0,
+    3.0,
+    46.0,
+    46.0,
+    22.0,
+    59.5,
+    29.0,
+    60.0,
+    26.0,
+    31.5,
+    71.0,
+    32.5,
+    28.0,
+    66.0,
+    23.5,
+    24.5,
+    18.5,
+    29.0,
+    28.0,
+    78.0,
+    30.0,
+    73.5,
+    16.0,
+    28.0,
+    25.0,
+    28.0,
+    29.0,
+    66.0,
+    26.5,
+    18.5,
+    43.5,
+    46.0
+   ],
+   "gpu_mem_used": [
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49192763392,
+    49205346304,
+    49278746624,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49295523840,
+    49383604224,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744
+   ],
+   "gpu_temp": [
+    46,
+    46,
+    46,
+    46,
+    46,
+    48,
+    52,
+    53,
+    53,
+    53,
+    53,
+    54,
+    54,
+    53,
+    54,
+    53,
+    53,
+    54,
+    54,
+    54,
+    52,
+    55,
+    53,
+    54,
+    55,
+    54,
+    54,
+    54,
+    55,
+    53,
+    55,
+    55,
+    52,
+    52,
+    54,
+    54,
+    54,
+    55,
+    55,
+    55,
+    54,
+    54,
+    54,
+    54,
+    54,
+    54,
+    54,
+    55,
+    55,
+    55,
+    54,
+    55,
+    54,
+    55,
+    55,
+    55,
+    55,
+    53,
+    55,
+    55
+   ],
+   "gpu_power": [
+    79.32,
+    65.35,
+    64.71,
+    64.62,
+    64.91,
+    91.65,
+    314.3,
+    428.65,
+    367.67,
+    455.67,
+    441.66,
+    390.44,
+    440.16,
+    375.92,
+    453.54,
+    370.98,
+    401.8,
+    438.74,
+    379.04,
+    440.69,
+    353.57,
+    419.67,
+    435.26,
+    406.67,
+    447.91,
+    390.07,
+    459.5,
+    448.73,
+    405.61,
+    374.45,
+    407.02,
+    468.77,
+    463.21,
+    376.39,
+    436.62,
+    444.21,
+    398.6,
+    447.75,
+    406.3,
+    391.79,
+    452.85,
+    427.37,
+    418.5,
+    444.64,
+    378.02,
+    407.95,
+    435.96,
+    404.27,
+    456.87,
+    440.61,
+    401.93,
+    421.84,
+    438.49,
+    398.65,
+    446.07,
+    421.03,
+    422.3,
+    368.39,
+    434.33,
+    457.92
+   ],
+   "gpu_pcie_rx_mb": [
+    0.91,
+    0.81,
+    0.86,
+    0.86,
+    0.81,
+    539.92,
+    4704.14,
+    5432.94,
+    3021.19,
+    4790.88,
+    4142.71,
+    2301.69,
+    8161.88,
+    3244.83,
+    5099.68,
+    7126.09,
+    2825.64,
+    6742.29,
+    2064.18,
+    5534.93,
+    4369.31,
+    7614.9,
+    4834.13,
+    2332.54,
+    7141.88,
+    4942.8,
+    3514.62,
+    5720.52,
+    6970.74,
+    14.21,
+    15.78,
+    30.52,
+    0.67,
+    4175.19,
+    7742.45,
+    4905.89,
+    4411.55,
+    4744.0,
+    6063.36,
+    4488.9,
+    7846.78,
+    7336.33,
+    4790.3,
+    4790.26,
+    4959.82,
+    3664.44,
+    6053.92,
+    4708.72,
+    5601.93,
+    6957.58,
+    5201.77,
+    7840.16,
+    7413.0,
+    3656.91,
+    7095.34,
+    5569.93,
+    6273.89,
+    9.44,
+    21.46,
+    30.47
+   ],
+   "cpu": [
+    2.2,
+    1.9,
+    2.8,
+    2.0,
+    4.5,
+    12.1,
+    9.2,
+    7.1,
+    7.5,
+    5.5,
+    13.3,
+    15.7,
+    18.9,
+    15.5,
+    7.4,
+    9.3,
+    7.1,
+    10.1,
+    9.1,
+    6.1,
+    9.7,
+    7.4,
+    9.1,
+    9.2,
+    13.5,
+    15.4,
+    6.6,
+    8.7,
+    6.0,
+    28.4,
+    94.8,
+    99.5,
+    60.1,
+    6.9,
+    5.7,
+    6.7,
+    6.1,
+    6.3,
+    13.9,
+    10.0,
+    6.4,
+    7.2,
+    5.7,
+    6.4,
+    6.6,
+    6.1,
+    5.5,
+    6.3,
+    6.6,
+    7.5,
+    6.5,
+    6.4,
+    17.9,
+    10.2,
+    6.4,
+    7.9,
+    6.6,
+    24.8,
+    93.3,
+    97.1
+   ],
+   "ram_used": [
+    61887766528,
+    61882646528,
+    61870043136,
+    61879148544,
+    61934235648,
+    61984825344,
+    62089637888,
+    62090121216,
+    62098157568,
+    62098403328,
+    62088568832,
+    62106021888,
+    61750501376,
+    61766746112,
+    61776080896,
+    61778165760,
+    61779779584,
+    61908017152,
+    61907206144,
+    61910032384,
+    62031020032,
+    62033805312,
+    62032531456,
+    61967249408,
+    61974036480,
+    61990858752,
+    61985890304,
+    61979660288,
+    61980233728,
+    62277226496,
+    62302523392,
+    62302191616,
+    62328995840,
+    62040481792,
+    62030888960,
+    62024949760,
+    62010191872,
+    62003638272,
+    62026113024,
+    62011977728,
+    61980151808,
+    61970239488,
+    61954445312,
+    61954318336,
+    61948420096,
+    61997678592,
+    62080253952,
+    62075203584,
+    62075965440,
+    62076403712,
+    62076399616,
+    62070407168,
+    62091988992,
+    62091988992,
+    62093479936,
+    62097862656,
+    62095917056,
+    62331494400,
+    62325518336,
+    62326792192
+   ],
+   "disk_read_mb": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    103.38,
+    66.8,
+    0.0,
+    24.38,
+    0.0,
+    55.65,
+    33.54,
+    0.11,
+    67.43,
+    0.0,
+    83.76,
+    1.78,
+    79.2,
+    71.21,
+    0.0,
+    79.44,
+    0.0,
+    77.02,
+    53.81,
+    0.7,
+    42.73,
+    0.0,
+    0.0,
+    0.0,
+    1.02,
+    5.65,
+    4.98,
+    3.42,
+    0.54,
+    0.0,
+    0.01,
+    0.02,
+    0.0,
+    0.02,
+    0.03,
+    0.0,
+    0.03,
+    0.02,
+    0.0,
+    0.01,
+    0.0,
+    0.01,
+    0.02,
+    0.0,
+    0.0,
+    12.64,
+    0.0,
+    71.65,
+    32.97,
+    0.0,
+    0.0,
+    0.0,
+    0.99,
+    5.21,
+    4.66
+   ],
+   "tok_s": [
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    48.0,
+    86.94,
+    102.06,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    56.0,
+    103.48,
+    102.58
+   ]
+  },
+  "time": 1790718052.712857
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/native-v0.1.26-IQ3_XXS-2x3090-auto/speed-32k-3.json

@@ -0,0 +1,721 @@
+{
+ "label": "native-v0.1.26-IQ3_XXS-2x3090-auto",
+ "mode": "speed",
+ "ctx": "32k",
+ "target_tokens": 31488,
+ "run": 3,
+ "run_id": "60c6cbe1b14a",
+ "corpus_repeated": false,
+ "time": "2026-09-29T21:40:52",
+ "prompt_tokens": 31603,
+ "completion_tokens": 256,
+ "ttft_s": 27.476,
+ "e2e_s": 30.102,
+ "api_prompt_tps": 1150.2,
+ "api_decode_tps": 97.1,
+ "engine_prompt_tps": 1158.9,
+ "engine_decode_tps": 96.9,
+ "log_lines": [
+  "strata serve: prompt 31603 tokens = 0 reused + 31603 read in 27271 ms (1158.9 tok/s), 256 generated in 2641 ms (96.9 tok/s), drafts accepted 154 of 209, 2 checkpoints",
+  "strata serve: decode expert cache hit rate: 99.6% (148622 hits / 149275 lookups)"
+ ],
+ "answer_head": "An analysis of the last file (`docs/DETAILS.md`) and a proposed improvement follows.\n\n### What the file does\n\n`docs/DETAILS.md` serves as the comprehensive technical manual for the Strata engine. It bridges the gap between the high-level marketing in the `README.md` and the raw source code. Specific",
+ "metrics_after": {
+  "engine": {
+   "model": "qwen3.8-flash-next-iq3_xxs",
+   "max_context": 32768,
+   "images": false,
+   "version": "0.1.26",
+   "context": 32768,
+   "kv": "int8",
+   "kv_resident": 0,
+   "expert_slots": 11883,
+   "expert_cache_mib": 17564,
+   "spec": 6,
+   "mtp_max": 4,
+   "lookup": 3,
+   "vram_free_mib": 645,
+   "cvec": 0,
+   "arena_mib": 40925,
+   "pool_workers": 19,
+   "pcie_frac": "0.55",
+   "spec_min_p": "0.50",
+   "engine": "0.1.26"
+  },
+  "live": {
+   "state": "idle",
+   "queued": 0,
+   "phase": null,
+   "prompt_tokens": null,
+   "prompt_read": null,
+   "prompt_total": null,
+   "generated": null,
+   "max_tokens": null,
+   "elapsed_s": null,
+   "tok_s": null,
+   "tok_s_mean": null,
+   "tok_s_window_s": null
+  },
+  "requests": [
+   {
+    "projection": null,
+    "time": 1790718052.8930452,
+    "duration_s": 29.9,
+    "finish": "length",
+    "prompt_tokens": 31603,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 27270.7,
+    "decode_ms": 2641.1,
+    "decode_tok_s": 96.9,
+    "hit_rate": 0.996
+   },
+   {
+    "projection": null,
+    "time": 1790718022.860633,
+    "duration_s": 29.4,
+    "finish": "length",
+    "prompt_tokens": 31633,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 26690.6,
+    "decode_ms": 2648.4,
+    "decode_tok_s": 96.7,
+    "hit_rate": 0.991
+   },
+   {
+    "projection": null,
+    "time": 1790717992.6885722,
+    "duration_s": 29.5,
+    "finish": "length",
+    "prompt_tokens": 28314,
+    "reused": 0,
+    "output_tokens": 256,
+    "engine_generated": 256,
+    "prompt_ms": 26416.6,
+    "decode_ms": 3057.9,
+    "decode_tok_s": 83.7,
+    "hit_rate": 0.989
+   }
+  ],
+  "requests_kept": 3,
+  "totals": {
+   "since": 1790717968.2500567,
+   "requests": 3,
+   "prompt_tokens": 91550,
+   "reused": 0,
+   "output_tokens": 768,
+   "prompt_ms": 80377.9,
+   "decode_ms": 8347.4
+  },
+  "hardware": {
+   "gpus": [
+    {
+     "index": 0,
+     "util": 43,
+     "mem_used": 25196167168,
+     "mem_total": 25769803776,
+     "temp": 52,
+     "power": 242.95
+    },
+    {
+     "index": 1,
+     "util": 65,
+     "mem_used": 24208408576,
+     "mem_total": 25769803776,
+     "temp": 55,
+     "power": 253.875
+    }
+   ],
+   "gpu_util": 54.0,
+   "gpu_mem_used": 49404575744,
+   "gpu_mem_total": 51539607552,
+   "gpu_temp": 55,
+   "gpu_power": 496.825,
+   "gpu_power_limit": 560.0,
+   "gpu_pcie_gen": 4,
+   "gpu_pcie_gen_max": 4,
+   "gpu_pcie_width": 16,
+   "gpu_pcie_rx_mb": 11.77734375,
+   "gpu_pcie_tx_mb": 120.830078125,
+   "cpu": 85.4,
+   "ram_used": 62122582016,
+   "ram_total": 177175990272,
+   "disk_read_mb": 2.9330443322466575,
+   "disk_write_mb": 0.08616042102070966,
+   "tok_s": 82.32972941826796,
+   "tok_s_mean": 95.05553044960695
+  },
+  "hardware_static": {
+   "gpu_name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090",
+   "gpu_count": 2,
+   "cpu_name": "AMD EPYC 7453 28-Core Processor",
+   "cores": 20,
+   "threads": 20,
+   "psutil": true
+  },
+  "history": {
+   "gpu_util": [
+    31.0,
+    3.0,
+    46.0,
+    46.0,
+    22.0,
+    59.5,
+    29.0,
+    60.0,
+    26.0,
+    31.5,
+    71.0,
+    32.5,
+    28.0,
+    66.0,
+    23.5,
+    24.5,
+    18.5,
+    29.0,
+    28.0,
+    78.0,
+    30.0,
+    73.5,
+    16.0,
+    28.0,
+    25.0,
+    28.0,
+    29.0,
+    66.0,
+    26.5,
+    18.5,
+    43.5,
+    46.0,
+    0.0,
+    64.5,
+    26.0,
+    23.0,
+    72.5,
+    27.0,
+    66.0,
+    24.5,
+    76.5,
+    60.5,
+    27.5,
+    57.5,
+    24.5,
+    32.0,
+    66.5,
+    26.0,
+    27.0,
+    64.5,
+    29.5,
+    52.5,
+    65.5,
+    24.0,
+    23.5,
+    85.0,
+    30.5,
+    9.5,
+    46.0,
+    54.0
+   ],
+   "gpu_mem_used": [
+    49295523840,
+    49383604224,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744,
+    49404575744
+   ],
+   "gpu_temp": [
+    55,
+    53,
+    55,
+    55,
+    52,
+    52,
+    54,
+    54,
+    54,
+    55,
+    55,
+    55,
+    54,
+    54,
+    54,
+    54,
+    54,
+    54,
+    54,
+    55,
+    55,
+    55,
+    54,
+    55,
+    54,
+    55,
+    55,
+    55,
+    55,
+    53,
+    55,
+    55,
+    53,
+    54,
+    55,
+    55,
+    54,
+    55,
+    55,
+    55,
+    56,
+    54,
+    55,
+    55,
+    55,
+    55,
+    55,
+    55,
+    55,
+    54,
+    55,
+    55,
+    55,
+    55,
+    56,
+    55,
+    56,
+    54,
+    56,
+    55
+   ],
+   "gpu_power": [
+    405.61,
+    374.45,
+    407.02,
+    468.77,
+    463.21,
+    376.39,
+    436.62,
+    444.21,
+    398.6,
+    447.75,
+    406.3,
+    391.79,
+    452.85,
+    427.37,
+    418.5,
+    444.64,
+    378.02,
+    407.95,
+    435.96,
+    404.27,
+    456.87,
+    440.61,
+    401.93,
+    421.84,
+    438.49,
+    398.65,
+    446.07,
+    421.03,
+    422.3,
+    368.39,
+    434.33,
+    457.92,
+    400.91,
+    379.28,
+    435.92,
+    440.53,
+    380.92,
+    451.07,
+    408.21,
+    436.33,
+    435.94,
+    414.69,
+    425.45,
+    442.33,
+    389.05,
+    457.45,
+    433.14,
+    411.6,
+    454.98,
+    417.2,
+    434.56,
+    447.81,
+    430.75,
+    413.26,
+    446.25,
+    404.97,
+    437.77,
+    402.11,
+    426.81,
+    496.82
+   ],
+   "gpu_pcie_rx_mb": [
+    6970.74,
+    14.21,
+    15.78,
+    30.52,
+    0.67,
+    4175.19,
+    7742.45,
+    4905.89,
+    4411.55,
+    4744.0,
+    6063.36,
+    4488.9,
+    7846.78,
+    7336.33,
+    4790.3,
+    4790.26,
+    4959.82,
+    3664.44,
+    6053.92,
+    4708.72,
+    5601.93,
+    6957.58,
+    5201.77,
+    7840.16,
+    7413.0,
+    3656.91,
+    7095.34,
+    5569.93,
+    6273.89,
+    9.44,
+    21.46,
+    30.47,
+    0.67,
+    4040.72,
+    6771.71,
+    4774.76,
+    3710.08,
+    6013.11,
+    4218.43,
+    89.64,
+    5065.25,
+    4264.21,
+    7538.94,
+    3521.39,
+    1962.57,
+    7673.88,
+    4194.59,
+    12185.76,
+    7580.33,
+    5805.73,
+    5337.71,
+    5530.07,
+    5019.52,
+    94.17,
+    4176.62,
+    7140.59,
+    4804.28,
+    0.62,
+    11.11,
+    11.78
+   ],
+   "cpu": [
+    6.0,
+    28.4,
+    94.8,
+    99.5,
+    60.1,
+    6.9,
+    5.7,
+    6.7,
+    6.1,
+    6.3,
+    13.9,
+    10.0,
+    6.4,
+    7.2,
+    5.7,
+    6.4,
+    6.6,
+    6.1,
+    5.5,
+    6.3,
+    6.6,
+    7.5,
+    6.5,
+    6.4,
+    17.9,
+    10.2,
+    6.4,
+    7.9,
+    6.6,
+    24.8,
+    93.3,
+    97.1,
+    28.6,
+    7.9,
+    6.3,
+    6.6,
+    5.7,
+    7.6,
+    16.1,
+    12.3,
+    6.8,
+    6.9,
+    6.2,
+    7.1,
+    5.6,
+    6.4,
+    6.7,
+    5.7,
+    7.5,
+    6.5,
+    6.0,
+    8.4,
+    15.8,
+    7.4,
+    8.5,
+    6.1,
+    5.7,
+    7.0,
+    74.1,
+    85.4
+   ],
+   "ram_used": [
+    61980233728,
+    62277226496,
+    62302523392,
+    62302191616,
+    62328995840,
+    62040481792,
+    62030888960,
+    62024949760,
+    62010191872,
+    62003638272,
+    62026113024,
+    62011977728,
+    61980151808,
+    61970239488,
+    61954445312,
+    61954318336,
+    61948420096,
+    61997678592,
+    62080253952,
+    62075203584,
+    62075965440,
+    62076403712,
+    62076399616,
+    62070407168,
+    62091988992,
+    62091988992,
+    62093479936,
+    62097862656,
+    62095917056,
+    62331494400,
+    62325518336,
+    62326792192,
+    62020993024,
+    62018838528,
+    62010839040,
+    62003871744,
+    61990277120,
+    61970149376,
+    61773103104,
+    61820641280,
+    61810376704,
+    61790760960,
+    61782196224,
+    61814075392,
+    61809737728,
+    61808422912,
+    61930074112,
+    61927755776,
+    61912608768,
+    61800587264,
+    61798178816,
+    61793333248,
+    61882507264,
+    61882978304,
+    61885661184,
+    61886173184,
+    61886357504,
+    62129778688,
+    62122844160,
+    62122582016
+   ],
+   "disk_read_mb": [
+    0.0,
+    1.02,
+    5.65,
+    4.98,
+    3.42,
+    0.54,
+    0.0,
+    0.01,
+    0.02,
+    0.0,
+    0.02,
+    0.03,
+    0.0,
+    0.03,
+    0.02,
+    0.0,
+    0.01,
+    0.0,
+    0.01,
+    0.02,
+    0.0,
+    0.0,
+    12.64,
+    0.0,
+    71.65,
+    32.97,
+    0.0,
+    0.0,
+    0.0,
+    0.99,
+    5.21,
+    4.66,
+    2.02,
+    0.04,
+    0.0,
+    0.02,
+    0.04,
+    0.0,
+    0.04,
+    0.0,
+    0.05,
+    0.05,
+    0.0,
+    0.05,
+    0.04,
+    0.0,
+    0.04,
+    0.04,
+    0.0,
+    0.02,
+    0.03,
+    0.0,
+    0.01,
+    0.0,
+    0.16,
+    0.0,
+    0.0,
+    0.0,
+    3.86,
+    2.93
+   ],
+   "tok_s": [
+    0.0,
+    48.0,
+    86.94,
+    102.06,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    56.0,
+    103.48,
+    102.58,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    0.0,
+    114.9,
+    82.33
+   ]
+  },
+  "time": 1790718083.3185816
+ }
+}

bench/results/2026-09-29-rtx3090-epyc-milan/data/strata-bench.py

@@ -0,0 +1,304 @@
+#!/usr/bin/env python3
+"""Strata benchmark harness (contribution plan, PR 1 step 1.2).
+
+Follows the upstream method (bench/results/2026-09-28-speed-0114): one code-agent prompt per
+length, greedy, 256 generated tokens, MTP as configured by setup. Every run gets a unique
+prefix, so conversation checkpoints cannot skip the prompt. Numbers come from the engine log
+lines the owner asks for in issue #74. /metrics and API timing are stored as cross-checks.
+
+Stdlib only. Examples:
+    ./strata-bench.py --label IQ3_XXS-1x3090 --log ./strata-iq3_xxs.log
+  ./strata-bench.py --label Coder-2x3090-auto --contexts 16k,28k --mode needle
+  ./strata-bench.py --url http://127.0.0.1:9292 --model flash-next-strata --label via-llama-swap
+"""
+
+import argparse
+import json
+import os
+import re
+import statistics
+import sys
+import time
+import urllib.error
+import urllib.request
+import uuid
+from datetime import datetime
+from pathlib import Path
+
+NOMINAL = {"1k": 1024, "2.7k": 2765, "4k": 4096, "8k": 8192, "16k": 16384, "28k": 28672,
+           "32k": 32768, "64k": 65536, "128k": 131072, "262k": 262144}
+CODE_EXT = {".py", ".cpp", ".cc", ".h", ".hpp", ".cu", ".cuh", ".js", ".md", ".txt", ".cmake"}
+SKIP_DIRS = {".git", ".venv", "build", "engine", "third_party", "node_modules", "models",
+             "packs", "mtp", "bench", "data", "__pycache__", "Strata-data"}
+LOG_PATTERNS = [
+    re.compile(r"decode expert cache hit rate", re.I),
+    re.compile(r"\bPCIe\b"),
+    re.compile(r"prompt .*tok/s.*generated .*tok/s", re.I),
+    re.compile(r"reus|checkpoint", re.I),
+    re.compile(r"layer split", re.I),
+]
+TOKS_PER_S = re.compile(r"([\d][\d,]*(?:\.\d+)?)\s*tok(?:ens)?/s", re.I)
+
+
+def load_corpus(dirs):
+    parts = []
+    for d in dirs:
+        root = Path(d)
+        for p in sorted(root.rglob("*")):
+            if p.suffix.lower() not in CODE_EXT or not p.is_file():
+                continue
+            if any(s in SKIP_DIRS for s in p.relative_to(root).parts[:-1]):
+                continue
+            text = p.read_text(encoding="utf-8", errors="ignore")
+            if text.strip():
+                parts.append(f"### {p.relative_to(root).as_posix()}\n```\n{text}\n```\n")
+    if not parts:
+        sys.exit(f"no source files found in {dirs}")
+    return parts
+
+
+def build_text(parts, n_chars):
+    out, size, i, repeated = [], 0, 0, False
+    while size < n_chars:
+        if i and i % len(parts) == 0:
+            repeated = True
+        chunk = parts[i % len(parts)]
+        out.append(chunk)
+        size += len(chunk)
+        i += 1
+    return "".join(out)[:n_chars], repeated
+
+
+def post_stream(url, payload, api_key, timeout):
+    req = urllib.request.Request(
+        f"{url}/v1/chat/completions", data=json.dumps(payload).encode(),
+        headers={"Content-Type": "application/json", "Authorization": f"Bearer {api_key or 'none'}"})
+    t0 = time.perf_counter()
+    t_first, text, reasoning, usage, chunks = None, [], [], {}, 0
+    with urllib.request.urlopen(req, timeout=timeout) as resp:
+        for raw in resp:
+            line = raw.decode("utf-8", errors="ignore").strip()
+            if not line.startswith("data:"):
+                continue
+            data = line[5:].strip()
+            if data == "[DONE]":
+                break
+            try:
+                chunk = json.loads(data)
+            except json.JSONDecodeError:
+                continue
+            if chunk.get("usage"):
+                usage = chunk["usage"]
+            for ch in chunk.get("choices") or []:
+                delta = ch.get("delta") or {}
+                piece = delta.get("content") or ""
+                rpiece = delta.get("reasoning_content") or ""
+                if (piece or rpiece) and t_first is None:
+                    t_first = time.perf_counter()
+                if piece or rpiece:
+                    chunks += 1
+                text.append(piece)
+                reasoning.append(rpiece)
+    t_end = time.perf_counter()
+    return {"t_start": t0, "t_first": t_first or t_end, "t_end": t_end, "text": "".join(text),
+            "reasoning": "".join(reasoning), "usage": usage, "chunks": chunks}
+
+
+def get_json(url, path, api_key, timeout=10):
+    try:
+        req = urllib.request.Request(f"{url}{path}", headers={"Authorization": f"Bearer {api_key or 'none'}"})
+        with urllib.request.urlopen(req, timeout=timeout) as r:
+            body = r.read().decode("utf-8", errors="ignore")
+        try:
+            return json.loads(body)
+        except json.JSONDecodeError:
+            return body
+    except (urllib.error.URLError, TimeoutError) as e:
+        return {"error": str(e)}
+
+
+def wait_healthy(url, api_key, timeout):
+    deadline = time.time() + timeout
+    while time.time() < deadline:
+        h = get_json(url, "/health", api_key, timeout=5)
+        if not (isinstance(h, dict) and "error" in h):
+            return True
+        time.sleep(5)
+    return False
+
+
+def log_offset(log):
+    return log.stat().st_size if log and log.exists() else 0
+
+
+def log_lines_since(log, offset):
+    if not log or not log.exists():
+        return []
+    time.sleep(0.5)
+    with log.open("r", encoding="utf-8", errors="ignore") as f:
+        f.seek(offset)
+        new = f.read().splitlines()
+    return [ln for ln in new if any(p.search(ln) for p in LOG_PATTERNS)]
+
+
+def engine_numbers(lines):
+    """Best effort: the last 'prompt ... tok/s ... generated ... tok/s' line -> (prompt, decode)."""
+    for ln in reversed(lines):
+        if LOG_PATTERNS[2].search(ln):
+            vals = [float(v.replace(",", "")) for v in TOKS_PER_S.findall(ln)]
+            if len(vals) >= 2:
+                return vals[0], vals[1]
+    return None, None
+
+
+def target_tokens(key, max_tokens):
+    n = NOMINAL[key]
+    # leave room for the answer, like upstream's 259,943-token "262K" prompt
+    return n - max_tokens - 1024 if n >= 32768 else n
+
+
+def make_messages(mode, parts, n_chars, run_id):
+    body, repeated = build_text(parts, n_chars)
+    system = f"[bench run {run_id}] You are a coding agent working in the repository shown below."
+    if mode == "needle":
+        word = uuid.uuid4().hex[:8].upper()
+        mid = len(body) // 2
+        body = f"{body[:mid]}\n# NOTE: the secret code word is {word}.\n{body[mid:]}"
+        task = "What is the secret code word mentioned in the files above? Answer with the word only."
+    else:
+        word = None
+        task = ("Task: explain what the last file above does, then propose one concrete improvement "
+                "as a unified diff.")
+    msgs = [{"role": "system", "content": system},
+            {"role": "user", "content": f"Repository files:\n\n{body}\n\n{task}"}]
+    return msgs, word, repeated
+
+
+def run_once(a, parts, key, run, cpt):
+    tgt = target_tokens(key, a.max_tokens)
+    run_id = uuid.uuid4().hex[:12]
+    msgs, word, repeated = make_messages(a.mode, parts, int(tgt * cpt), run_id)
+    payload = {"model": a.model, "messages": msgs, "max_tokens": a.max_tokens, "stream": True,
+               "stream_options": {"include_usage": True}, "temperature": 0,
+               "reasoning_effort": a.reasoning_effort}
+    off = log_offset(a.log)
+    rec = {"label": a.label, "mode": a.mode, "ctx": key, "target_tokens": tgt, "run": run,
+           "run_id": run_id, "corpus_repeated": repeated, "time": datetime.now().isoformat(timespec="seconds")}
+    try:
+        r = post_stream(a.url, payload, a.api_key, a.timeout)
+    except Exception as e:  # noqa: BLE001 - record and continue with the next cell
+        rec["error"] = str(e)[:300]
+        return rec, cpt
+    pt = r["usage"].get("prompt_tokens")
+    ct = r["usage"].get("completion_tokens") or r["chunks"]
+    ttft = r["t_first"] - r["t_start"]
+    dec_t = r["t_end"] - r["t_first"]
+    lines = log_lines_since(a.log, off)
+    e_prompt, e_decode = engine_numbers(lines)
+    rec.update({
+        "prompt_tokens": pt, "completion_tokens": ct,
+        "ttft_s": round(ttft, 3), "e2e_s": round(r["t_end"] - r["t_start"], 3),
+        "api_prompt_tps": round(pt / ttft, 1) if pt and ttft > 0 else None,
+        "api_decode_tps": round((ct - 1) / dec_t, 1) if ct and ct > 1 and dec_t > 0 else None,
+        "engine_prompt_tps": e_prompt, "engine_decode_tps": e_decode,
+        "log_lines": lines, "answer_head": r["text"][:300],
+    })
+    if word:
+        rec["needle_found"] = word in r["text"].upper() or word in r["reasoning"].upper()
+    if a.mode == "two-turn":
+        follow = msgs + [{"role": "assistant", "content": r["text"]},
+                         {"role": "user", "content": "Summarize your answer in one sentence."}]
+        off2 = log_offset(a.log)
+        r2 = post_stream(a.url, {**payload, "messages": follow, "max_tokens": 64}, a.api_key, a.timeout)
+        rec["turn2_ttft_s"] = round(r2["t_first"] - r2["t_start"], 3)
+        rec["turn2_prompt_tokens"] = r2["usage"].get("prompt_tokens")
+        rec["turn2_log_lines"] = log_lines_since(a.log, off2)
+    if pt:
+        cpt = (int(tgt * cpt) + 400) / pt  # refine chars per token for the next cells
+    return rec, cpt
+
+
+def write_md(path, recs, label):
+    def med(vals):
+        vals = [v for v in vals if v is not None]
+        return round(statistics.median(vals), 1) if vals else None
+
+    rows = ["| ctx | prompt tokens | runs | TTFT s (med) | prompt tok/s engine / API (med) "
+            "| decode tok/s engine / API (run 1) | decode tok/s engine / API (med) | needle |",
+            "|---|---|---|---|---|---|---|---|"]
+    for key in dict.fromkeys(r["ctx"] for r in recs):
+        rs = [r for r in recs if r["ctx"] == key and "error" not in r]
+        errs = sum(1 for r in recs if r["ctx"] == key and "error" in r)
+        if not rs:
+            rows.append(f"| {key} | - | 0 ({errs} errors) | | | | | |")
+            continue
+        first = rs[0]
+        needle = "" if rs[0].get("needle_found") is None else f"{sum(r['needle_found'] for r in rs)}/{len(rs)}"
+        rows.append(
+            f"| {key} | {rs[0]['prompt_tokens']} | {len(rs)}{f' ({errs} err)' if errs else ''} "
+            f"| {med(r['ttft_s'] for r in rs)} "
+            f"| {med(r['engine_prompt_tps'] for r in rs)} / {med(r['api_prompt_tps'] for r in rs)} "
+            f"| {first['engine_decode_tps']} / {first['api_decode_tps']} "
+            f"| {med(r['engine_decode_tps'] for r in rs)} / {med(r['api_decode_tps'] for r in rs)} | {needle} |")
+    path.write_text(f"# {label}\n\n" + "\n".join(rows) + "\n", encoding="utf-8")
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--url", default="http://127.0.0.1:8090")
+    ap.add_argument("--model", default="strata", help="model name sent (llama-swap: the model id)")
+    ap.add_argument("--api-key", default=os.environ.get("STRATA_API_KEY", ""))
+    ap.add_argument("--label", required=True, help="e.g. IQ3_XXS-1x3090-calibrated")
+    ap.add_argument("--contexts", default="1k,4k,32k,128k", help=f"comma list of {','.join(NOMINAL)}")
+    ap.add_argument("--repeats", type=int, default=3)
+    ap.add_argument("--max-tokens", type=int, default=256)
+    ap.add_argument("--reasoning-effort", default="none")
+    ap.add_argument("--mode", choices=["speed", "needle", "two-turn"], default="speed")
+    ap.add_argument("--source-dir", action="append", default=None,
+                    help="code corpus for the prompts (default: current directory, the Strata repo)")
+    ap.add_argument("--log", type=Path, default=None,
+                    help="engine log, e.g. ./strata-iq3_xxs.log")
+    ap.add_argument("--out", type=Path, default=Path(f"./benchmark-output/{datetime.now():%Y-%m-%d}"))
+    ap.add_argument("--timeout", type=float, default=1800, help="per request, seconds (262K prefill takes minutes)")
+    ap.add_argument("--chars-per-token", type=float, default=3.4, help="start value, refined after each run")
+    a = ap.parse_args()
+
+    keys = [k.strip().lower() for k in a.contexts.split(",") if k.strip()]
+    bad = [k for k in keys if k not in NOMINAL]
+    if bad:
+        sys.exit(f"unknown context(s): {bad}")
+    parts = load_corpus(a.source_dir or ["."])
+    if a.log and not a.log.exists():
+        print(f"warning: log {a.log} not found; engine numbers will be empty", file=sys.stderr)
+    if not wait_healthy(a.url, a.api_key, 900):
+        sys.exit(f"{a.url}/health not reachable")
+
+    out = a.out / a.label
+    (out / "data").mkdir(parents=True, exist_ok=True)
+    recs, cpt = [], a.chars_per_token
+    for key in keys:
+        for run in range(1, a.repeats + 1):
+            rec, cpt = run_once(a, parts, key, run, cpt)
+            rec["metrics_after"] = get_json(a.url, "/metrics", a.api_key)
+            recs.append(rec)
+            (out / "data" / f"{a.mode}-{key}-{run}.json").write_text(json.dumps(rec, indent=1), encoding="utf-8")
+            status = rec.get("error") or (f"prompt {rec['prompt_tokens']} tok, TTFT {rec['ttft_s']} s, "
+                                          f"decode engine {rec['engine_decode_tps']} / API {rec['api_decode_tps']} tok/s")
+            print(f"[{a.label}] {a.mode} {key} run {run}: {status}", flush=True)
+
+    matrix_path = out / "matrix.json"
+    merged = {}
+    if matrix_path.exists():
+        for record in json.loads(matrix_path.read_text(encoding="utf-8")):
+            merged[(record["mode"], record["ctx"], record["run"])] = record
+    for record in recs:
+        key = (record["mode"], record["ctx"], record["run"])
+        merged[key] = {k: v for k, v in record.items() if k != "metrics_after"}
+    combined = list(merged.values())
+    matrix_path.write_text(json.dumps(combined, indent=1), encoding="utf-8")
+    write_md(out / "matrix.md", combined, f"{a.label} ({a.mode})")
+    print(f"written: {out}/matrix.md, matrix.json, data/")
+
+
+if __name__ == "__main__":
+    main()

bench/results/2026-09-29-rtx3090-epyc-milan/hw.txt

@@ -0,0 +1,86 @@
+
+## date (UTC)
+2026-09-29T22:07:24Z
+
+## strata version
+v0.1.26
+
+## os
+PRETTY_NAME="Ubuntu 24.04.5 LTS"
+VERSION_ID="24.04"
+6.8.0-139-generic
+
+## virtualization
+kvm
+
+## cpu
+CPU(s):                                  20
+Model name:                              AMD EPYC 7453 28-Core Processor
+Thread(s) per core:                      1
+Core(s) per socket:                      20
+Socket(s):                               1
+L3 cache:                                320 MiB (20 instances)
+NUMA node(s):                            1
+
+## cpu flags (relevant)
+avx2 f16c fma
+
+## numa
+available: 1 nodes (0)
+node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
+node 0 size: 168968 MB
+node 0 free: 47460 MB
+node distances:
+node   0
+  0:  10
+
+## memory
+               total        used        free      shared  buff/cache   available
+Mem:             165          14          46           0         105         150
+Swap:              0           0           0
+
+## dimms
+run as root for dmidecode
+(in a VM these are virtual DIMMs: fill in the host section below by hand)
+
+## memlock limit
+21627928
+
+## gpu
+index, name, driver_version, memory.total [MiB], pcie.link.gen.current, pcie.link.gen.max, pcie.link.width.current, pcie.link.width.max, power.limit [W], power.default_limit [W], power.max_limit [W], power.draw [W], clocks.max.sm [MHz]
+0, NVIDIA GeForce RTX 3090, 595.91.07, 24576 MiB, 1, 4, 16, 16, 280.00 W, 350.00 W, 366.00 W, 35.23 W, 2130 MHz
+1, NVIDIA GeForce RTX 3090, 595.91.07, 24576 MiB, 1, 4, 16, 16, 280.00 W, 420.00 W, 450.00 W, 28.60 W, 2100 MHz
+
+## gpu topology
+	GPU0	GPU1	CPU Affinity	NUMA Affinity	GPU NUMA ID
+GPU0	 X 	PHB	0-19	0		N/A
+GPU1	PHB	 X 	0-19	0		N/A
+
+Legend:
+
+  X    = Self
+  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
+  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
+  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
+  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
+  PIX  = Connection traversing at most a single PCIe bridge
+  NV#  = Connection traversing a bonded set of # NVLinks
+
+## gpu processes (must be empty before benchmarking)
+pid, process_name, used_gpu_memory [MiB]
+
+## docker
+Docker version 29.8.1, build 4a63305
+NVIDIA Container Toolkit CLI version 1.18.1
+
+## disk (model data)
+Filesystem      Size  Used Avail Use% Mounted on
+/dev/sdb1       541G  409G  106G  80% <storage>
+
+## host (fill in manually)
+- Hypervisor: Proxmox VE (KVM guest; host version not visible inside the VM)
+- CPU: AMD EPYC 7453 28-Core Processor (Milan); 20 vCPU exposed, AVX2, no AVX-512 visible
+- RAM: 165 GiB visible to the guest; physical DIMM layout and bandwidth not measured
+- VM: 20 vCPU, 1 socket x 20 cores x 1 thread, 1 NUMA node; ballooning and pinning not verified
+- GPUs: 2x RTX 3090 24 GB; PCIe Gen4 x16 maximum reported for each (current idle link generation varies); configured power limit 280 W each
+- Storage: ext4 guest data disk, 541 GiB; host storage layout not verified from the guest

bench/results/2026-09-29-rtx3090-epyc-milan/matrix.md

@@ -0,0 +1,48 @@
+# Native benchmark matrix (v0.1.26)
+
+Prompt speeds are engine tok/s medians; output lists the first-run value and the three-run median. `prompt tokens` is the median measured count. Draft acceptance is the mean of per-run accepted/offered values from engine logs.
+
+## IQ3_XXS, one RTX 3090
+
+| Context | Prompt tokens | Prompt tok/s | Decode first / median tok/s | TTFT median | Draft acceptance |
+|---|---:|---:|---:|---:|---:|
+| 1K | 1,135 | 869.7 | 93.0 / 93.0 | 1.33 s | 0.738 |
+| 4K | 4,200 | 1,772.2 | 91.7 / 89.2 | 2.41 s | 0.763 |
+| 32K | 31,602 | 2,160.4 | 90.6 / 90.6 | 14.83 s | 0.739 |
+| 128K | 129,913 | 2,173.6 | 79.3 / 79.3 | 60.53 s | 0.723 |
+| 262K | 259,589 | 1,725.6 | 63.0 / 67.3 | 152.10 s | 0.697 |
+
+## 32K model and GPU comparison
+
+| Model | GPUs / split | Prompt tokens | Prompt tok/s median | Decode first / median tok/s | TTFT median | Draft acceptance |
+|---|---|---:|---:|---:|---:|---:|
+| Coder IQ1_M | 1x 3090 | 31,601 | 2,502.4 | 52.7 / 66.8 | 12.80 s | 0.760 |
+| Coder IQ1_M | 2x 3090, auto K=19 | 31,601 | 2,704.3 | 88.5 / 98.6 | 11.85 s | 0.908 |
+| IQ3_XXS | 2x 3090, auto K=24 | 31,603 | 1,158.9 | 83.7 / 96.7 | 26.90 s | 0.690 |
+| IQ3_S | 2x 3090, auto K=24 | 31,604 | 749.3 | 83.1 / 106.3 | 42.27 s | 0.871 |
+
+## Coder IQ1_M fixed layer split
+
+One initial 32K run per K, then three-run repeats for K=22–30. Prompts are independent and adaptive-cache behavior varies; compare the medians cautiously.
+
+| Split | Successful runs | Prompt tok/s median | Decode tok/s median | Notes |
+|---|---:|---:|---:|---|
+| Auto (K=19) | 3 | 2,704.3 | 98.6 | Layers 0–18 / 19–47 |
+| K=18 | 1 | 2,324.6 | 83.1 | Repeat startup OOM while uploading a dense weight |
+| K=22 | 3 | 2,951.6 | 103.3 | Layers 0–21 / 22–47 |
+| K=24 | 3 | 3,123.2 | 100.2 | Layers 0–23 / 24–47 |
+| K=26 | 3 | 3,205.2 | 93.5 | Layers 0–25 / 26–47 |
+| K=28 | 3 | 3,034.9 | 96.6 | Layers 0–27 / 28–47 |
+| K=30 | 3 | 2,920.6 | 96.1 | Layers 0–29 / 30–47 |
+
+## Correctness
+
+| Model / setup | Check | Result |
+|---|---|---|
+| IQ3_XXS, 1x 3090 | Needles 8K / 16K / 28K | 3/3 found |
+| IQ3_XXS, 1x 3090 | 16K two-turn chat | 15,892 / 15,915 prompt tokens reused on turn two |
+| IQ3_XXS, 2x 3090 auto K=24 | Needles 8K / 16K / 28K | 3/3 found |
+| IQ3_XXS, 2x 3090 auto K=24 | 16K two-turn chat | 15,628 / 15,672 prompt tokens reused on turn two |
+| IQ3_S, 2x 3090 auto K=24 | Needles 8K / 16K / 28K | 3/3 found |
+| IQ3_S, 2x 3090 auto K=24 | 16K two-turn chat | 15,632 / 15,676 prompt tokens reused on turn two |
+| Coder, 2x 3090 | Needles / two-turn | Not run |

bench/results/2026-09-30-community-rtx-5090/BUILD.json

@@ -0,0 +1,14 @@
+{
+ "source": "local",
+ "version": "0.1.29",
+ "archs": [
+  120
+ ],
+ "vision": "gpu",
+ "cuda_dirs": [
+  "/usr/local/cuda/bin",
+  "/usr/local/cuda/lib64"
+ ],
+ "src": "1437bc9455f7c0af",
+ "vision_src": "4b80385716918cce"
+}

bench/results/2026-09-30-community-rtx-5090/Dockerfile

@@ -0,0 +1,4 @@
+FROM nvidia/cuda@sha256:5dc1bca23d05bd37b011be68ec470c03b403a5da07ec3a86e41af9470e9d0cc6
+RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends build-essential cmake ninja-build git ca-certificates curl python3 python3-venv python3-dev pkg-config && rm -rf /var/lib/apt/lists/*
+ENV HOME=/workspace/home PYTHONUNBUFFERED=1
+WORKDIR /workspace/Strata

bench/results/2026-09-30-community-rtx-5090/README.md

@@ -0,0 +1,167 @@
+# Community benchmark: RTX 5090, Core Ultra 9 285K
+
+Measured on 2026-09-30 by [hagope](https://github.com/hagope), on a Linux
+machine called OMAN. This tests Strata 0.1.29 with the original Flash-Next
+IQ2_XS, one GPU, and a 131,072-token context limit.
+
+Median decode throughput was **179.4 tok/s at 4,096 prompt tokens, 175.7 tok/s
+at 32,768, and 165.0 tok/s at 128,000**. These are synthetic code-explanation
+requests with greedy decoding and a 256-token output cap. They do not establish
+general answer quality or performance on other workloads.
+
+## Hardware and software
+
+- NVIDIA GeForce RTX 5090; 32,607 MiB reported VRAM; 575 W power limit.
+  PCIe capability: Gen 5, x16. The engine's startup transfer probe reported
+  50.0 GB/s host-to-device. GPU clocks were not fixed for this test.
+- Intel Core Ultra 9 285K; 24 logical CPUs. The engine selected AVX2 and
+  23 expert-pool workers plus its host thread.
+- 64 GB installed RAM; Linux reported 62.19 GiB usable. Local NVMe storage:
+  WDC WDS200T2B0C-00PXH0. 8 GiB system swap.
+- Ubuntu 24.04.4 LTS, kernel 6.8.0-142-generic, NVIDIA driver 595.84.
+- Source commit `d6708a4aae15b4860000d54c8af9e84d684bce09`; engine 0.1.29.
+  Locally compiled CUDA architecture 120, including the GPU vision helper.
+  See [BUILD.json](BUILD.json).
+- CUDA 13.0.2 container; NVCC 13.0.88; GCC 13.3.0; Python 3.12.3.
+  Base image: `nvidia/cuda@sha256:5dc1bca23d05bd37b011be68ec470c03b403a5da07ec3a86e41af9470e9d0cc6`.
+  Installed Python dependencies are listed in [python-packages.txt](python-packages.txt).
+- The regular inference server, GPU TTS, and GPU embedding service were paused.
+  Other CPU services remained running. The GPU was dedicated to Strata and its
+  vision helper; this was not a completely isolated operating system.
+
+## Model and configuration
+
+Model: `ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF`, revision
+`ed59f92082b1e93c0e96d60a8b11aab089b52f09`:
+
+- `IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf`
+- `IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00002-of-00002.gguf`
+- `mmproj-Qwen3.8-Flash-Next-BF16.gguf`
+
+All three local GGUF SHA-256 hashes matched the revision's published LFS hashes.
+[model-provenance.json](model-provenance.json) contains filenames, byte sizes,
+revisions, and verified hashes. MTP was fetched by the installer from
+`Qwen/Qwen3.8-Flash-Next`; its API reported revision
+`de4b8e4d43b917e7706784d8bb445c9af86a3540` at verification time. The installer
+uses `main` for those tensor-range downloads. Their individual output hashes
+are retained in [mtp-manifest.json](mtp-manifest.json); the full BF16 upstream
+shards were not downloaded or independently hash-verified.
+
+The unmodified installer prepared the native IQ2_XS pack and Q2_0 MTP experts.
+[artifact-hashes.json](artifact-hashes.json) identifies the executables,
+default expert profile, tokenizer, pack metadata/dense weights, and draft pack.
+
+- Context 131,072; INT8 KV; 32,768 KV cells per attention layer resident on GPU.
+- Expert cache `auto`: 17,463 slots, 23.44 GiB VRAM, profile-prefilled with
+  **no eviction**. The bundled expert profile was used without calibration.
+- Prefill `auto` selected chunks of 8,192 tokens, borrowing 3,421 expert-cache
+  slots during prompt processing.
+- MTP `--spec 4 --spec-min-p 0.5`; built-in suffix drafting remained enabled.
+- Low-RAM mode off; GPU vision enabled, 1,024 image tokens and a 700 MiB VRAM
+  reserve. The vision helper warmed up; benchmark requests contained no images.
+- Experimental speed projection off; no custom control vectors or calibration.
+- Reasoning disabled, temperature 0, maximum 256 generated tokens per speed run.
+
+The complete measured configuration is [strata-iq2_xs.json](strata-iq2_xs.json).
+Startup choices and every request's timing/draft statistics are in
+[engine.log](engine.log).
+
+## Method and reproduction
+
+Use the attached [Dockerfile](Dockerfile), the pinned source commit, and an empty
+data directory. Mount a working directory as `/workspace`, with the checkout
+at `/workspace/Strata`. Run setup inside the CUDA container with GPU access,
+host IPC, and unlimited memlock:
+
+```bash
+bash ./setup.sh --family qwen --model IQ2_XS --context 131072 --kv int8 \
+  --vision gpu --experimental-speed-projection off --gpu 0 --low-ram off \
+  --build --yes --no-start --port 18080 --data-dir /workspace/Strata-data
+```
+
+Launch the prepared server on loopback inside a container using host networking:
+
+```bash
+/workspace/Strata/.venv/bin/python serve/server.py --engine strata \
+  --config strata-iq2_xs.json --host 127.0.0.1 --port 18080
+```
+
+Run [benchmark.py](benchmark.py) inside that container, after placing the script
+at `/workspace/benchmark.py`. It generates deterministic synthetic Python
+functions, adds a different nonce near the start of each request, and counts the
+complete rendered chat prompt using Strata's tokenizer. It adjusts the filler
+to the target token count and verifies the count against the server afterward.
+The resulting request hashes are in [results.json](results.json).
+
+```bash
+/workspace/Strata/.venv/bin/python /workspace/benchmark.py
+/workspace/Strata/.venv/bin/python tools/needle_bench.py \
+  --url http://127.0.0.1:18080 --lengths 32k,128k --depths 10,50,90 \
+  --out /workspace/results/needles.json
+```
+
+One short warm-up request was excluded. Three runs at each length were executed
+serially in increasing-length order on the same loaded engine. All nine speed
+requests processed their entire prompt: **zero reused tokens**. The GPU expert
+cache was filled at startup and retained between requests; no engine restart or
+OS page-cache reset separated them. Downloads and file verification had already
+warmed the filesystem cache. Loading time is excluded.
+
+The script measures streaming TTFT from immediately before the HTTP request
+until the first nonempty text delta, ignoring empty role chunks and keep-alives.
+With reasoning off, that delta contains answer text. Total latency ends at
+stream completion. Both timings include HTTP and frontend tokenization on OMAN
+over loopback. Engine prompt throughput uses freshly read tokens divided by
+`prompt_ms`; decode throughput uses `engine_generated / decode_ms`. The JSON
+preserves the engine timing values and output text for each run.
+
+## Results
+
+Each cell below is the median **[minimum–maximum]** of three runs. Every request
+generated 256 tokens and stopped at the output limit, with no failed or cancelled
+speed requests.
+
+| Prompt tokens | Reused | Prompt tok/s | Decode tok/s | TTFT seconds | Total seconds |
+| ---: | ---: | ---: | ---: | ---: | ---: |
+| 4,096 | 0 | 4,269.8 [3,510.2–4,273.3] | 179.4 [161.3–184.8] | 0.974 [0.973–1.183] | 2.392 [2.351–2.761] |
+| 32,768 | 0 | 5,543.2 [5,527.1–5,549.6] | 175.7 [166.1–181.8] | 5.950 [5.942–5.968] | 7.390 [7.348–7.498] |
+| 128,000 | 0 | 5,778.7 [5,767.0–5,791.9] | 165.0 [154.8–169.3] | 22.262 [22.212–22.308] | 23.850 [23.765–23.856] |
+
+Raw per-run records: [results.json](results.json). Calculated aggregates:
+[summary.json](summary.json). The lower first 4K run is retained in the range.
+Decode expert-cache hit rates were 97.8–99.7% for these nine requests.
+
+[monitor.py](monitor.py) sampled the host once per second, starting before model
+load and ending after the recall checks. [telemetry.jsonl](telemetry.jsonl)
+contains 250 samples over 255.9 seconds, with the following observed peaks:
+
+- GPU memory: **31,785 MiB (31.04 GiB)**, system-wide on the selected GPU.
+- Host RAM: **45.54 GiB**, calculated as `MemTotal - MemAvailable`. This includes
+  Strata, other services, and non-reclaimable system memory; it is not process RSS.
+- Swap allocation increased from **0.25 MiB to 425.50 MiB**, including increases
+  during the speed suite. Swap I/O rates were not measured, so this is not a
+  claim that inference ran without paging. No out-of-memory error occurred.
+
+Sampling and the server's own telemetry were active during inference. These
+are sampled peaks; brief allocation spikes between samples may be missed.
+Aggregates are also recorded in [memory-summary.json](memory-summary.json).
+
+## Recall and limitations
+
+The repository's unchanged `tools/needle_bench.py` found **all six needles**:
+depths 10%, 50%, and 90% at both requested lengths. Actual prompt lengths were
+30,804–30,805 tokens for `32k` and 126,318–126,320 for `128k`. All answers exactly
+matched the expected code words. See [needles.json](needles.json).
+
+The final 128K recall case reused 16,384 prompt tokens; the other five reused
+zero. Recall latency is therefore not used in the fresh-prompt speed table.
+The script chooses needle depth by character position, not token position.
+
+This is one machine, one quantization, one configuration, and a small synthetic
+workload. Long output, sampled decoding, thinking, coding-task correctness,
+vision, tool use, multi-request concurrency, and a sustained thermal run were
+not evaluated. The built-in needle check measures recall on these six inputs,
+not general model accuracy. No same-workload baseline on another engine or GPU
+was run; do not interpret the README's different-hardware numbers as a controlled
+comparison. The 128,000-token speed prompt also does not fill every token of the
+131,072-token context window.

bench/results/2026-09-30-community-rtx-5090/artifact-hashes.json

@@ -0,0 +1,82 @@
+[
+  {
+    "file": "Strata-data/mtp/mtp-q2_0.gguf",
+    "bytes": 889017216,
+    "sha256": "a22206a4d74eafbf31063a84610292535fb3b49af9a339963817fd04aed81e4d"
+  },
+  {
+    "file": "Strata-data/mtp/rt/dense.bin",
+    "bytes": 116099072,
+    "sha256": "c724dc0b0822ada5d2977bf5bde821605feabaa64ea2e0045b67ca656329070a"
+  },
+  {
+    "file": "Strata-data/mtp/rt/dense.txt",
+    "bytes": 1851,
+    "sha256": "991456cf4eb266eb8a888c349e548d11bc640f5dbd9149439aa9786af0f732fd"
+  },
+  {
+    "file": "Strata-data/mtp/rt/draft_vocab.bin",
+    "bytes": 425196,
+    "sha256": "b1e1d3a7a9e4bf862dcd5923ce661fb59bbd07907e594df5cf86a62ac235cb91"
+  },
+  {
+    "file": "Strata-data/mtp/rt/experts.bin",
+    "bytes": 707788800,
+    "sha256": "09398406be61f1f54c93861f449e48b8df0bfccbc9ec9b2b7636775a6ea9244f"
+  },
+  {
+    "file": "Strata-data/packs/iq2_xs/dense.bin",
+    "bytes": 1538035200,
+    "sha256": "d2af3d8331e5c7a1985d1d589747230f9d56719a6d33c7cddc9ee6b81cde6b13"
+  },
+  {
+    "file": "Strata-data/packs/iq2_xs/index.txt",
+    "bytes": 93686,
+    "sha256": "19e781af360ca3b83fd90fe09e757ac9aa2a9dcada26810ddc9e933bdb112091"
+  },
+  {
+    "file": "Strata-data/packs/iq2_xs/native_experts.txt",
+    "bytes": 3289,
+    "sha256": "48df605acb89c7e8c3176a51cec777c2ed344ea439e8f3a37e5ce4f9dcb6946c"
+  },
+  {
+    "file": "Strata-data/packs/iq2_xs/tokenizer/chat_template.jinja",
+    "bytes": 9993,
+    "sha256": "12827f24b742ea4e80cdc12dbcf9622227056b9f797252a3149263d4f9aaadce"
+  },
+  {
+    "file": "Strata-data/packs/iq2_xs/tokenizer/merges.txt",
+    "bytes": 3353258,
+    "sha256": "4ba698529766c7a2f5052083ea10cdf2485577f5237a332ee291054eb11e3332"
+  },
+  {
+    "file": "Strata-data/packs/iq2_xs/tokenizer/token_type.json",
+    "bytes": 744960,
+    "sha256": "5088c8c298fc06af8382ddb3b76c888703ac2634e82263b97eedcc2ad202738b"
+  },
+  {
+    "file": "Strata-data/packs/iq2_xs/tokenizer/tokenizer.json",
+    "bytes": 541,
+    "sha256": "c0bd435efbc6c3b28964b47999afe16c4ad268c7c7c4e8cb3af410796a6f30bf"
+  },
+  {
+    "file": "Strata-data/packs/iq2_xs/tokenizer/vocab.json",
+    "bytes": 5737005,
+    "sha256": "4ba64f0332abcfb0b600b7df1537d1e836e68009d5fb7cdb77339738dc6365c4"
+  },
+  {
+    "file": "Strata/data/expert-profile.bin",
+    "bytes": 196632,
+    "sha256": "8f59b4aa8873209dff11c11e37bcda9529a1335b724a1afeea37bf6388975baf"
+  },
+  {
+    "file": "Strata/engine/strata",
+    "bytes": 43158256,
+    "sha256": "b7f18df1314ad438952ca11bb5ce5f23b41e7a572462426383d2ad8209fcf367"
+  },
+  {
+    "file": "Strata/engine/strata-vision",
+    "bytes": 74993888,
+    "sha256": "0d4c955f255ba4c3a9f6f456d63292754fae440a8a39cb13b9b6564938448e19"
+  }
+]

bench/results/2026-09-30-community-rtx-5090/benchmark.py

@@ -0,0 +1,135 @@
+#!/usr/bin/env python3
+"""Serial, fresh-prompt Strata benchmark. Run inside its prepared environment."""
+import argparse
+import hashlib
+import json
+import statistics
+import sys
+import time
+import urllib.request
+from pathlib import Path
+
+
+def get(url):
+    with urllib.request.urlopen(url, timeout=30) as response:
+        return json.load(response)
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--root", type=Path, default=Path("/workspace/Strata"))
+    parser.add_argument("--pack", type=Path, default=Path("/workspace/Strata-data/packs/iq2_xs"))
+    parser.add_argument("--url", default="http://127.0.0.1:18080")
+    parser.add_argument("--out", type=Path, default=Path("/workspace/results"))
+    parser.add_argument("--targets", default="4096,32768,128000")
+    parser.add_argument("--runs", type=int, default=3)
+    args = parser.parse_args()
+    args.out.mkdir(parents=True, exist_ok=True)
+    sys.path[:0] = [str(args.root), str(args.root / "tools")]
+    from strata_tokenizer import Tokenizer
+    from serve.frontend import ChatTemplate, openai_to_messages
+    directory = args.pack / "tokenizer"
+    vocab = json.loads((directory / "vocab.json").read_text())
+    tokens = [None] * len(vocab)
+    for token, number in vocab.items():
+        tokens[number] = token
+    tokenizer = Tokenizer(tokens, (directory / "merges.txt").read_text().splitlines(),
+                          json.loads((directory / "token_type.json").read_text()))
+    template = ChatTemplate(directory / "chat_template.jinja")
+    initial = get(args.url + "/v1/status")
+    (args.out / "initial-status.json").write_text(json.dumps(initial, indent=2) + "\n")
+    rows = []
+
+    def request(content, maximum=256):
+        return {"model": "strata", "messages": [{"role": "user", "content": content}],
+                "temperature": 0, "reasoning_effort": "none", "max_tokens": maximum,
+                "stream": True, "stream_options": {"include_usage": True}}
+
+    def count(req):
+        messages, tools, kwargs = openai_to_messages(req)
+        return len(tokenizer.encode(template.render(messages, tools, **kwargs), parse_special=True))
+
+    def perform(label, req, expected=None):
+        body = json.dumps(req).encode()
+        (args.out / (label + "-request.json")).write_bytes(body + b"\n")
+        print("start", label, "tokens", expected, flush=True)
+        chunks, texts, first, finish = [], [], None, None
+        started = time.perf_counter()
+        wire = urllib.request.Request(args.url + "/v1/chat/completions", data=body,
+                                      headers={"Content-Type": "application/json"})
+        with urllib.request.urlopen(wire, timeout=1800) as response:
+            for line in response:
+                if not line.startswith(b"data: "):
+                    continue
+                payload = line[6:].strip()
+                if payload == b"[DONE]":
+                    break
+                chunk = json.loads(payload)
+                chunks.append(chunk)
+                for choice in chunk.get("choices", []):
+                    delta = choice.get("delta", {})
+                    text = (delta.get("content") or "") + (delta.get("reasoning_content") or "")
+                    if text:
+                        first = first if first is not None else time.perf_counter() - started
+                        texts.append(text)
+                    finish = choice.get("finish_reason") or finish
+        elapsed = time.perf_counter() - started
+        metrics = get(args.url + "/metrics")
+        engine = metrics["requests"][0]
+        usage = next((chunk["usage"] for chunk in reversed(chunks) if "usage" in chunk), {})
+        row = {"label": label, "expected_prompt_tokens": expected,
+               "request_sha256": hashlib.sha256(body).hexdigest(),
+               "client_ttft_s": first, "client_elapsed_s": elapsed, "finish_reason": finish,
+               "usage": usage, "engine": engine, "text": "".join(texts)}
+        if expected is not None and engine["prompt_tokens"] != expected:
+            raise RuntimeError(f"token count mismatch: {expected} vs {engine['prompt_tokens']}")
+        if not texts:
+            raise RuntimeError(f"no text received for {label}")
+        (args.out / (label + "-raw.json")).write_text(json.dumps({"chunks": chunks, "row": row}, indent=2) + "\n")
+        rows.append(row)
+        (args.out / "results.json").write_text(json.dumps(rows, indent=2) + "\n")
+        print("done", label, json.dumps({"ttft_s": first, "elapsed_s": elapsed, "engine": engine}), flush=True)
+        return row
+
+    perform("warmup", request("Reply with exactly the word READY.", 16))
+    # Each nonce occurs before the filler so previous long prompts cannot be reused.
+    filler = "\n".join(f"def task_{i:05d}(value: int) -> int: return (value * {(i % 97) + 1} + {i}) % 100003"
+                       for i in range(12000))
+    ending = ("\n\nWrite a detailed explanation of the code above. Discuss deterministic integer transforms, "
+              "modulo arithmetic, testing, naming, complexity, and maintainability. Write at least 600 words.")
+    for target in map(int, args.targets.split(",")):
+        for run in range(1, args.runs + 1):
+            prefix = f"Benchmark nonce: series-{target}-trial-{run}.\nReview this synthetic Python module:\n"
+            lo, hi = 0, len(filler)
+            while lo < hi:
+                middle = (lo + hi + 1) // 2
+                if count(request(prefix + filler[:middle] + ending)) <= target:
+                    lo = middle
+                else:
+                    hi = middle - 1
+            req = request(prefix + filler[:lo] + ending)
+            actual = count(req)
+            if actual < target - 20 or actual > target:
+                raise RuntimeError(f"could not construct target {target}: {actual}")
+            perform(f"tokens-{target}-run-{run}", req, actual)
+    measured = [row for row in rows if row["label"] != "warmup"]
+    groups = {}
+    for target in map(int, args.targets.split(",")):
+        subset = [r for r in measured if r["label"].startswith(f"tokens-{target}-")]
+        values = {
+            "prompt_tokens": [r["engine"]["prompt_tokens"] for r in subset],
+            "cached_tokens": [r["engine"].get("reused", 0) for r in subset],
+            "output_tokens": [r["engine"]["engine_generated"] for r in subset],
+            "client_ttft_s": [r["client_ttft_s"] for r in subset],
+            "client_elapsed_s": [r["client_elapsed_s"] for r in subset],
+            "prefill_tok_s": [(r["engine"]["prompt_tokens"] - (r["engine"].get("reused") or 0)) / (r["engine"]["prompt_ms"] / 1000) for r in subset],
+            "decode_tok_s": [r["engine"]["engine_generated"] / (r["engine"]["decode_ms"] / 1000) for r in subset],
+        }
+        groups[str(target)] = {key: {"median": statistics.median(v), "min": min(v), "max": max(v)}
+                              for key, v in values.items()}
+    (args.out / "summary.json").write_text(json.dumps(groups, indent=2) + "\n")
+    print("summary", json.dumps(groups), flush=True)
+
+
+if __name__ == "__main__":
+    main()

bench/results/2026-09-30-community-rtx-5090/engine.log

@@ -0,0 +1,91 @@
+load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
+load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
+load_hparams: more info: https://github.com/ggml-org/llama.cpp/issues/16842
+
+strata-vision: warmed up at 1024 image tokens
+strata generate: PCIe probe: 50.0 GB/s host->device -> pcie_frac 0.55 (default 0.55)
+strata generate: this CPU has no AVX-512: the expert kernels run on AVX-2 (multi-token for the i-quant gate/up rows)
+strata generate: native pack: /workspace/Strata-data/packs/iq2_xs experts (largest blob 1.51 MB), token embedding type 23 in mapped host memory (322 MiB)
+strata generate: 1466 MiB of weights loaded from /workspace/Strata-data/packs/iq2_xs (302 canonical tensors skipped: served natively)
+strata generate: 300 native projection matrices, 1474.85 MiB of weights
+strata generate: KV streaming: 32768 of 131072 cells per QSA layer in VRAM, the K/V in 1.55 GiB of pinned RAM
+strata generate: PLE on, table 320001536 rows of /workspace/Strata-data/models/IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00002-of-00002.gguf
+strata mtp: draft layer loaded, 835 MiB of VRAM (experts 675, dense 111), files read in 0.23 s (3441 MiB/s)
+strata generate: expert arena: cudaHostRegister PORTABLE ok; MAP_HUGETLB unavailable (no hugetlb pool configured?); using 4 KB pages
+strata generate: loaded 33.02 GiB at 2.21 GiB/s
+strata generate: profile /workspace/Strata/data/expert-profile.bin: 24576 ranked pairs, built for 24576 slots
+strata generate: experimental native Q5_K head, 337715200 bytes
+strata generate: expert cache auto: 24.26 GiB free, 700 MiB reserved (+143 MiB for the draft head) -> 16663 slots
+strata generate: expert cache 17463 slots, 23.44 GiB of VRAM; policy is
+strata generate: the GPU computes the experts in the cache; it rounds differently from the CPU,
+                 so a reply can differ slightly from a run without the cache (same quality:
+                 bench/results/2026-09-27-cache-parity).
+                 PROFILE, ranked by routing frequency, no eviction.
+strata generate: pre-filled 17463 of 17463 slots from the profile; slot 0 verified
+strata generate: R4 hit path ON - resident experts are computed on the GPU
+strata generate: 23 expert-pool workers + the host thread
+strata generate: session is up (engine 0.1.29)
+strata generate: token graph hit path: 17463 resident experts, decided on the device
+strata serve: prompt chunk auto: 8192 tokens
+strata serve: the prompt path borrows 3421 cache slots (4.62 GiB)
+strata verify: window up to 6 tokens, 57.1 MiB of device buffers
+strata mtp: draft head over 106299 tokens (137.9 MiB)
+strata serve: 412 MiB of VRAM free with everything loaded
+strata verify: captured the 6-token window (upload no error, sync no error)
+strata verify: captured the 1-token window (upload no error, sync no error)
+strata verify: captured the 4-token window (upload no error, sync no error)
+strata serve: prompt 19 tokens = 0 reused + 19 read in 135 ms (140.9 tok/s), 2 generated in 57 ms (34.9 tok/s), drafts accepted 3 of 3, 1 checkpoints
+strata serve: decode expert cache hit rate: 90.8% (2037 hits / 2244 lookups)
+strata serve: KV streaming: 92.31% of 936 block reads hit VRAM, 0.3 MiB read from RAM
+strata verify: captured the 3-token window (upload no error, sync no error)
+strata verify: captured the 2-token window (upload no error, sync no error)
+strata serve: prompt 4096 tokens = 0 reused + 4096 read in 1167 ms (3510.2 tok/s), 256 generated in 1587 ms (161.3 tok/s), drafts accepted 149 of 226, 1 checkpoints
+strata serve: decode expert cache hit rate: 97.8% (155423 hits / 158942 lookups)
+strata serve: KV streaming: 99.39% of 2095092 block reads hit VRAM, 51.8 MiB read from RAM
+strata serve: suffix drafts: 3 windows, 6 of 9 drafts accepted
+strata serve: prompt 4096 tokens = 0 reused + 4096 read in 959 ms (4269.7 tok/s), 256 generated in 1385 ms (184.8 tok/s), drafts accepted 158 of 221, 1 checkpoints
+strata serve: decode expert cache hit rate: 99.3% (151655 hits / 152786 lookups)
+strata serve: KV streaming: 99.36% of 2002680 block reads hit VRAM, 51.7 MiB read from RAM
+strata serve: suffix drafts: 1 windows, 2 of 3 drafts accepted
+strata serve: prompt 4096 tokens = 0 reused + 4096 read in 958 ms (4273.4 tok/s), 256 generated in 1427 ms (179.4 tok/s), drafts accepted 160 of 226, 1 checkpoints
+strata serve: decode expert cache hit rate: 98.9% (153580 hits / 155293 lookups)
+strata serve: KV streaming: 99.37% of 2039616 block reads hit VRAM, 52.0 MiB read from RAM
+strata serve: suffix drafts: 2 windows, 2 of 8 drafts accepted
+strata serve: prompt 32768 tokens = 0 reused + 32768 read in 5929 ms (5527.1 tok/s), 256 generated in 1541 ms (166.1 tok/s), drafts accepted 148 of 218, 2 checkpoints
+strata serve: decode expert cache hit rate: 99.4% (155252 hits / 156260 lookups)
+strata serve: KV streaming: 97.72% of 2045772 block reads hit VRAM, 187.5 MiB read from RAM
+strata serve: prompt 32768 tokens = 0 reused + 32768 read in 5911 ms (5543.1 tok/s), 256 generated in 1408 ms (181.8 tok/s), drafts accepted 160 of 212, 2 checkpoints
+strata serve: decode expert cache hit rate: 99.4% (147298 hits / 148150 lookups)
+strata serve: KV streaming: 97.53% of 1941048 block reads hit VRAM, 193.2 MiB read from RAM
+strata serve: prompt 32768 tokens = 0 reused + 32768 read in 5905 ms (5549.6 tok/s), 256 generated in 1457 ms (175.7 tok/s), drafts accepted 153 of 206, 2 checkpoints
+strata serve: decode expert cache hit rate: 99.5% (148308 hits / 149107 lookups)
+strata serve: KV streaming: 97.56% of 1953372 block reads hit VRAM, 191.7 MiB read from RAM
+strata serve: suffix drafts: 2 windows, 3 of 8 drafts accepted
+strata serve: prompt 128000 tokens = 0 reused + 128000 read in 22100 ms (5791.9 tok/s), 256 generated in 1654 ms (154.8 tok/s), drafts accepted 149 of 218, 6 checkpoints
+strata serve: decode expert cache hit rate: 99.4% (155334 hits / 156245 lookups)
+strata serve: KV streaming: 96.13% of 2045748 block reads hit VRAM, 318.7 MiB read from RAM
+strata serve: prompt 128000 tokens = 0 reused + 128000 read in 22151 ms (5778.7 tok/s), 256 generated in 1512 ms (169.3 tok/s), drafts accepted 159 of 228, 6 checkpoints
+strata serve: decode expert cache hit rate: 99.7% (155406 hits / 155913 lookups)
+strata serve: KV streaming: 96.26% of 2039616 block reads hit VRAM, 307.0 MiB read from RAM
+strata serve: prompt 128000 tokens = 0 reused + 128000 read in 22195 ms (5767.0 tok/s), 256 generated in 1551 ms (165.0 tok/s), drafts accepted 154 of 221, 6 checkpoints
+strata serve: decode expert cache hit rate: 99.7% (155405 hits / 155914 lookups)
+strata serve: KV streaming: 96.22% of 2039628 block reads hit VRAM, 310.9 MiB read from RAM
+strata serve: suffix drafts: 1 windows, 1 of 2 drafts accepted
+strata serve: prompt 30804 tokens = 0 reused + 30804 read in 6382 ms (4826.8 tok/s), 9 generated in 55 ms (163.1 tok/s), drafts accepted 6 of 6, 2 checkpoints
+strata serve: decode expert cache hit rate: 92.6% (3832 hits / 4139 lookups)
+strata serve: KV streaming: 78.28% of 92424 block reads hit VRAM, 80.8 MiB read from RAM
+strata serve: prompt 30804 tokens = 0 reused + 30804 read in 5709 ms (5395.4 tok/s), 9 generated in 61 ms (147.0 tok/s), drafts accepted 6 of 6, 2 checkpoints
+strata serve: decode expert cache hit rate: 94.7% (3971 hits / 4192 lookups)
+strata serve: KV streaming: 78.55% of 92424 block reads hit VRAM, 79.9 MiB read from RAM
+strata serve: prompt 30805 tokens = 0 reused + 30805 read in 5718 ms (5387.6 tok/s), 10 generated in 66 ms (150.4 tok/s), drafts accepted 9 of 9, 2 checkpoints
+strata serve: decode expert cache hit rate: 96.3% (5885 hits / 6108 lookups)
+strata serve: KV streaming: 82.56% of 117072 block reads hit VRAM, 82.3 MiB read from RAM
+strata serve: prompt 126320 tokens = 0 reused + 126320 read in 22281 ms (5669.4 tok/s), 10 generated in 65 ms (154.9 tok/s), drafts accepted 7 of 7, 6 checkpoints
+strata serve: decode expert cache hit rate: 97.7% (5115 hits / 5233 lookups)
+strata serve: KV streaming: 77.02% of 104760 block reads hit VRAM, 97.0 MiB read from RAM
+strata serve: prompt 126318 tokens = 0 reused + 126318 read in 22453 ms (5626.0 tok/s), 9 generated in 105 ms (86.0 tok/s), drafts accepted 6 of 14, 6 checkpoints
+strata serve: decode expert cache hit rate: 97.4% (9224 hits / 9474 lookups)
+strata serve: KV streaming: 82.65% of 160212 block reads hit VRAM, 112.0 MiB read from RAM
+strata serve: prompt 126318 tokens = 16384 reused + 109934 read in 19791 ms (5554.7 tok/s), 9 generated in 78 ms (115.2 tok/s), drafts accepted 7 of 9, 6 checkpoints
+strata serve: decode expert cache hit rate: 98.1% (6539 hits / 6669 lookups)
+strata serve: KV streaming: 87.22% of 283464 block reads hit VRAM, 146.0 MiB read from RAM

bench/results/2026-09-30-community-rtx-5090/final-status.json

@@ -0,0 +1 @@
+{"service": "strata", "model": "qwen3.8-flash-next-iq2_xs", "engine": "0.1.29", "started": 1790772696, "uptime_s": 196, "cache_max_tokens": 131072, "context": {"native": 131072, "max_positions": 131072}, "concurrency": {"serving": 1, "requested": 1}, "dialects": ["/v1/chat/completions", "/v1/messages"], "vision": {"enabled": true, "available": true, "error": null}, "activity": {"requests": 16, "in_flight": 0, "last_request_at": 1790772892}, "last_timings": {"cache_n": 16384, "prompt_n": 109934, "prompt_ms": 19791.2, "prompt_per_token_ms": 0.18, "prompt_per_second": 5554.7, "predicted_n": 9, "predicted_ms": 78.1, "predicted_per_token_ms": 8.678, "predicted_per_second": 115.2, "draft_n": 9, "draft_n_accepted": 7, "at": 1790772892}, "machine": {"at": 1790772892, "gpu": {"name": "NVIDIA GeForce RTX 5090", "used_mib": 32281.0, "total_mib": 32607.0, "util_pct": 100, "temp_c": 72, "power_w": 528.823}, "ram": {"used_gib": 45.4, "total_gib": 62.2}}}

bench/results/2026-09-30-community-rtx-5090/initial-status.json

@@ -0,0 +1,46 @@
+{
+  "service": "strata",
+  "model": "qwen3.8-flash-next-iq2_xs",
+  "engine": "0.1.29",
+  "started": 1790772696,
+  "uptime_s": 2,
+  "cache_max_tokens": 131072,
+  "context": {
+    "native": 131072,
+    "max_positions": 131072
+  },
+  "concurrency": {
+    "serving": 1,
+    "requested": 1
+  },
+  "dialects": [
+    "/v1/chat/completions",
+    "/v1/messages"
+  ],
+  "vision": {
+    "enabled": true,
+    "available": true,
+    "error": null
+  },
+  "activity": {
+    "requests": 0,
+    "in_flight": 0,
+    "last_request_at": null
+  },
+  "last_timings": null,
+  "machine": {
+    "at": 1790772698,
+    "gpu": {
+      "name": "NVIDIA GeForce RTX 5090",
+      "used_mib": 32195.0,
+      "total_mib": 32607.0,
+      "util_pct": 0,
+      "temp_c": 36,
+      "power_w": 83.403
+    },
+    "ram": {
+      "used_gib": 44.4,
+      "total_gib": 62.2
+    }
+  }
+}

bench/results/2026-09-30-community-rtx-5090/memory-summary.json

@@ -0,0 +1,10 @@
+{
+  "samples": 250,
+  "sample_duration_s": 255.8539924621582,
+  "sampling_method": "nominal 1 s; /proc/meminfo and nvidia-smi; startup through recall checks",
+  "ram_peak_GiB": 45.54487228393555,
+  "gpu_peak_MiB": 31785.0,
+  "swap_start_MiB": 0.25,
+  "swap_peak_MiB": 425.5,
+  "swap_end_MiB": 425.5
+}

bench/results/2026-09-30-community-rtx-5090/model-provenance.json

@@ -0,0 +1,46 @@
+[
+  {
+    "repository": "ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF",
+    "revision": "ed59f92082b1e93c0e96d60a8b11aab089b52f09",
+    "gguf_files": [
+      {
+        "filename": "IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf",
+        "lfs": {
+          "sha256": "92cee27ae5bbadcd732416a0f7a7f0acc092399dbbe8f5a5efa707c2ec0a49d7",
+          "size": 39225954592,
+          "pointerSize": 136
+        },
+        "size": 39225954592,
+        "actual_sha256": "92cee27ae5bbadcd732416a0f7a7f0acc092399dbbe8f5a5efa707c2ec0a49d7",
+        "verified": true
+      },
+      {
+        "filename": "IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00002-of-00002.gguf",
+        "lfs": {
+          "sha256": "316b46f3a2dbd68c900f43136ab9449f9dcc3725dfd8c794847c204bc161e113",
+          "size": 28800138432,
+          "pointerSize": 136
+        },
+        "size": 28800138432,
+        "actual_sha256": "316b46f3a2dbd68c900f43136ab9449f9dcc3725dfd8c794847c204bc161e113",
+        "verified": true
+      },
+      {
+        "filename": "mmproj-Qwen3.8-Flash-Next-BF16.gguf",
+        "lfs": {
+          "sha256": "b1a82259702816a5330d7bd7607cd9676b11780e79ff7348c21103ff3ce49bd0",
+          "size": 907543008,
+          "pointerSize": 134
+        },
+        "size": 907543008,
+        "actual_sha256": "b1a82259702816a5330d7bd7607cd9676b11780e79ff7348c21103ff3ce49bd0",
+        "verified": true
+      }
+    ]
+  },
+  {
+    "repository": "Qwen/Qwen3.8-Flash-Next",
+    "revision": "de4b8e4d43b917e7706784d8bb445c9af86a3540",
+    "gguf_files": []
+  }
+]

bench/results/2026-09-30-community-rtx-5090/monitor.py

@@ -0,0 +1,18 @@
+"""Sample system RAM, swap and GPU memory once per second during the trial."""
+import json
+import subprocess
+import sys
+import time
+from pathlib import Path
+
+with Path(sys.argv[1] if len(sys.argv) > 1 else 'telemetry.jsonl').open('a') as output:
+    while True:
+        memory = {}
+        for line in Path('/proc/meminfo').read_text().splitlines():
+            key, value = line.split(':', 1)
+            if key in ('MemTotal', 'MemAvailable', 'SwapTotal', 'SwapFree'):
+                memory[key + '_KiB'] = int(value.strip().split()[0])
+        gpu = subprocess.check_output(['nvidia-smi', '--query-gpu=memory.used,memory.total,utilization.gpu,power.draw,temperature.gpu', '--format=csv,noheader,nounits'], text=True).strip()
+        output.write(json.dumps({'epoch_s': time.time(), 'memory': memory, 'gpu': gpu}) + '\n')
+        output.flush()
+        time.sleep(1)

bench/results/2026-09-30-community-rtx-5090/mtp-manifest.json

@@ -0,0 +1,429 @@
+[
+ {
+  "name": "mtp.fc_embedding.weight",
+  "shard": "model-00120-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   2560,
+   2560
+  ],
+  "start": 1935412640,
+  "end": 1948519839,
+  "bytes": 13107200,
+  "file": "tensors/mtp.fc_embedding.weight.bin",
+  "sha256": "241d6e87ded317a61b1aeb3ca4e28c53f1caba09ad5be66092e91fe5e3e14751"
+ },
+ {
+  "name": "mtp.fc_hidden.weight",
+  "shard": "model-00124-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   2560,
+   2560
+  ],
+  "start": 1716599928,
+  "end": 1729707127,
+  "bytes": 13107200,
+  "file": "tensors/mtp.fc_hidden.weight.bin",
+  "sha256": "97c8e427ad0177e2c713ed24a397d815b523eda15ef453f1ae6463f6c70a8f96"
+ },
+ {
+  "name": "mtp.hyper_connection_mixer.hc_norm.weight",
+  "shard": "model-00037-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   10240
+  ],
+  "start": 1742326680,
+  "end": 1742347159,
+  "bytes": 20480,
+  "file": "tensors/mtp.hyper_connection_mixer.hc_norm.weight.bin",
+  "sha256": "ce948a398e15aeaa403c424cda34e6eed296ab2570d591bb5ff3da642b255d64"
+ },
+ {
+  "name": "mtp.hyper_connection_mixer.input_mix_weight_down.weight",
+  "shard": "model-00041-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   320,
+   10240
+  ],
+  "start": 1935412688,
+  "end": 1941966287,
+  "bytes": 6553600,
+  "file": "tensors/mtp.hyper_connection_mixer.input_mix_weight_down.weight.bin",
+  "sha256": "c633d2841fda463e82045e0b93fb8cb8d511ec0c9eda616020f01aca18d71947"
+ },
+ {
+  "name": "mtp.hyper_connection_mixer.input_mix_weight_up.weight",
+  "shard": "model-00043-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   10240,
+   320
+  ],
+  "start": 1832518416,
+  "end": 1839072015,
+  "bytes": 6553600,
+  "file": "tensors/mtp.hyper_connection_mixer.input_mix_weight_up.weight.bin",
+  "sha256": "6565285f0ba2923703252763cfc11d8a9bda40ab42e7548e12464b04dbdbec13"
+ },
+ {
+  "name": "mtp.layers.0.attn_hyper_connection.block_inject_weight.weight",
+  "shard": "model-00047-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   4,
+   10240
+  ],
+  "start": 1716599984,
+  "end": 1716681903,
+  "bytes": 81920,
+  "file": "tensors/mtp.layers.0.attn_hyper_connection.block_inject_weight.weight.bin",
+  "sha256": "c8afff64e7f54412d02120097c4255033c55245e1c1eed84cc630d428ccb864c"
+ },
+ {
+  "name": "mtp.layers.0.attn_hyper_connection.hc_norm.weight",
+  "shard": "model-00049-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   10240
+  ],
+  "start": 1935412672,
+  "end": 1935433151,
+  "bytes": 20480,
+  "file": "tensors/mtp.layers.0.attn_hyper_connection.hc_norm.weight.bin",
+  "sha256": "1fb619aeaaf637ccd70f46f1f8f8116e6a46448cb43026b246de8ee8286ed809"
+ },
+ {
+  "name": "mtp.layers.0.attn_hyper_connection.input_mix_weight_down.weight",
+  "shard": "model-00053-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   320,
+   10240
+  ],
+  "start": 1832518424,
+  "end": 1839072023,
+  "bytes": 6553600,
+  "file": "tensors/mtp.layers.0.attn_hyper_connection.input_mix_weight_down.weight.bin",
+  "sha256": "e80ac8bf3e270e9d5a949ec7c913329dc8de017d69851334820fae7b8c86a7f5"
+ },
+ {
+  "name": "mtp.layers.0.attn_hyper_connection.input_mix_weight_up.weight",
+  "shard": "model-00057-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   10240,
+   320
+  ],
+  "start": 1935412680,
+  "end": 1941966279,
+  "bytes": 6553600,
+  "file": "tensors/mtp.layers.0.attn_hyper_connection.input_mix_weight_up.weight.bin",
+  "sha256": "f80f1fd44e1183e7764ea496b11ff89ead9dd758f861d4e79a770bf54956b1a5"
+ },
+ {
+  "name": "mtp.layers.0.mlp.experts.down_proj",
+  "shard": "model-00062-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   512,
+   2560,
+   640
+  ],
+  "start": 1832518400,
+  "end": 3510239999,
+  "bytes": 1677721600,
+  "file": "tensors/mtp.layers.0.mlp.experts.down_proj.bin",
+  "sha256": "539c46234f73bac70450f731f40e715f77ac32faf77def0f3f800e58aaabbbc0"
+ },
+ {
+  "name": "mtp.layers.0.mlp.experts.gate_up_proj",
+  "shard": "model-00060-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   512,
+   1280,
+   2560
+  ],
+  "start": 115918328,
+  "end": 3471361527,
+  "bytes": 3355443200,
+  "file": "tensors/mtp.layers.0.mlp.experts.gate_up_proj.bin",
+  "sha256": "839e98f9642ebc884b178abd2090b6dc8b0bcd81ed92d7f7a08314c6a68f96d5"
+ },
+ {
+  "name": "mtp.layers.0.mlp.gate.weight",
+  "shard": "model-00066-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   512,
+   2560
+  ],
+  "start": 1716599952,
+  "end": 1719221391,
+  "bytes": 2621440,
+  "file": "tensors/mtp.layers.0.mlp.gate.weight.bin",
+  "sha256": "f0d35fa41b360a08bc0c5b1e5766aa42df960291716ee189208b5d2599462c20"
+ },
+ {
+  "name": "mtp.layers.0.mlp.shared_expert.down_proj.weight",
+  "shard": "model-00076-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   2560,
+   640
+  ],
+  "start": 1935412680,
+  "end": 1938689479,
+  "bytes": 3276800,
+  "file": "tensors/mtp.layers.0.mlp.shared_expert.down_proj.weight.bin",
+  "sha256": "d2ced8c81367cc494599d1a8cdc279c3dd6579b628ded05acd7495e99cee01d8"
+ },
+ {
+  "name": "mtp.layers.0.mlp.shared_expert.gate_proj.weight",
+  "shard": "model-00072-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   640,
+   2560
+  ],
+  "start": 1832518528,
+  "end": 1835795327,
+  "bytes": 3276800,
+  "file": "tensors/mtp.layers.0.mlp.shared_expert.gate_proj.weight.bin",
+  "sha256": "f4a81109165853b3b64f36d809f98094b599763364070f5fa079200e1c386d4c"
+ },
+ {
+  "name": "mtp.layers.0.mlp.shared_expert.up_proj.weight",
+  "shard": "model-00072-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   640,
+   2560
+  ],
+  "start": 1835795328,
+  "end": 1839072127,
+  "bytes": 3276800,
+  "file": "tensors/mtp.layers.0.mlp.shared_expert.up_proj.weight.bin",
+  "sha256": "172586861eea53efbff10afc43ded5b6e49c83879e9efc01e089c652bbb0a156"
+ },
+ {
+  "name": "mtp.layers.0.mlp.shared_expert_gate.weight",
+  "shard": "model-00070-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   1,
+   2560
+  ],
+  "start": 1832518400,
+  "end": 1832523519,
+  "bytes": 5120,
+  "file": "tensors/mtp.layers.0.mlp.shared_expert_gate.weight.bin",
+  "sha256": "acdd306d9996d356e4667a16ff63af6fef086fc7f570f1bc00bbe882a2a93f6e"
+ },
+ {
+  "name": "mtp.layers.0.mlp_hyper_connection.block_inject_weight.weight",
+  "shard": "model-00078-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   4,
+   10240
+  ],
+  "start": 1832518424,
+  "end": 1832600343,
+  "bytes": 81920,
+  "file": "tensors/mtp.layers.0.mlp_hyper_connection.block_inject_weight.weight.bin",
+  "sha256": "c5782dd92b479a1f8ef43d683fe9c917fd9e8fba81f23c1aae8817bfe4b476a6"
+ },
+ {
+  "name": "mtp.layers.0.mlp_hyper_connection.hc_norm.weight",
+  "shard": "model-00082-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   10240
+  ],
+  "start": 1935412656,
+  "end": 1935433135,
+  "bytes": 20480,
+  "file": "tensors/mtp.layers.0.mlp_hyper_connection.hc_norm.weight.bin",
+  "sha256": "21a504876b5e7b788301557c68dfb757999430eba36c66ec8ab96a63604dbfd1"
+ },
+ {
+  "name": "mtp.layers.0.mlp_hyper_connection.input_mix_weight_down.weight",
+  "shard": "model-00086-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   320,
+   10240
+  ],
+  "start": 1935412696,
+  "end": 1941966295,
+  "bytes": 6553600,
+  "file": "tensors/mtp.layers.0.mlp_hyper_connection.input_mix_weight_down.weight.bin",
+  "sha256": "b71240036dea954e034d5ae04e097bcd7ac51cb28a44e4c93d198af3447278a1"
+ },
+ {
+  "name": "mtp.layers.0.mlp_hyper_connection.input_mix_weight_up.weight",
+  "shard": "model-00088-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   10240,
+   320
+  ],
+  "start": 1832518424,
+  "end": 1839072023,
+  "bytes": 6553600,
+  "file": "tensors/mtp.layers.0.mlp_hyper_connection.input_mix_weight_up.weight.bin",
+  "sha256": "40756866bf2a5e82c28e407f717b6fa07614d593f78e79c7ceb2cd4ffe6dda94"
+ },
+ {
+  "name": "mtp.layers.0.self_attn.indexer.index_qk_proj.weight",
+  "shard": "model-00090-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   640,
+   2560
+  ],
+  "start": 1832518416,
+  "end": 1835795215,
+  "bytes": 3276800,
+  "file": "tensors/mtp.layers.0.self_attn.indexer.index_qk_proj.weight.bin",
+  "sha256": "2befafae6e944c1fc651fd73d4d527eb6f138a6c1679065b003128ee3150e859"
+ },
+ {
+  "name": "mtp.layers.0.self_attn.indexer.k_layernorm.weight",
+  "shard": "model-00094-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   128
+  ],
+  "start": 1935412672,
+  "end": 1935412927,
+  "bytes": 256,
+  "file": "tensors/mtp.layers.0.self_attn.indexer.k_layernorm.weight.bin",
+  "sha256": "2db621f6a08b4085756fc69d60a4a1bd1d7fbf5d406456d2477075842efaae54"
+ },
+ {
+  "name": "mtp.layers.0.self_attn.indexer.q_layernorm.weight",
+  "shard": "model-00100-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   128
+  ],
+  "start": 1716599968,
+  "end": 1716600223,
+  "bytes": 256,
+  "file": "tensors/mtp.layers.0.self_attn.indexer.q_layernorm.weight.bin",
+  "sha256": "afaf59f68e251939970ffdeacad0667dffc657a522bea62cdc44a52816fc49d6"
+ },
+ {
+  "name": "mtp.layers.0.self_attn.k_norm.weight",
+  "shard": "model-00102-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   256
+  ],
+  "start": 1935412648,
+  "end": 1935413159,
+  "bytes": 512,
+  "file": "tensors/mtp.layers.0.self_attn.k_norm.weight.bin",
+  "sha256": "8f924b0357cf936e14fc34518205d000e1423aa60626827df81b0804fae8cdfd"
+ },
+ {
+  "name": "mtp.layers.0.self_attn.k_proj.weight",
+  "shard": "model-00108-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   512,
+   2560
+  ],
+  "start": 1832518624,
+  "end": 1835140063,
+  "bytes": 2621440,
+  "file": "tensors/mtp.layers.0.self_attn.k_proj.weight.bin",
+  "sha256": "1a295dc2d6e633e808aeb19e00c97856a1817bbfcc83f5b610e2bfcacfcb8c38"
+ },
+ {
+  "name": "mtp.layers.0.self_attn.o_proj.weight",
+  "shard": "model-00106-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   2560,
+   6144
+  ],
+  "start": 1832518400,
+  "end": 1863975679,
+  "bytes": 31457280,
+  "file": "tensors/mtp.layers.0.self_attn.o_proj.weight.bin",
+  "sha256": "9d2fcf0ac6ac19de87f638f710cfd1af6ad14141e75b7d3d5d29fbd06f374219"
+ },
+ {
+  "name": "mtp.layers.0.self_attn.q_norm.weight",
+  "shard": "model-00112-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   256
+  ],
+  "start": 1935412664,
+  "end": 1935413175,
+  "bytes": 512,
+  "file": "tensors/mtp.layers.0.self_attn.q_norm.weight.bin",
+  "sha256": "57957aa1e7c5a3ac628bff0ec5522ad55a39b8c0b8bd2d324ae640e3771617ef"
+ },
+ {
+  "name": "mtp.layers.0.self_attn.q_proj.weight",
+  "shard": "model-00108-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   12288,
+   2560
+  ],
+  "start": 1835140064,
+  "end": 1898054623,
+  "bytes": 62914560,
+  "file": "tensors/mtp.layers.0.self_attn.q_proj.weight.bin",
+  "sha256": "5220aef3d288f40759f7356af9ef453e5ac0256a38749a9bb803a4e9db58150d"
+ },
+ {
+  "name": "mtp.layers.0.self_attn.v_proj.weight",
+  "shard": "model-00108-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   512,
+   2560
+  ],
+  "start": 1898054624,
+  "end": 1900676063,
+  "bytes": 2621440,
+  "file": "tensors/mtp.layers.0.self_attn.v_proj.weight.bin",
+  "sha256": "39cb12f5424e965fa71fb81db0ee230c701b7b1e0b72c36641fde7e0e41bff41"
+ },
+ {
+  "name": "mtp.pre_fc_norm_embedding.weight",
+  "shard": "model-00114-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   2560
+  ],
+  "start": 1832518392,
+  "end": 1832523511,
+  "bytes": 5120,
+  "file": "tensors/mtp.pre_fc_norm_embedding.weight.bin",
+  "sha256": "04c4a570850e06f2d8913da8220d54d4c7f87db6eb6d45480b938e8ba41d6a86"
+ },
+ {
+  "name": "mtp.pre_fc_norm_hidden.weight",
+  "shard": "model-00118-of-00131.safetensors",
+  "dtype": "BF16",
+  "shape": [
+   10240
+  ],
+  "start": 1716599944,
+  "end": 1716620423,
+  "bytes": 20480,
+  "file": "tensors/mtp.pre_fc_norm_hidden.weight.bin",
+  "sha256": "b933b44559879449b1baf2af8385c5c237347307e3fd452394870cb53e6eab6d"
+ }
+]

bench/results/2026-09-30-community-rtx-5090/needles.json

@@ -0,0 +1,56 @@
+[
+ {
+  "length": "32k",
+  "depth": 10,
+  "prompt_tokens": 30804,
+  "seconds": 6.5,
+  "found": true,
+  "word": "meadow-copper-504",
+  "answer": "meadow-copper-504"
+ },
+ {
+  "length": "32k",
+  "depth": 50,
+  "prompt_tokens": 30804,
+  "seconds": 5.8,
+  "found": true,
+  "word": "falcon-quartz-940",
+  "answer": "falcon-quartz-940"
+ },
+ {
+  "length": "32k",
+  "depth": 90,
+  "prompt_tokens": 30805,
+  "seconds": 5.8,
+  "found": true,
+  "word": "willow-cobalt-696",
+  "answer": "willow-cobalt-696"
+ },
+ {
+  "length": "128k",
+  "depth": 10,
+  "prompt_tokens": 126320,
+  "seconds": 22.5,
+  "found": true,
+  "word": "falcon-saffron-138",
+  "answer": "falcon-saffron-138"
+ },
+ {
+  "length": "128k",
+  "depth": 50,
+  "prompt_tokens": 126318,
+  "seconds": 22.8,
+  "found": true,
+  "word": "quartz-tundra-528",
+  "answer": "quartz-tundra-528"
+ },
+ {
+  "length": "128k",
+  "depth": 90,
+  "prompt_tokens": 126318,
+  "seconds": 20.1,
+  "found": true,
+  "word": "quartz-glacier-192",
+  "answer": "quartz-glacier-192"
+ }
+]

bench/results/2026-09-30-community-rtx-5090/python-packages.txt

@@ -0,0 +1,15 @@
+certifi==2026.7.22
+charset-normalizer==3.5.2
+cmake==4.4.3
+idna==3.20
+Jinja2==3.1.6
+MarkupSafe==3.0.3
+ninja==1.13.2
+numpy==2.5.3
+pillow==12.3.0
+psutil==7.2.2
+PyYAML==6.0.3
+regex==2026.9.29
+requests==2.34.2
+tqdm==4.70.1
+urllib3==2.8.0

bench/results/2026-09-30-community-rtx-5090/results.json

@@ -0,0 +1,312 @@
+[
+  {
+    "label": "warmup",
+    "expected_prompt_tokens": null,
+    "request_sha256": "b63e57873441a996a529b246a66cf12de70a7c671777f99329354d5c195f788f",
+    "client_ttft_s": 0.15764825499991275,
+    "client_elapsed_s": 0.19575888999997915,
+    "finish_reason": "stop",
+    "usage": {
+      "prompt_tokens": 19,
+      "completion_tokens": 2,
+      "total_tokens": 21,
+      "prompt_tokens_details": {
+        "cached_tokens": 0
+      }
+    },
+    "engine": {
+      "projection": null,
+      "time": 1790772698.967025,
+      "duration_s": 0.2,
+      "finish": "stop",
+      "prompt_tokens": 19,
+      "reused": 0,
+      "output_tokens": 2,
+      "engine_generated": 2,
+      "prompt_ms": 134.9,
+      "decode_ms": 57.4,
+      "decode_tok_s": 34.8,
+      "hit_rate": 0.908
+    },
+    "text": "READY"
+  },
+  {
+    "label": "tokens-4096-run-1",
+    "expected_prompt_tokens": 4096,
+    "request_sha256": "5d94c652067354bbe72930d2bdad3886564f08cf2546b32411af02d33ad3ce05",
+    "client_ttft_s": 1.1827884900003482,
+    "client_elapsed_s": 2.7608979090000503,
+    "finish_reason": "length",
+    "usage": {
+      "prompt_tokens": 4096,
+      "completion_tokens": 256,
+      "total_tokens": 4352,
+      "prompt_tokens_details": {
+        "cached_tokens": 0
+      }
+    },
+    "engine": {
+      "projection": null,
+      "time": 1790772699.5014164,
+      "duration_s": 2.8,
+      "finish": "length",
+      "prompt_tokens": 4096,
+      "reused": 0,
+      "output_tokens": 256,
+      "engine_generated": 256,
+      "prompt_ms": 1166.9,
+      "decode_ms": 1587.0,
+      "decode_tok_s": 161.3,
+      "hit_rate": 0.978
+    },
+    "text": "# Analysis of Synthetic Python Module: Deterministic Integer Transforms\n\nThe provided Python module consists of a series of functions, `task_00000` through `task_00112`, each implementing a specific deterministic integer transformation. While the code appears repetitive, it serves as a clear example of modular arithmetic, function design, and the trade-offs between explicit implementation and algorithmic abstraction. Below is a detailed discussion covering the mathematical basis, testing implications, naming conventions, complexity, and maintainability.\n\n## 1. Mathematical Basis: Deterministic Transforms and Modulo Arithmetic\n\nAt its core, every function in this module performs a linear transformation on an input integer `value`. The general pattern for `task_NNNNN` is:\n\n$$ f(x) = (x \\cdot a + b) \\pmod{M} $$\n\nWhere:\n- $M = 100003$ is a fixed modulus.\n- $a$ is the multiplier.\n- $b$ is the additive constant.\n\nFor most functions (e.g., `task_00000` to `task_00096`), the multiplier $a$ and the additive constant $b$ follow a"
+  },
+  {
+    "label": "tokens-4096-run-2",
+    "expected_prompt_tokens": 4096,
+    "request_sha256": "8da8a04e70e4d916f6630729082bdfb9d6ebbb47117aad60bc0541bce198a213",
+    "client_ttft_s": 0.9743533680002656,
+    "client_elapsed_s": 2.3506501550000394,
+    "finish_reason": "length",
+    "usage": {
+      "prompt_tokens": 4096,
+      "completion_tokens": 256,
+      "total_tokens": 4352,
+      "prompt_tokens_details": {
+        "cached_tokens": 0
+      }
+    },
+    "engine": {
+      "projection": null,
+      "time": 1790772702.5992615,
+      "duration_s": 2.3,
+      "finish": "length",
+      "prompt_tokens": 4096,
+      "reused": 0,
+      "output_tokens": 256,
+      "engine_generated": 256,
+      "prompt_ms": 959.3,
+      "decode_ms": 1385.0,
+      "decode_tok_s": 184.8,
+      "hit_rate": 0.993
+    },
+    "text": "# Analysis of Synthetic Python Module: Deterministic Integer Transforms\n\nThe provided Python module consists of a series of functions, `task_00000` through `task_00112`, each implementing a specific deterministic integer transformation. While the code appears repetitive, it serves as a clear example of modular arithmetic, function design, and the trade-offs between explicit implementation and algorithmic abstraction. Below is a detailed discussion of the code's structure, mathematical properties, testing implications, and maintainability.\n\n## 1. Mathematical Structure and Deterministic Transforms\n\nAt its core, every function in this module implements a linear congruential generator (LCG) step or a simple affine transformation modulo a prime number. The general form of each function is:\n\n$$ f_n(x) = (x \\cdot a_n + b_n) \\pmod{M} $$\n\nWhere:\n- $M = 100003$ is a fixed modulus.\n- $a_n$ is the multiplier for function $n$.\n- $b_n$ is the additive constant for function $n$.\n\nFor the first 97 functions (`task_00000` to `task_00096`"
+  },
+  {
+    "label": "tokens-4096-run-3",
+    "expected_prompt_tokens": 4096,
+    "request_sha256": "946079edb58e764050805886156dc8f877cc29e1ca8d01c84e9a1e7917d49492",
+    "client_ttft_s": 0.9733456560002196,
+    "client_elapsed_s": 2.391696067000339,
+    "finish_reason": "length",
+    "usage": {
+      "prompt_tokens": 4096,
+      "completion_tokens": 256,
+      "total_tokens": 4352,
+      "prompt_tokens_details": {
+        "cached_tokens": 0
+      }
+    },
+    "engine": {
+      "projection": null,
+      "time": 1790772705.2866879,
+      "duration_s": 2.4,
+      "finish": "length",
+      "prompt_tokens": 4096,
+      "reused": 0,
+      "output_tokens": 256,
+      "engine_generated": 256,
+      "prompt_ms": 958.5,
+      "decode_ms": 1427.2,
+      "decode_tok_s": 179.4,
+      "hit_rate": 0.989
+    },
+    "text": "# Analysis of Synthetic Python Module: Deterministic Integer Transforms\n\nThe provided Python module consists of a series of functions, `task_00000` through `task_00112`, each implementing a specific deterministic integer transformation. While the code appears repetitive, it serves as a clear example of modular arithmetic, function design, and the trade-offs between explicit implementation and algorithmic abstraction. Below is a detailed discussion of the code\u2019s structure, mathematical properties, testing implications, naming conventions, complexity, and maintainability.\n\n## 1. Mathematical Structure and Deterministic Transforms\n\nEach function follows the general form:\n```python\ndef task_XXXXX(value: int) -> int:\n    return (value * multiplier + addend) % MOD\n```\nwhere `MOD = 100003`.\n\nFor most functions, the multiplier and addend are derived from the function index `i`:\n- `multiplier = i + 1`\n- `addend = i`\n\nThus, for `task_00000`, `multiplier = 1`, `addend = 0`.\nFor `task_00001`, `multiplier = 2`, `"
+  },
+  {
+    "label": "tokens-32768-run-1",
+    "expected_prompt_tokens": 32768,
+    "request_sha256": "e37cfc2ab9867d91130927440df24a57255a736b8d0d829d07d828e22e3de572",
+    "client_ttft_s": 5.9679448089996185,
+    "client_elapsed_s": 7.498308491999978,
+    "finish_reason": "length",
+    "usage": {
+      "prompt_tokens": 32768,
+      "completion_tokens": 256,
+      "total_tokens": 33024,
+      "prompt_tokens_details": {
+        "cached_tokens": 0
+      }
+    },
+    "engine": {
+      "projection": null,
+      "time": 1790772708.3260226,
+      "duration_s": 7.5,
+      "finish": "length",
+      "prompt_tokens": 32768,
+      "reused": 0,
+      "output_tokens": 256,
+      "engine_generated": 256,
+      "prompt_ms": 5928.6,
+      "decode_ms": 1541.2,
+      "decode_tok_s": 166.1,
+      "hit_rate": 0.994
+    },
+    "text": "Based on the provided Python module, here is a detailed technical review and explanation.\n\n### Overview of the Module\n\nThe provided code defines a series of functions, `task_00000` through `task_00888`, that perform deterministic integer transformations. Each function takes an integer `value` and returns a new integer based on a specific linear formula involving multiplication, addition, and modulo arithmetic. The benchmark nonce `series-32768-trial-1` suggests this code is part of a larger test suite or performance benchmark, likely designed to evaluate computational consistency, hash generation, or state machine transitions.\n\n### Mathematical Logic and Deterministic Transforms\n\nAt its core, every function in this module implements a linear congruential generator (LCG) style transformation, albeit with varying parameters. The general structure of each function is:\n\n$$ f(x) = (x \\cdot a + b) \\pmod{100003} $$\n\nWhere:\n*   $x$ is the input `value`.\n*   $a$ is the multiplier (coefficient).\n*   $b$ is the additive constant.\n*   $100003$ is the modulus."
+  },
+  {
+    "label": "tokens-32768-run-2",
+    "expected_prompt_tokens": 32768,
+    "request_sha256": "ed2dfaa11aecad9f4563f33bb5f5d4b8704486508700b8fa572722cc1225ee26",
+    "client_ttft_s": 5.950024114999906,
+    "client_elapsed_s": 7.348446455000158,
+    "finish_reason": "length",
+    "usage": {
+      "prompt_tokens": 32768,
+      "completion_tokens": 256,
+      "total_tokens": 33024,
+      "prompt_tokens_details": {
+        "cached_tokens": 0
+      }
+    },
+    "engine": {
+      "projection": null,
+      "time": 1790772716.4558623,
+      "duration_s": 7.3,
+      "finish": "length",
+      "prompt_tokens": 32768,
+      "reused": 0,
+      "output_tokens": 256,
+      "engine_generated": 256,
+      "prompt_ms": 5911.4,
+      "decode_ms": 1408.3,
+      "decode_tok_s": 181.8,
+      "hit_rate": 0.994
+    },
+    "text": "Based on the provided synthetic Python module, here is a detailed technical review and explanation.\n\n### 1. Structural Analysis and Pattern Recognition\n\nThe module consists of a series of functions named `task_00000` through `task_00888` (the list appears truncated at `task_00888`, but the pattern implies a larger set). Each function follows a strict, deterministic signature: `def task_XXXXX(value: int) -> int`.\n\nThe core logic within each function is a linear transformation followed by a modulo operation:\n$$ f(x) = (x \\cdot a + b) \\pmod{100003} $$\n\nWhere:\n*   $x$ is the input `value`.\n*   $a$ is the multiplier.\n*   $b$ is the additive constant.\n*   $100003$ is the modulus.\n\n#### The Multiplier Pattern ($a$)\nThe multiplier $a$ follows a cyclic pattern. It starts at 1 and increments by 1 for each function until it reaches 97. After `task_00096` (where $a=97$), the"
+  },
+  {
+    "label": "tokens-32768-run-3",
+    "expected_prompt_tokens": 32768,
+    "request_sha256": "2a3c7342622e80c3cd8b37e062f0a2f64640c119ece8cccbf168639041eac079",
+    "client_ttft_s": 5.941602006999801,
+    "client_elapsed_s": 7.389697677000186,
+    "finish_reason": "length",
+    "usage": {
+      "prompt_tokens": 32768,
+      "completion_tokens": 256,
+      "total_tokens": 33024,
+      "prompt_tokens_details": {
+        "cached_tokens": 0
+      }
+    },
+    "engine": {
+      "projection": null,
+      "time": 1790772724.435358,
+      "duration_s": 7.4,
+      "finish": "length",
+      "prompt_tokens": 32768,
+      "reused": 0,
+      "output_tokens": 256,
+      "engine_generated": 256,
+      "prompt_ms": 5904.6,
+      "decode_ms": 1457.1,
+      "decode_tok_s": 175.7,
+      "hit_rate": 0.995
+    },
+    "text": "Based on the provided Python module, here is a detailed technical review and explanation.\n\n### 1. Structural Overview and Pattern Analysis\n\nThe provided code consists of a series of function definitions, ranging from `task_00000` to `task_00888`. Each function follows a strict, repetitive structure:\n\n```python\ndef task_XXXXX(value: int) -> int:\n    return (value * A + B) % 100003\n```\n\nThis module is a synthetic representation of a **linear congruential generator (LCG)** or a set of **affine transformations** applied to an integer input. The core logic is consistent across all functions, with only the coefficients `A` (multiplier) and `B` (addend) changing.\n\n### 2. Deterministic Integer Transforms and Modulo Arithmetic\n\nThe mathematical core of this module is the expression `(value * A + B) % 100003`. This is a classic example of a **deterministic integer transform**.\n\n*   **Determinism:** For any given input `value`, the output is always the same. There is no randomness, state, or side effects. This makes"
+  },
+  {
+    "label": "tokens-128000-run-1",
+    "expected_prompt_tokens": 128000,
+    "request_sha256": "3235aac3f934cf1d889c25973ca172b355509dc2522fdfb463fbefe979a9075d",
+    "client_ttft_s": 22.212050286999784,
+    "client_elapsed_s": 23.856042405000153,
+    "finish_reason": "length",
+    "usage": {
+      "prompt_tokens": 128000,
+      "completion_tokens": 256,
+      "total_tokens": 128256,
+      "prompt_tokens_details": {
+        "cached_tokens": 0
+      }
+    },
+    "engine": {
+      "projection": null,
+      "time": 1790772733.6124191,
+      "duration_s": 23.8,
+      "finish": "length",
+      "prompt_tokens": 128000,
+      "reused": 0,
+      "output_tokens": 256,
+      "engine_generated": 256,
+      "prompt_ms": 22099.7,
+      "decode_ms": 1653.5,
+      "decode_tok_s": 154.8,
+      "hit_rate": 0.994
+    },
+    "text": "Based on the provided synthetic Python module, here is a detailed technical review and explanation.\n\n### Overview of the Module\n\nThe provided code consists of a massive collection of function definitions, ranging from `task_00000` to `task_03404` (and potentially beyond, as the input cuts off). Each function follows a strict, repetitive pattern. The core logic of every function is identical in structure, differing only in the specific integer constants used for multiplication and addition.\n\nThe general form of any function `task_N` is:\n```python\ndef task_N(value: int) -> int:\n    return (value * multiplier + addend) % 100003\n```\n\n### 1. Deterministic Integer Transforms and Modulo Arithmetic\n\nThe primary mechanism in this module is a **deterministic linear transform** followed by a **modulo operation**.\n\n**The Linear Transform:**\nFor any given `value`, the function computes `value * multiplier + addend`. This is an affine transformation. In mathematics, an affine transformation is a function that preserves points, straight lines, and planes. Here, it maps an input integer to an output integer based on a fixed slope (`multiplier`) and"
+  },
+  {
+    "label": "tokens-128000-run-2",
+    "expected_prompt_tokens": 128000,
+    "request_sha256": "5f4b8bc9f0afc7765320abd89960c0938e0d7cf5974e962fdbd0613fef31c8d4",
+    "client_ttft_s": 22.262365384000077,
+    "client_elapsed_s": 23.764614322999932,
+    "finish_reason": "length",
+    "usage": {
+      "prompt_tokens": 128000,
+      "completion_tokens": 256,
+      "total_tokens": 128256,
+      "prompt_tokens_details": {
+        "cached_tokens": 0
+      }
+    },
+    "engine": {
+      "projection": null,
+      "time": 1790772759.195895,
+      "duration_s": 23.7,
+      "finish": "length",
+      "prompt_tokens": 128000,
+      "reused": 0,
+      "output_tokens": 256,
+      "engine_generated": 256,
+      "prompt_ms": 22150.5,
+      "decode_ms": 1511.7,
+      "decode_tok_s": 169.3,
+      "hit_rate": 0.997
+    },
+    "text": "# Analysis of the Synthetic Python Module\n\nThe provided Python module consists of a massive collection of functions, ranging from `task_00000` to `task_03404` (and potentially beyond, as the code is truncated). Each function follows a strict, repetitive pattern: it accepts an integer `value`, applies a linear transformation, and returns the result modulo a fixed prime number, `100003`.\n\n## 1. Mathematical Structure and Deterministic Transforms\n\nAt its core, every function in this module implements a **linear congruential generator (LCG)** step or a simple affine transformation. The general formula for any function `task_N` is:\n\n$$ f_N(x) = (x \\cdot a_N + b_N) \\pmod{M} $$\n\nWhere:\n*   $M = 100003$ is the modulus.\n*   $a_N$ is the multiplier for task $N$.\n*   $b_N$ is the additive constant for task $N$.\n\n### The Modulus: 100003\nThe choice of `100003` is significant. It is a **prime number**. In"
+  },
+  {
+    "label": "tokens-128000-run-3",
+    "expected_prompt_tokens": 128000,
+    "request_sha256": "5c3fb71db9f9f44765f28389d1032f2c8b6687d1bf5ca6f33aeb924072cfa6e4",
+    "client_ttft_s": 22.308453708999878,
+    "client_elapsed_s": 23.850161715000013,
+    "finish_reason": "length",
+    "usage": {
+      "prompt_tokens": 128000,
+      "completion_tokens": 256,
+      "total_tokens": 128256,
+      "prompt_tokens_details": {
+        "cached_tokens": 0
+      }
+    },
+    "engine": {
+      "projection": null,
+      "time": 1790772784.6969988,
+      "duration_s": 23.8,
+      "finish": "length",
+      "prompt_tokens": 128000,
+      "reused": 0,
+      "output_tokens": 256,
+      "engine_generated": 256,
+      "prompt_ms": 22195.2,
+      "decode_ms": 1551.4,
+      "decode_tok_s": 165.0,
+      "hit_rate": 0.997
+    },
+    "text": "Based on the provided synthetic Python module, here is a detailed technical review and explanation.\n\n### Overview of the Module\n\nThe provided code consists of a massive collection of function definitions, ranging from `task_00000` to `task_03404`. Each function accepts a single integer argument `value` and returns an integer. The core logic within every function is identical in structure, differing only in the specific coefficients used for multiplication and addition.\n\nThe general pattern for any function `task_N` is:\n```python\ndef task_N(value: int) -> int:\n    return (value * multiplier + addend) % 100003\n```\n\n### Deterministic Integer Transforms and Modulo Arithmetic\n\nThe functions implement a **deterministic linear transform** over the integers. In mathematics, this is an affine transformation defined by $f(x) = ax + b$. However, the code applies this transformation within a finite field defined by the modulus $M = 100003$.\n\n1.  **Modulo Arithmetic**: The use of `% 100003` ensures that the output is always within the range $[0, 100002"
+  }
+]

bench/results/2026-09-30-community-rtx-5090/strata-iq2_xs.json

@@ -0,0 +1,50 @@
+{
+ "exe": "/workspace/Strata/engine/strata",
+ "args": [
+  "--pack",
+  "/workspace/Strata-data/packs/iq2_xs",
+  "--native",
+  "/workspace/Strata-data/models/IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf",
+  "--ple-gguf",
+  "/workspace/Strata-data/models/IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00002-of-00002.gguf",
+  "--expert-profile",
+  "/workspace/Strata/data/expert-profile.bin",
+  "--expert-cache",
+  "auto",
+  "--prefill",
+  "auto",
+  "--spec",
+  "4",
+  "--spec-min-p",
+  "0.5",
+  "--mtp",
+  "/workspace/Strata-data/mtp/rt",
+  "--max-context",
+  "131072",
+  "--kv",
+  "int8",
+  "--kv-resident",
+  "32768",
+  "--vision",
+  "--vram-reserve-mib",
+  "700"
+ ],
+ "cwd": "/workspace/Strata",
+ "tokenizer": "/workspace/Strata-data/packs/iq2_xs/tokenizer",
+ "model_name": "qwen3.8-flash-next-iq2_xs",
+ "log": "/workspace/Strata/strata-iq2_xs.log",
+ "lib_dirs": [
+  "/usr/local/cuda/bin",
+  "/usr/local/cuda/lib64"
+ ],
+ "port": 18080,
+ "gpu": 0,
+ "gpus_asked": true,
+ "vision": {
+  "exe": "/workspace/Strata/engine/strata-vision",
+  "mmproj": "/workspace/Strata-data/models/mmproj-Qwen3.8-Flash-Next-BF16.gguf",
+  "model": "/workspace/Strata-data/models/IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf",
+  "gpu": true,
+  "max_tokens": 1024
+ }
+}

bench/results/2026-09-30-community-rtx-5090/summary.json

@@ -0,0 +1,113 @@
+{
+  "4096": {
+    "prompt_tokens": {
+      "median": 4096,
+      "min": 4096,
+      "max": 4096
+    },
+    "cached_tokens": {
+      "median": 0,
+      "min": 0,
+      "max": 0
+    },
+    "output_tokens": {
+      "median": 256,
+      "min": 256,
+      "max": 256
+    },
+    "client_ttft_s": {
+      "median": 0.9743533680002656,
+      "min": 0.9733456560002196,
+      "max": 1.1827884900003482
+    },
+    "client_elapsed_s": {
+      "median": 2.391696067000339,
+      "min": 2.3506501550000394,
+      "max": 2.7608979090000503
+    },
+    "prefill_tok_s": {
+      "median": 4269.780047951632,
+      "min": 3510.1551118347757,
+      "max": 4273.343766301513
+    },
+    "decode_tok_s": {
+      "median": 179.37219730941703,
+      "min": 161.31064902331443,
+      "max": 184.8375451263538
+    }
+  },
+  "32768": {
+    "prompt_tokens": {
+      "median": 32768,
+      "min": 32768,
+      "max": 32768
+    },
+    "cached_tokens": {
+      "median": 0,
+      "min": 0,
+      "max": 0
+    },
+    "output_tokens": {
+      "median": 256,
+      "min": 256,
+      "max": 256
+    },
+    "client_ttft_s": {
+      "median": 5.950024114999906,
+      "min": 5.941602006999801,
+      "max": 5.9679448089996185
+    },
+    "client_elapsed_s": {
+      "median": 7.389697677000186,
+      "min": 7.348446455000158,
+      "max": 7.498308491999978
+    },
+    "prefill_tok_s": {
+      "median": 5543.187738945089,
+      "min": 5527.105893465573,
+      "max": 5549.571520509433
+    },
+    "decode_tok_s": {
+      "median": 175.69144190515408,
+      "min": 166.10433428497274,
+      "max": 181.77945040119295
+    }
+  },
+  "128000": {
+    "prompt_tokens": {
+      "median": 128000,
+      "min": 128000,
+      "max": 128000
+    },
+    "cached_tokens": {
+      "median": 0,
+      "min": 0,
+      "max": 0
+    },
+    "output_tokens": {
+      "median": 256,
+      "min": 256,
+      "max": 256
+    },
+    "client_ttft_s": {
+      "median": 22.262365384000077,
+      "min": 22.212050286999784,
+      "max": 22.308453708999878
+    },
+    "client_elapsed_s": {
+      "median": 23.850161715000013,
+      "min": 23.764614322999932,
+      "max": 23.856042405000153
+    },
+    "prefill_tok_s": {
+      "median": 5778.6505947947,
+      "min": 5767.012687427912,
+      "max": 5791.933827156025
+    },
+    "decode_tok_s": {
+      "median": 165.01224700270723,
+      "min": 154.82310250982763,
+      "max": 169.34576966329297
+    }
+  }
+}

bench/results/2026-09-30-community-rtx-5090/telemetry.jsonl

@@ -0,0 +1,250 @@
+{"epoch_s": 1790772668.438457, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 58332840, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "2, 32607, 1, 60.13, 36"}
+{"epoch_s": 1790772669.461572, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 57749256, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "1374, 32607, 25, 53.81, 36"}
+{"epoch_s": 1790772670.483304, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 56952232, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "3719, 32607, 2, 130.61, 36"}
+{"epoch_s": 1790772671.5050805, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 56947812, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "3719, 32607, 3, 86.30, 36"}
+{"epoch_s": 1790772672.5268102, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 56930224, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "4949, 32607, 3, 62.43, 36"}
+{"epoch_s": 1790772673.5491986, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 54141020, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "6875, 32607, 2, 59.82, 36"}
+{"epoch_s": 1790772674.584606, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 46827008, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "6875, 32607, 0, 58.26, 36"}
+{"epoch_s": 1790772675.6100278, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 39487436, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "6875, 32607, 0, 57.70, 36"}
+{"epoch_s": 1790772676.6331685, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 32761416, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "6875, 32607, 0, 57.36, 36"}
+{"epoch_s": 1790772677.6556065, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 24833136, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "6875, 32607, 0, 57.03, 36"}
+{"epoch_s": 1790772679.1176229, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 20348260, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "6893, 32607, 0, 57.33, 36"}
+{"epoch_s": 1790772680.1513186, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19836220, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "6941, 32607, 0, 58.46, 36"}
+{"epoch_s": 1790772681.18351, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19863628, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "6941, 32607, 0, 57.35, 36"}
+{"epoch_s": 1790772682.2173855, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19918256, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "6941, 32607, 0, 56.50, 36"}
+{"epoch_s": 1790772683.2403264, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19836728, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "6941, 32607, 0, 57.14, 36"}
+{"epoch_s": 1790772684.263194, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19812576, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "6941, 32607, 0, 57.01, 36"}
+{"epoch_s": 1790772685.2866416, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19832496, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8388348}, "gpu": "6941, 32607, 0, 56.75, 36"}
+{"epoch_s": 1790772686.3135202, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19950804, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8387324}, "gpu": "6941, 32607, 0, 56.74, 36"}
+{"epoch_s": 1790772687.3367112, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19874724, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8386812}, "gpu": "6941, 32607, 0, 56.90, 36"}
+{"epoch_s": 1790772688.3587284, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19842500, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8386556}, "gpu": "6941, 32607, 0, 57.05, 36"}
+{"epoch_s": 1790772689.3818972, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19944936, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8384252}, "gpu": "6941, 32607, 0, 57.04, 36"}
+{"epoch_s": 1790772690.406354, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19871504, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8379900}, "gpu": "6941, 32607, 0, 56.74, 36"}
+{"epoch_s": 1790772691.4390683, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19883588, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8376316}, "gpu": "6941, 32607, 0, 31.70, 35"}
+{"epoch_s": 1790772692.470097, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19895676, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8372220}, "gpu": "6941, 32607, 0, 20.18, 35"}
+{"epoch_s": 1790772693.5028179, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 20038008, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8203772}, "gpu": "6941, 32607, 0, 11.67, 35"}
+{"epoch_s": 1790772694.5281374, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 20156748, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8075004}, "gpu": "31267, 32607, 33, 28.99, 35"}
+{"epoch_s": 1790772695.5503595, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 19302012, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8075004}, "gpu": "31351, 32607, 1, 96.33, 36"}
+{"epoch_s": 1790772696.5729709, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18691716, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8075004}, "gpu": "31699, 32607, 1, 88.24, 36"}
+{"epoch_s": 1790772697.5949724, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18600496, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8075004}, "gpu": "31699, 32607, 0, 83.40, 36"}
+{"epoch_s": 1790772698.6135545, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18594848, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8075004}, "gpu": "31699, 32607, 0, 56.88, 36"}
+{"epoch_s": 1790772699.6351354, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18384636, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31729, 32607, 38, 71.98, 36"}
+{"epoch_s": 1790772700.6604307, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18135500, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31769, 32607, 99, 227.76, 45"}
+{"epoch_s": 1790772701.682456, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18126208, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31783, 32607, 95, 359.78, 43"}
+{"epoch_s": 1790772702.706684, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18118008, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31783, 32607, 2, 244.34, 38"}
+{"epoch_s": 1790772703.7335, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18095652, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31783, 32607, 98, 409.97, 43"}
+{"epoch_s": 1790772704.7526512, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18096236, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31783, 32607, 97, 346.22, 45"}
+{"epoch_s": 1790772705.7763088, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18088644, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31783, 32607, 99, 290.87, 48"}
+{"epoch_s": 1790772706.799191, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18089132, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31783, 32607, 100, 370.80, 45"}
+{"epoch_s": 1790772707.8183546, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18072284, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31783, 32607, 97, 348.14, 45"}
+{"epoch_s": 1790772708.8408382, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18071772, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31783, 32607, 99, 204.77, 49"}
+{"epoch_s": 1790772709.8652713, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18084484, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31785, 32607, 99, 502.69, 49"}
+{"epoch_s": 1790772710.8869991, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18091232, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31785, 32607, 99, 477.46, 52"}
+{"epoch_s": 1790772711.9077418, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18051376, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31785, 32607, 100, 453.75, 50"}
+{"epoch_s": 1790772712.945486, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18059000, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31785, 32607, 98, 483.63, 48"}
+{"epoch_s": 1790772713.9662626, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18068208, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 8013308}, "gpu": "31785, 32607, 99, 482.43, 53"}
+{"epoch_s": 1790772714.9933336, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17987384, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 93, 346.04, 47"}
+{"epoch_s": 1790772716.0165942, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17960560, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 96, 347.16, 48"}
+{"epoch_s": 1790772717.039366, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18132960, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 211.30, 51"}
+{"epoch_s": 1790772718.0643644, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18138544, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 508.40, 53"}
+{"epoch_s": 1790772719.0876768, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18147832, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 475.77, 55"}
+{"epoch_s": 1790772720.1091926, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18103444, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 457.60, 53"}
+{"epoch_s": 1790772721.1299815, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18100992, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 507.29, 54"}
+{"epoch_s": 1790772722.1500635, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18112212, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 476.48, 52"}
+{"epoch_s": 1790772723.1765406, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18011920, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 97, 382.48, 50"}
+{"epoch_s": 1790772724.1962311, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18017628, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 0, 274.45, 45"}
+{"epoch_s": 1790772725.21881, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18114148, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 365.93, 54"}
+{"epoch_s": 1790772726.2438593, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18128360, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 479.71, 54"}
+{"epoch_s": 1790772727.2632627, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18143844, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 510.50, 56"}
+{"epoch_s": 1790772728.2869742, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18150700, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 459.35, 56"}
+{"epoch_s": 1790772729.3204784, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18162336, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 480.67, 58"}
+{"epoch_s": 1790772730.3480747, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18062716, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 98, 494.31, 53"}
+{"epoch_s": 1790772731.3745997, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18056756, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 94, 311.79, 52"}
+{"epoch_s": 1790772732.3941443, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18080332, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 0, 272.26, 47"}
+{"epoch_s": 1790772733.4142647, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18087264, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 0, 134.10, 46"}
+{"epoch_s": 1790772734.4370096, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18208276, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 222.96, 54"}
+{"epoch_s": 1790772735.4588313, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18211508, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 513.78, 58"}
+{"epoch_s": 1790772736.4805841, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18202380, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 513.06, 58"}
+{"epoch_s": 1790772737.503986, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18205432, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 494.72, 57"}
+{"epoch_s": 1790772738.5267396, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18196648, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 514.25, 59"}
+{"epoch_s": 1790772739.5491192, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18054812, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 496.71, 60"}
+{"epoch_s": 1790772740.5723188, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18048512, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 515.80, 58"}
+{"epoch_s": 1790772741.5950322, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17928312, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 513.71, 58"}
+{"epoch_s": 1790772742.6171873, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17944052, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 496.11, 60"}
+{"epoch_s": 1790772743.639043, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17940264, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 513.91, 60"}
+{"epoch_s": 1790772744.6600034, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17833128, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 52, 496.12, 54"}
+{"epoch_s": 1790772745.6818032, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17835068, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 497.48, 61"}
+{"epoch_s": 1790772746.7033856, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17830900, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 516.30, 62"}
+{"epoch_s": 1790772747.7276258, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17715960, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 496.96, 61"}
+{"epoch_s": 1790772748.7482834, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17719684, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 513.01, 60"}
+{"epoch_s": 1790772749.7694225, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17610640, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 507.09, 59"}
+{"epoch_s": 1790772750.7923675, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17603368, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 485.92, 60"}
+{"epoch_s": 1790772751.812411, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17595640, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 479.74, 64"}
+{"epoch_s": 1790772752.8319893, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17586396, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 515.18, 63"}
+{"epoch_s": 1790772753.8533502, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17622816, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 458.79, 63"}
+{"epoch_s": 1790772754.8752813, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17623492, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 479.38, 61"}
+{"epoch_s": 1790772755.9013746, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17629392, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 49, 450.65, 56"}
+{"epoch_s": 1790772756.922106, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17626016, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 308.26, 58"}
+{"epoch_s": 1790772757.9418297, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17607068, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 0, 295.22, 53"}
+{"epoch_s": 1790772758.9605587, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17603144, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 0, 130.95, 52"}
+{"epoch_s": 1790772759.9839687, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18282120, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 210.90, 61"}
+{"epoch_s": 1790772761.006678, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18282984, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 515.74, 62"}
+{"epoch_s": 1790772762.0434818, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18162840, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 520.51, 62"}
+{"epoch_s": 1790772763.067759, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18150136, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 501.61, 62"}
+{"epoch_s": 1790772764.089317, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18166848, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 520.68, 64"}
+{"epoch_s": 1790772765.1115272, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18031696, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 500.77, 64"}
+{"epoch_s": 1790772766.1324067, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18034652, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 521.99, 64"}
+{"epoch_s": 1790772767.1545649, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17937296, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 521.65, 66"}
+{"epoch_s": 1790772768.176144, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17929464, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 501.06, 64"}
+{"epoch_s": 1790772769.198718, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17920788, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 520.32, 65"}
+{"epoch_s": 1790772770.236203, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17799416, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 497.43, 64"}
+{"epoch_s": 1790772771.257868, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17792564, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 523.67, 66"}
+{"epoch_s": 1790772772.2799609, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17801924, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 520.28, 67"}
+{"epoch_s": 1790772773.3021274, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17687256, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 498.60, 66"}
+{"epoch_s": 1790772774.335996, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17678064, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 521.94, 66"}
+{"epoch_s": 1790772775.35813, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17557356, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 521.58, 67"}
+{"epoch_s": 1790772776.3823485, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17565428, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 489.46, 66"}
+{"epoch_s": 1790772777.405725, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17569088, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 100, 487.07, 68"}
+{"epoch_s": 1790772778.4254801, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17570184, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7990524}, "gpu": "31785, 32607, 99, 520.51, 66"}
+{"epoch_s": 1790772779.4490557, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17625120, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 470.02, 67"}
+{"epoch_s": 1790772780.473841, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17616188, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 487.32, 64"}
+{"epoch_s": 1790772781.499789, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17505864, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 488.28, 67"}
+{"epoch_s": 1790772782.523191, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17517408, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 98, 328.98, 62"}
+{"epoch_s": 1790772783.5435653, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17539636, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 303.74, 57"}
+{"epoch_s": 1790772784.565325, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17527416, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 136.73, 56"}
+{"epoch_s": 1790772785.5888178, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18242896, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 217.75, 64"}
+{"epoch_s": 1790772786.612396, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18244028, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 524.56, 66"}
+{"epoch_s": 1790772787.6342883, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18194804, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 526.80, 66"}
+{"epoch_s": 1790772788.6568644, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18203332, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 507.39, 67"}
+{"epoch_s": 1790772789.6793633, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18191904, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 521.05, 67"}
+{"epoch_s": 1790772790.7027128, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18079592, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 503.11, 67"}
+{"epoch_s": 1790772791.7242851, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18083776, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 527.45, 67"}
+{"epoch_s": 1790772792.7456532, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17980956, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 98, 505.96, 67"}
+{"epoch_s": 1790772793.768554, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17984860, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 521.32, 68"}
+{"epoch_s": 1790772794.7907064, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17972956, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 527.85, 66"}
+{"epoch_s": 1790772795.8138406, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17850288, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 505.34, 68"}
+{"epoch_s": 1790772796.8353448, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17848116, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 527.36, 69"}
+{"epoch_s": 1790772797.8563817, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17846184, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 526.18, 69"}
+{"epoch_s": 1790772798.8780813, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17723912, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 503.19, 68"}
+{"epoch_s": 1790772799.899205, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17715168, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 523.83, 69"}
+{"epoch_s": 1790772800.9237013, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17593528, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 526.99, 70"}
+{"epoch_s": 1790772801.9461932, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17589112, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 492.53, 69"}
+{"epoch_s": 1790772802.9673676, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17602032, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 497.32, 69"}
+{"epoch_s": 1790772803.9868026, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17581748, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 524.68, 70"}
+{"epoch_s": 1790772805.006854, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17479100, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 467.85, 71"}
+{"epoch_s": 1790772806.027366, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17469612, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 98, 494.48, 66"}
+{"epoch_s": 1790772807.054402, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17465264, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 508.83, 66"}
+{"epoch_s": 1790772808.0764651, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17451560, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 98, 341.96, 65"}
+{"epoch_s": 1790772809.0972805, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18242708, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 326.58, 60"}
+{"epoch_s": 1790772810.1193252, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18224312, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 231.55, 67"}
+{"epoch_s": 1790772811.1443434, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18251360, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 98, 513.76, 65"}
+{"epoch_s": 1790772812.1641836, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18257168, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 507.31, 68"}
+{"epoch_s": 1790772813.188216, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18264160, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 475.36, 68"}
+{"epoch_s": 1790772814.2106729, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18280248, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 491.24, 70"}
+{"epoch_s": 1790772815.2351635, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18226696, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 43, 408.46, 65"}
+{"epoch_s": 1790772816.2571764, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18232216, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 516.37, 70"}
+{"epoch_s": 1790772817.2793994, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18237844, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 492.67, 71"}
+{"epoch_s": 1790772818.3027427, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18258420, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 85, 480.93, 69"}
+{"epoch_s": 1790772819.3249714, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18262736, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 520.06, 70"}
+{"epoch_s": 1790772820.3447847, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18263392, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 495.94, 70"}
+{"epoch_s": 1790772821.3689287, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18252008, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 387.84, 70"}
+{"epoch_s": 1790772822.391625, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18251760, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 530.36, 70"}
+{"epoch_s": 1790772823.4142845, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18246696, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 499.02, 69"}
+{"epoch_s": 1790772824.4363837, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18255784, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 474.00, 72"}
+{"epoch_s": 1790772825.4570014, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18249508, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 534.05, 68"}
+{"epoch_s": 1790772826.4777033, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18265224, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 488.60, 68"}
+{"epoch_s": 1790772827.5001867, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18241316, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 338.84, 69"}
+{"epoch_s": 1790772828.5244417, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18234208, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 529.48, 69"}
+{"epoch_s": 1790772829.5457335, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18255808, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 532.12, 72"}
+{"epoch_s": 1790772830.5685184, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18259268, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 507.47, 71"}
+{"epoch_s": 1790772831.5907855, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18256224, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 532.53, 71"}
+{"epoch_s": 1790772832.613174, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18178900, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 531.43, 64"}
+{"epoch_s": 1790772833.6395922, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18163872, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 510.42, 72"}
+{"epoch_s": 1790772834.660667, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18164500, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 529.48, 72"}
+{"epoch_s": 1790772835.682373, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18067600, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 514.49, 71"}
+{"epoch_s": 1790772836.7051973, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18067096, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 530.68, 71"}
+{"epoch_s": 1790772837.7258525, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17941220, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 52, 511.56, 71"}
+{"epoch_s": 1790772838.747495, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17946888, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 532.64, 72"}
+{"epoch_s": 1790772839.7696228, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17948264, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 529.32, 72"}
+{"epoch_s": 1790772840.7897382, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17841772, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 513.67, 71"}
+{"epoch_s": 1790772841.811278, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17838756, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 528.76, 72"}
+{"epoch_s": 1790772842.8329659, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17838812, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 531.90, 71"}
+{"epoch_s": 1790772843.8551986, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17717288, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 490.94, 72"}
+{"epoch_s": 1790772844.8746738, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17722396, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 528.11, 71"}
+{"epoch_s": 1790772845.8973172, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17723676, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 493.89, 72"}
+{"epoch_s": 1790772846.91816, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17609420, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 85, 479.12, 71"}
+{"epoch_s": 1790772847.939702, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17606604, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 528.62, 70"}
+{"epoch_s": 1790772848.9587715, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17605816, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 482.68, 71"}
+{"epoch_s": 1790772849.9811416, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18225312, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 2, 287.55, 62"}
+{"epoch_s": 1790772851.0086215, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18219312, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 495.29, 70"}
+{"epoch_s": 1790772852.0307431, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18212012, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 532.53, 73"}
+{"epoch_s": 1790772853.0534828, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18241908, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 513.63, 71"}
+{"epoch_s": 1790772854.0752146, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18240392, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 534.22, 71"}
+{"epoch_s": 1790772855.0970914, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18192424, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 531.69, 73"}
+{"epoch_s": 1790772856.119393, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18176072, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 511.80, 72"}
+{"epoch_s": 1790772857.1404707, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18174816, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 531.85, 73"}
+{"epoch_s": 1790772858.1629918, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18066588, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 52, 512.33, 71"}
+{"epoch_s": 1790772859.1853132, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18066664, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 512.57, 72"}
+{"epoch_s": 1790772860.206362, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18066844, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 532.25, 71"}
+{"epoch_s": 1790772861.2286925, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17941852, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 511.81, 74"}
+{"epoch_s": 1790772862.2505696, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17961916, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 530.91, 72"}
+{"epoch_s": 1790772863.2736192, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17842052, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 52, 515.36, 72"}
+{"epoch_s": 1790772864.2948847, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17837396, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 530.06, 72"}
+{"epoch_s": 1790772865.315923, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17838760, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 531.20, 71"}
+{"epoch_s": 1790772866.3383734, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17722852, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 77, 489.91, 74"}
+{"epoch_s": 1790772867.3589919, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17723784, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 531.00, 72"}
+{"epoch_s": 1790772868.3802085, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17724116, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 492.00, 72"}
+{"epoch_s": 1790772869.4032836, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17610436, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 519.67, 68"}
+{"epoch_s": 1790772870.4251704, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17593928, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 482.79, 71"}
+{"epoch_s": 1790772871.446933, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17625500, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 490.11, 71"}
+{"epoch_s": 1790772872.4693067, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18115856, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 42, 376.82, 63"}
+{"epoch_s": 1790772873.4927008, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18130848, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 413.33, 71"}
+{"epoch_s": 1790772874.517678, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18129660, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 531.46, 71"}
+{"epoch_s": 1790772875.539969, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18122464, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 96, 513.83, 71"}
+{"epoch_s": 1790772876.5611775, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18124472, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 525.10, 73"}
+{"epoch_s": 1790772877.5850198, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18126548, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 533.66, 72"}
+{"epoch_s": 1790772878.6076293, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18073280, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 511.73, 72"}
+{"epoch_s": 1790772879.629884, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 18098184, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 535.61, 74"}
+{"epoch_s": 1790772880.6510599, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17972812, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 531.98, 73"}
+{"epoch_s": 1790772881.675439, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17976576, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 507.68, 72"}
+{"epoch_s": 1790772882.699754, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17973148, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 533.44, 72"}
+{"epoch_s": 1790772883.722258, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17861120, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 508.82, 72"}
+{"epoch_s": 1790772884.745278, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17864404, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 531.88, 72"}
+{"epoch_s": 1790772885.769315, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17866576, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 530.83, 74"}
+{"epoch_s": 1790772886.7924066, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17752540, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 488.78, 72"}
+{"epoch_s": 1790772887.817747, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17751168, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 99, 527.99, 69"}
+{"epoch_s": 1790772888.8397095, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17735964, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 496.67, 72"}
+{"epoch_s": 1790772889.8620257, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17625844, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 470.64, 71"}
+{"epoch_s": 1790772890.884023, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17629264, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 528.82, 72"}
+{"epoch_s": 1790772891.9039407, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17631268, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 100, 475.71, 71"}
+{"epoch_s": 1790772892.925761, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17658484, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 242.06, 62"}
+{"epoch_s": 1790772893.947652, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17660788, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 124.52, 62"}
+{"epoch_s": 1790772894.9685812, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17662644, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 94.57, 60"}
+{"epoch_s": 1790772895.9877603, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17666884, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 70.56, 59"}
+{"epoch_s": 1790772897.009143, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17675152, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 69.94, 59"}
+{"epoch_s": 1790772898.0305054, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17677208, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 69.68, 58"}
+{"epoch_s": 1790772899.0518734, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17686920, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 69.10, 58"}
+{"epoch_s": 1790772900.0737712, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17694368, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 68.64, 57"}
+{"epoch_s": 1790772901.0949237, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17701040, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 68.54, 57"}
+{"epoch_s": 1790772902.1162262, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17704052, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 68.07, 56"}
+{"epoch_s": 1790772903.1371994, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17716036, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 67.54, 56"}
+{"epoch_s": 1790772904.1573613, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17729424, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 66.79, 56"}
+{"epoch_s": 1790772905.178784, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17734376, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 66.78, 55"}
+{"epoch_s": 1790772906.2002432, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17741828, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 66.73, 55"}
+{"epoch_s": 1790772907.2226708, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17744104, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 66.62, 54"}
+{"epoch_s": 1790772908.2435381, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17741708, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 65.50, 54"}
+{"epoch_s": 1790772909.2710078, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17724256, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 40.74, 53"}
+{"epoch_s": 1790772910.2964864, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17746392, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 31.75, 53"}
+{"epoch_s": 1790772911.3218958, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17745548, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 19.67, 53"}
+{"epoch_s": 1790772912.3480182, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17746012, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 14.41, 52"}
+{"epoch_s": 1790772913.3754818, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17733056, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 14.42, 52"}
+{"epoch_s": 1790772914.402011, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17739736, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 14.55, 52"}
+{"epoch_s": 1790772915.4289236, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17733576, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 14.24, 51"}
+{"epoch_s": 1790772916.4575953, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17753656, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 14.35, 51"}
+{"epoch_s": 1790772917.4829435, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17752892, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 14.47, 51"}
+{"epoch_s": 1790772918.5089724, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17753536, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 12.31, 51"}
+{"epoch_s": 1790772919.5349586, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17755604, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 12.49, 51"}
+{"epoch_s": 1790772920.5655951, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17766328, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 14.14, 50"}
+{"epoch_s": 1790772921.5931342, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 17743004, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "31785, 32607, 0, 14.21, 50"}
+{"epoch_s": 1790772922.637384, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 20934704, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "30128, 32607, 4, 16.71, 50"}
+{"epoch_s": 1790772924.2924495, "memory": {"MemTotal_KiB": 65208820, "MemAvailable_KiB": 37200652, "SwapTotal_KiB": 8388604, "SwapFree_KiB": 7952892}, "gpu": "2, 32607, 0, 60.26, 50"}

bench/results/2026-09-30-tc-gemv-sm75/README.md

@@ -0,0 +1,57 @@
+# BF16 GEMV on Turing tensor cores (sm_75): measured negative
+
+i7-8700K + RTX 2070 8 GB (sm_75, 36 SMs), engine 0.1.27 tree (`bf16_gemv.cu` is unchanged through 0.1.30),
+qwen3.8-flash-next (~125.7B MoE, GGUF-verified) Q2_0 in production. Isolated benchmark only -
+[`bench_tc_gemv.cu`](bench_tc_gemv.cu) touches nothing in the inference path. Question: the QSA prompt attention
+already rides the Turing tensor cores (`mma.sync.m16n8k8`, the #270 path); can the decode GEMVs do the same?
+
+## Setup
+
+- Baseline: the production `bf16_gemv` warp kernel, transcribed verbatim
+- TC variant: `mma.sync.aligned.m16n8k8.row.col.f32.f16.f16.f32` on BF16->FP16 (the sm_75 MMA has native f16 only).
+  BF16->FP16 conversion is exact (0 / 2,000,000 values inexact - bf16's 7-bit mantissa fits in fp16's 10 bits); the
+  A-fragment packing is the same one #270 validated (`a0` = row `gid`, k columns `2t..2t+1`; broadcast makes
+  `a0 == a1 == {x[2t], x[2t+1]}`)
+- Three TC variants to cover the conversion overhead from every angle: W preconverted to fp16 once at load
+  (`pc`), W converted in-register every call (`in`), and a fused ks=1 form that writes the output directly and skips
+  the reduce kernel (`fused`)
+- The k-split across blocks (`ks`) is swept over all values that divide n_in, and the best is reported - the TC
+  kernel gets its best occupancy
+- Parity against an fp64 host reference; 2000 iterations x 3 reps, warmed CUDA events
+
+## Results (best TC config per shape)
+
+| Shape (decode, n_in = 2560, BF16 W) | Baseline us | Best TC us | Speedup | Config |
+| --- | ---: | ---: | ---: | --- |
+| ssm_alpha/beta [2560, 48] | 8.73 | 11.32 | 0.77x | pc, ks=16 |
+| indexer.k_proj [2560, 128] | 7.14 | 12.09 | 0.59x | pc, ks=16 |
+| indexer.q_proj / router [2560, 512] | 9.48 | 15.78 | 0.60x | pc, ks=4 |
+| ple_value [2560, 2560] | 35.08 | 77.73 | 0.45x | pc, ks=40 |
+
+- Numerics pass: TC max error vs fp64 reference 1.7e-6 .. 7.2e-6 (fp16-accumulation level), baseline
+  0.7e-6 .. 2.3e-6. The TC variant computes the same thing, it is just slower.
+- The warp kernel is already bandwidth-bound at 276-374 GB/s effective W bandwidth (374 GB/s on the 13 MB
+  [2560, 2560] shape); the TC path reaches only 166-169 GB/s (4-byte lane loads, conversion instructions, in-tile
+  launch overhead).
+- A single-token GEMV broadcasts one token across the whole m16 MMA tile, so 15/16 of the MMA throughput is thrown
+  away - but the baseline is not compute-bound, so the wasted throughput buys nothing. The k-split sweep and the
+  fused variant do not recover it.
+- One-time W conversion cost is negligible (2 ms for 13 MB), so preconverting W for the TC path does not help.
+
+## How to rerun
+
+```
+nvcc -arch=sm_75 -O3 -o bench_tc_gemv.exe bench_tc_gemv.cu
+./bench_tc_gemv.exe
+```
+
+Prints the table above, per-shape latency/parity and the conversion exactness scan. On a newer card, adjust
+`-arch`; the packed-A layout comment in the source documents the fragment mapping it relies on.
+
+## Conclusion
+
+**Keep the warp kernel. Tensor cores are not a promising direction for the decode GEMVs on sm_75**, and this was a
+measured loss (1.3-2.2x slower), not a failed attempt to optimize. The Turing MMA stays worthwhile where the input
+really has m16 rows of work: the QSA prompt-attention tiles, which is where #270 uses it. Someone looking to
+re-evaluate on future hardware (a card with native bf16 or denser GEMV batching) can rerun this instead of
+rebuilding it.

bench/results/2026-09-30-tc-gemv-sm75/bench_tc_gemv.cu

@@ -0,0 +1,305 @@
+// bench_tc_gemv.cu - ISOLATED benchmark: BF16 GEMV baseline vs Tensor Core (mma.m16n8k8, BF16->FP16) on sm_75.
+// Nothing in the engine changes. Shapes are the REAL Strata decode GEMVs (BF16 weights, n_in = n_embd = 2560):
+//   ssm_alpha / ssm_beta   [2560, 48]   project_bf16(split=true)  -> bf16_gemv_split(TPR=32) -> warp kernel
+//   indexer.k_proj         [2560, 128]  project_bf16(split=false) -> bf16_gemv -> warp kernel
+//   indexer.q_proj / ffn_gate_inp / mlp.gate [2560, 512]  warp / split(TPR=32) -> warp kernel
+//   ple_value              [2560, 2560] warp
+// Baseline = bf16_gemv_warp_kernel transcribed verbatim from src/kernels/cuda/bf16_gemv.cu (production).
+// TC variant = same maths; A = x broadcast down m16 (single-token GEMV), k-chunk = 8;
+//   (a) W preconverted bf16->fp16 once, x converted in-kernel (timed)   [tc_preconv]
+//   (b) fully inline: W read as bf16 and converted in-register           [tc_inline]
+// Numerics: fp64 host reference; max abs err for baseline and both TC variants; bf16->fp16 exactness scan.
+#include <cuda_runtime.h>
+#include <cuda_fp16.h>
+#include <cstdio>
+#include <cstdint>
+#include <cstring>
+#include <cstdlib>
+#include <cmath>
+#include <vector>
+#include <string>
+#include <functional>
+#include <algorithm>
+
+#define CHECK(e) do { cudaError_t err_ = (e); if (err_ != cudaSuccess) { \
+    fprintf(stderr, "CUDA %s:%d %s\n", __FILE__, __LINE__, cudaGetErrorString(err_)); exit(1); } } while (0)
+
+__device__ __forceinline__ float bf16_f32(uint16_t v) { return __int_as_float((int) ((unsigned) v << 16)); }
+static float bf16_f32_host(uint16_t v) { unsigned u = (unsigned) v << 16; float f; memcpy(&f, &u, 4); return f; }
+static uint16_t f32_bf16_host(float f) {
+    unsigned u, r; memcpy(&u, &f, 4);
+    r = ((u >> 16) ^ (u >> 15)) & 1; u += 0x7fff + r;
+    return (uint16_t) (u >> 16);
+}
+
+// ============ baseline: production warp kernel (verbatim semantics from bf16_gemv.cu) ============
+__global__ void gemv_warp(const uint16_t* __restrict__ x, const uint16_t* __restrict__ w,
+                          float* __restrict__ y, long long n_in, long long n_out) {
+    const long long o = (long long) blockIdx.x * (blockDim.x >> 5) + (threadIdx.x >> 5);
+    if (o >= n_out) return;
+    const int lane = threadIdx.x & 31;
+    const uint16_t* row = w + o * n_in;
+    float acc = 0.0f;
+    for (long long i = lane; i < n_in; i += 32)
+        acc += bf16_f32(x[i]) * bf16_f32(row[i]);
+    for (int off = 16; off > 0; off >>= 1) acc += __shfl_down_sync(0xFFFFFFFFu, acc, off);
+    if (lane == 0) y[o] = acc;
+}
+
+// ============ TC: A(x) broadcast, k-split over blocks, partial D row 0 -> scratch ============
+// A fragment (m16n8k8, .f16): lane l: a0 = A[m=l>>2][k=2(l&3)], a1 = A[m][k=2(l&3)+1],
+//                               a2/a3 = same cols at row m+8.  Broadcast => a2=a0, a3=a1.
+// B fragment (.f16, col-major): lane l: b0 = B[k=2(l&3)][n=(l>>2)], b1 = B[k=2(l&3)+1][n].
+//   B[k][n] = W[n][k], so lane l loads W[n][2(l&3)], W[n][2(l&3)+1] - coalesced within the row.
+// D fragment: c0/c1 = D[m=l>>2][cols 2(l&3), +1], c2/c3 = row m+8.  All m rows identical -> row 0.
+__global__ void tc_preconv(const uint16_t* __restrict__ x_b, const uint16_t* __restrict__ w_f,
+                           float* __restrict__ partials, long long n_in, long long n_out, int ks) {
+    const int tile = blockIdx.x, ks_i = blockIdx.y, lane = threadIdx.x & 31;
+    const long long n_base = (long long) tile * 8;
+    const long long k_per = n_in / (long long) ks;
+    const long long k0 = (long long) ks_i * k_per;
+    const int t2 = 2 * (lane & 3);
+    float c0 = 0.f, c1 = 0.f, c2 = 0.f, c3 = 0.f;
+    for (long long kk0 = k0; kk0 < k0 + k_per; kk0 += 8) {
+        // A (2 x f16x2): a0 = {A[m][2t], A[m+8][2t]}, a1 = {A[m][2t+1], A[m+8][2t+1]} (broadcast -> dup halves)
+        unsigned xh = (unsigned) x_b[kk0 + t2] | ((unsigned) x_b[kk0 + t2 + 1] << 16);
+        // fp16 conversion in-register: bf16 -> f32 -> f16 (exact within fp16 range)
+        __half2 ha = __floats2half2_rn(bf16_f32((uint16_t) (xh & 0xFFFF)), bf16_f32((uint16_t) (xh >> 16)));
+        __half2 hb = __floats2half2_rn(bf16_f32((uint16_t) (xh & 0xFFFF)), bf16_f32((uint16_t) (xh >> 16)));
+        unsigned a0 = *reinterpret_cast<unsigned*>(&ha);
+        unsigned a1 = *reinterpret_cast<unsigned*>(&hb);
+        // B is already fp16 here: load as __half2 directly (bf16 path must NOT convert twice)
+        __half2 wbh = *reinterpret_cast<const __half2*>(&w_f[(n_base + (lane >> 2)) * n_in + kk0 + t2]);
+        unsigned b0 = *reinterpret_cast<unsigned*>(&wbh);
+        asm volatile("mma.sync.aligned.m16n8k8.row.col.f32.f16.f16.f32 {%0,%1,%2,%3}, {%4,%5}, {%6}, {%0,%1,%2,%3};\n"
+                     : "+f"(c0), "+f"(c1), "+f"(c2), "+f"(c3)
+                     : "r"(a0), "r"(a1), "r"(b0));
+    }
+    if (lane < 4) { float* p = partials + (size_t) (tile * ks + ks_i) * 8; p[t2] = c0; p[t2 + 1] = c1; }
+}
+
+// same, but W stays bf16 and converts in-register (fully fair per-call cost)
+__global__ void tc_inline(const uint16_t* __restrict__ x_b, const uint16_t* __restrict__ w_b,
+                          float* __restrict__ partials, long long n_in, long long n_out, int ks) {
+    const int tile = blockIdx.x, ks_i = blockIdx.y, lane = threadIdx.x & 31;
+    const long long n_base = (long long) tile * 8;
+    const long long k_per = n_in / (long long) ks;
+    const long long k0 = (long long) ks_i * k_per;
+    const int t2 = 2 * (lane & 3);
+    float c0 = 0.f, c1 = 0.f, c2 = 0.f, c3 = 0.f;
+    for (long long kk0 = k0; kk0 < k0 + k_per; kk0 += 8) {
+        unsigned xh = (unsigned) x_b[kk0 + t2] | ((unsigned) x_b[kk0 + t2 + 1] << 16);
+        __half2 ha = __floats2half2_rn(bf16_f32((uint16_t) (xh & 0xFFFF)), bf16_f32((uint16_t) (xh >> 16)));
+        __half2 hb = __floats2half2_rn(bf16_f32((uint16_t) (xh & 0xFFFF)), bf16_f32((uint16_t) (xh >> 16)));
+        unsigned a0 = *reinterpret_cast<unsigned*>(&ha);
+        unsigned a1 = *reinterpret_cast<unsigned*>(&hb);
+        const uint16_t* wbv = w_b + (n_base + (lane >> 2)) * n_in + kk0 + t2;
+        __half2 wbh = __floats2half2_rn(bf16_f32(wbv[0]), bf16_f32(wbv[1]));
+        unsigned b0 = *reinterpret_cast<unsigned*>(&wbh);
+        asm volatile("mma.sync.aligned.m16n8k8.row.col.f32.f16.f16.f32 {%0,%1,%2,%3}, {%4,%5}, {%6}, {%0,%1,%2,%3};\n"
+                     : "+f"(c0), "+f"(c1), "+f"(c2), "+f"(c3)
+                     : "r"(a0), "r"(a1), "r"(b0));
+    }
+    if (lane < 4) { float* p = partials + (size_t) (tile * ks + ks_i) * 8; p[t2] = c0; p[t2 + 1] = c1; }
+}
+
+__global__ void tc_reduce(const float* __restrict__ partials, float* __restrict__ y, long long n_out, int ks) {
+    const long long o = blockIdx.x * blockDim.x + threadIdx.x;
+    if (o >= n_out) return;
+    float s = 0.f;
+    for (int q = 0; q < ks; ++q) s += partials[(size_t) ((o >> 3) * ks + q) * 8 + (o & 7)];
+    y[o] = s;
+}
+
+// variant with the OTHER candidate A layout: a0 = {A[gid][2t], A[gid+8][2t]} (same k-col, two rows),
+// a1 = same for col 2t+1. Broadcast => both halves of a0 hold x[2t], both halves of a1 hold x[2t+1].
+__global__ void tc_dupcol(const uint16_t* __restrict__ x_b, const uint16_t* __restrict__ w_b,
+                          float* __restrict__ partials, long long n_in, long long n_out, int ks) {
+    const int tile = blockIdx.x, ks_i = blockIdx.y, lane = threadIdx.x & 31;
+    const long long n_base = (long long) tile * 8;
+    const long long k_per = n_in / (long long) ks;
+    const long long k0 = (long long) ks_i * k_per;
+    const int t2 = 2 * (lane & 3);
+    float c0 = 0.f, c1 = 0.f, c2 = 0.f, c3 = 0.f;
+    for (long long kk0 = k0; kk0 < k0 + k_per; kk0 += 8) {
+        __half2 ha = __floats2half2_rn(bf16_f32(x_b[kk0 + t2]), bf16_f32(x_b[kk0 + t2]));
+        __half2 hb = __floats2half2_rn(bf16_f32(x_b[kk0 + t2 + 1]), bf16_f32(x_b[kk0 + t2 + 1]));
+        unsigned a0 = *reinterpret_cast<unsigned*>(&ha);
+        unsigned a1 = *reinterpret_cast<unsigned*>(&hb);
+        const uint16_t* wbv = w_b + (n_base + (lane >> 2)) * n_in + kk0 + t2;
+        __half2 wbh = __floats2half2_rn(bf16_f32(wbv[0]), bf16_f32(wbv[1]));
+        unsigned b0 = *reinterpret_cast<unsigned*>(&wbh);
+        asm volatile("mma.sync.aligned.m16n8k8.row.col.f32.f16.f16.f32 {%0,%1,%2,%3}, {%4,%5}, {%6}, {%0,%1,%2,%3};\n"
+                     : "+f"(c0), "+f"(c1), "+f"(c2), "+f"(c3)
+                     : "r"(a0), "r"(a1), "r"(b0));
+    }
+    if (lane < 4) { float* p = partials + (size_t) (tile * ks + ks_i) * 8; p[t2] = c0; p[t2 + 1] = c1; }
+}
+
+// FUSED, ks=1 only: with a single k-fragment the partial IS the answer -> write y directly, no reduce kernel,
+// no scratch. Fully inline bf16 W conversion (fairest per-call TC cost).
+__global__ void tc_fused(const uint16_t* __restrict__ x_b, const uint16_t* __restrict__ w_b,
+                         float* __restrict__ y, long long n_in, long long n_out) {
+    const int tile = blockIdx.x, lane = threadIdx.x & 31;
+    const long long n_base = (long long) tile * 8;
+    const int t2 = 2 * (lane & 3);
+    float c0 = 0.f, c1 = 0.f, c2 = 0.f, c3 = 0.f;
+    for (long long kk0 = 0; kk0 < n_in; kk0 += 8) {
+        __half2 ha = __floats2half2_rn(bf16_f32(x_b[kk0 + t2]), bf16_f32(x_b[kk0 + t2 + 1]));
+        __half2 hb = ha;   // broadcast: row gid+8 = row gid (single-token GEMV)
+        unsigned a0 = *reinterpret_cast<unsigned*>(&ha);
+        unsigned a1 = *reinterpret_cast<unsigned*>(&hb);
+        const uint16_t* wbv = w_b + (n_base + (lane >> 2)) * n_in + kk0 + t2;
+        __half2 wbh = __floats2half2_rn(bf16_f32(wbv[0]), bf16_f32(wbv[1]));
+        unsigned b0 = *reinterpret_cast<unsigned*>(&wbh);
+        asm volatile("mma.sync.aligned.m16n8k8.row.col.f32.f16.f16.f32 {%0,%1,%2,%3}, {%4,%5}, {%6}, {%0,%1,%2,%3};\n"
+                     : "+f"(c0), "+f"(c1), "+f"(c2), "+f"(c3)
+                     : "r"(a0), "r"(a1), "r"(b0));
+    }
+    if (lane < 4) { y[n_base + t2] = c0; y[n_base + t2 + 1] = c1; }
+}
+
+int main() {
+    CHECK(cudaSetDevice(0));
+    cudaDeviceProp prop; CHECK(cudaGetDeviceProperties(&prop, 0));
+    printf("device: %s, SMs %d, capability %d.%d\n", prop.name, prop.multiProcessorCount, prop.major, prop.minor);
+
+    struct Shape { const char* name; long long n_in, n_out; };
+    Shape shapes[] = {
+        {"ssm_alpha/beta [2560,48]", 2560, 48},
+        {"indexer.k_proj [2560,128]", 2560, 128},
+        {"q_proj/router [2560,512]", 2560, 512},
+        {"ple_value [2560,2560]", 2560, 2560},
+    };
+    auto rng = [](unsigned& s) { s = s * 1664525u + 1013904223u; return (float) ((s >> 8) & 0xFFFF) / 65535.0f - 0.5f; };
+    const int ITERS = 2000, WARM = 300;
+
+    printf("\n%-26s %13s | %s\n", "shape", "baseline us", "tc variants (pc = W preconverted, in = inline conversion, fused = no reduce), speedup vs baseline");
+    printf("%s\n", std::string(150, '-').c_str());
+
+    for (auto& sh : shapes) {
+        const long long n_in = sh.n_in, n_out = sh.n_out;
+        const size_t wn = (size_t) n_out * n_in;
+        std::vector<uint16_t> hw(wn), hx(n_in), hf(wn);
+        std::vector<float> hyb(n_out), hypc(n_out), hyin(n_out), hyfu(n_out);
+        std::vector<double> href(n_out);
+        unsigned seed = 991u + (unsigned) (n_out & 0xFFF);
+        for (size_t i = 0; i < wn; ++i) hw[i] = f32_bf16_host(rng(seed) * 0.3f);
+        for (long long i = 0; i < n_in; ++i) hx[i] = f32_bf16_host(rng(seed) * 3.0f);
+        for (long long o = 0; o < n_out; ++o) {
+            double s = 0.0;
+            for (long long i = 0; i < n_in; ++i) s += (double) bf16_f32_host(hx[i]) * (double) bf16_f32_host(hw[(size_t) o * n_in + i]);
+            href[o] = s;
+        }
+        // preconvert W to fp16 (bf16 -> fp16 is exact within fp16 range)
+        for (size_t i = 0; i < wn; ++i) { __half h = __float2half_rn(bf16_f32_host(hw[i])); memcpy(&hf[i], &h, 2); }
+
+        uint16_t *dx, *dw, *dwf; float *dy, *dyp;
+        CHECK(cudaMalloc(&dx, n_in * 2));
+        CHECK(cudaMalloc(&dw, wn * 2)); CHECK(cudaMalloc(&dwf, wn * 2));
+        CHECK(cudaMalloc(&dy, n_out * 4));
+        CHECK(cudaMalloc(&dyp, (size_t)(n_out / 8) * 65 * 8 * 4));
+        CHECK(cudaMemcpy(dx, hx.data(), n_in * 2, cudaMemcpyHostToDevice));
+        CHECK(cudaMemcpy(dw, hw.data(), wn * 2, cudaMemcpyHostToDevice));
+        CHECK(cudaMemcpy(dwf, hf.data(), wn * 2, cudaMemcpyHostToDevice));
+
+        // one-time W conversion cost (host->device, reported separately; the engine would fold this
+        // into the weight load exactly like Gemm::bf16 does for sm<80)
+        {
+            cudaEvent_t e0, e1; CHECK(cudaEventCreate(&e0)); CHECK(cudaEventCreate(&e1));
+            CHECK(cudaEventRecord(e0));
+            CHECK(cudaMemcpy(dwf, hf.data(), wn * 2, cudaMemcpyHostToDevice));
+            CHECK(cudaEventRecord(e1)); CHECK(cudaEventSynchronize(e1));
+            float ms = 0; CHECK(cudaEventElapsedTime(&ms, e0, e1));
+            printf("%-26s one-time W convert: %.2f ms (%.1f MB), tiles=%lld\n", sh.name, ms, wn * 2 / 1e6, n_out / 8);
+        }
+
+        int tiles = (int) (n_out / 8);
+        // valid ks must divide n_in so each k-chunk is a multiple of 8; sweep ALL of them, report the best
+        const int cands[11] = {1, 2, 4, 5, 8, 10, 16, 20, 32, 40, 64};
+
+        auto timeit = [&](const char* tag, const std::function<void()>& fn) -> double {
+            for (int i = 0; i < WARM; ++i) fn();
+            CHECK(cudaDeviceSynchronize());
+            double best = 1e30;
+            for (int rep = 0; rep < 3; ++rep) {
+                cudaEvent_t e0, e1; CHECK(cudaEventCreate(&e0)); CHECK(cudaEventCreate(&e1));
+                CHECK(cudaEventRecord(e0));
+                for (int i = 0; i < ITERS; ++i) fn();
+                CHECK(cudaEventRecord(e1)); CHECK(cudaEventSynchronize(e1));
+                float ms = 0; CHECK(cudaEventElapsedTime(&ms, e0, e1));
+                double us = ms * 1000.0 / ITERS;
+                if (us < best) best = us;
+            }
+            (void) tag; return best;
+        };
+
+        double t_base = timeit("base", [&]{ gemv_warp<<<(unsigned) ((n_out + 7) / 8), 256>>>(dx, dw, dy, n_in, n_out); });
+        gemv_warp<<<(unsigned) ((n_out + 7) / 8), 256>>>(dx, dw, dy, n_in, n_out);
+        CHECK(cudaMemcpy(hyb.data(), dy, n_out * 4, cudaMemcpyDeviceToHost));
+
+        // occupancy sweep over k-chunk count for the preconverted-W TC kernel
+        int best_ks = 1; double t_pc = 1e30;
+        for (int i = 0; i < 11; ++i) {
+            const int ksv = cands[i];
+            if (n_in % ksv != 0) continue;
+            dim3 g((unsigned) tiles, (unsigned) ksv);
+            double t = timeit("pcs", [&]{ tc_preconv<<<g, 32>>>(dx, dwf, dyp, n_in, n_out, ksv);
+                                          tc_reduce<<<(unsigned) ((n_out + 255) / 256), 256>>>(dyp, dy, n_out, ksv); });
+            if (t < t_pc) { t_pc = t; best_ks = ksv; }
+        }
+        dim3 gtc((unsigned) tiles, (unsigned) best_ks);
+        tc_preconv<<<gtc, 32>>>(dx, dwf, dyp, n_in, n_out, best_ks);
+        tc_reduce<<<(unsigned) ((n_out + 255) / 256), 256>>>(dyp, dy, n_out, best_ks);
+        CHECK(cudaMemcpy(hypc.data(), dy, n_out * 4, cudaMemcpyDeviceToHost));
+
+        double t_ti = timeit("tci", [&]{ tc_inline<<<gtc, 32>>>(dx, dw, dyp, n_in, n_out, best_ks);
+                                         tc_reduce<<<(unsigned) ((n_out + 255) / 256), 256>>>(dyp, dy, n_out, best_ks); });
+        tc_inline<<<gtc, 32>>>(dx, dw, dyp, n_in, n_out, best_ks);
+        tc_reduce<<<(unsigned) ((n_out + 255) / 256), 256>>>(dyp, dy, n_out, best_ks);
+        CHECK(cudaMemcpy(hyin.data(), dy, n_out * 4, cudaMemcpyDeviceToHost));
+
+        double t_fu = timeit("tcf", [&]{ tc_fused<<<(unsigned) tiles, 32>>>(dx, dw, dy, n_in, n_out); });
+        tc_fused<<<(unsigned) tiles, 32>>>(dx, dw, dy, n_in, n_out);
+        CHECK(cudaMemcpy(hyfu.data(), dy, n_out * 4, cudaMemcpyDeviceToHost));
+
+        double best_tc = t_pc; const char* best_var = "preconv";
+        if (t_ti < best_tc) { best_tc = t_ti; best_var = "inline"; }
+        if (t_fu < best_tc) { best_tc = t_fu; best_var = "fused"; }
+
+        double e_tc = 0, e_base = 0, sc = 0;
+        if (strcmp(best_var, "preconv") == 0) {
+            for (long long o = 0; o < n_out; ++o) { e_tc = std::max(e_tc, std::fabs((double) hypc[o] - href[o])); }
+        } else if (strcmp(best_var, "inline") == 0) {
+            for (long long o = 0; o < n_out; ++o) { e_tc = std::max(e_tc, std::fabs((double) hyin[o] - href[o])); }
+        } else {
+            for (long long o = 0; o < n_out; ++o) { e_tc = std::max(e_tc, std::fabs((double) hyfu[o] - href[o])); }
+        }
+        for (long long o = 0; o < n_out; ++o) {
+            e_base = std::max(e_base, std::fabs((double) hyb[o] - href[o]));
+            sc = std::max(sc, std::fabs(href[o]));
+        }
+        double gb_base = (double) (wn * 2) / t_base / 1e3;
+        double gb_tc = (double) (wn * 2) / best_tc / 1e3;
+        printf("%-26s %13.3f | pc %13.3f (ks=%d) | in %13.3f | fused %13.3f | BEST %s %7.3fx | maxerr tc %.2e base %.2e  (scale %.1f, BW base %.0f tc %.0f GB/s)\n",
+               sh.name, t_base, t_pc, best_ks, t_ti, t_fu, best_var, t_base / best_tc,
+               e_tc, e_base, sc, gb_base, gb_tc);
+
+        CHECK(cudaFree(dx)); CHECK(cudaFree(dw)); CHECK(cudaFree(dwf)); CHECK(cudaFree(dy)); CHECK(cudaFree(dyp));
+    }
+
+    {   // bf16 -> fp16 exactness scan
+        unsigned seed = 777u; double worst = 0; int bad = 0, n = 2000000;
+        for (int i = 0; i < n; ++i) {
+            float f = (rng(seed) * 2.0f - 1.0f) * 100.0f;
+            uint16_t b = f32_bf16_host(f);
+            __half h = __float2half_rn(bf16_f32_host(b));
+            if ((double) __half2float(h) != (double) bf16_f32_host(b)) {
+                bad++; worst = std::max(worst, std::fabs((double) __half2float(h) - (double) bf16_f32_host(b)));
+            }
+        }
+        printf("\nbf16->fp16 conversion exactness: %d / %d inexact, worst %.3g\n", bad, n, worst);
+    }
+    printf("BENCH_DONE\n");
+    return 0;
+}

cmake/hip_backend.cmake

@@ -1,23 +1,34 @@
-# Opt-in HIP configuration. Strata's CUDA-shaped kernels target wave32 RDNA3 / RDNA4 (64 KiB LDS per workgroup,
-# the signed dot4 instruction). CMake/compiler discovery stays machine-independent; pass CMAKE_HIP_COMPILER when it
+# Opt-in HIP configuration. Strata's CUDA-shaped kernels target wave32 RDNA2 / RDNA3 / RDNA4 (64 KiB LDS per
+# workgroup, a signed dot4 instruction). CMake/compiler discovery stays machine-independent; pass CMAKE_HIP_COMPILER when it
 # is not on PATH.
 if(NOT DEFINED CMAKE_HIP_ARCHITECTURES OR CMAKE_HIP_ARCHITECTURES STREQUAL "")
   set(CMAKE_HIP_ARCHITECTURES gfx1100 CACHE STRING "Strata HIP target architecture(s), e.g. gfx1100 or gfx1100;gfx1201")
 endif()
-# Validated on real cards: gfx1100 (RX 7900 XT / XTX) and gfx1201 (RX 9070 / 9070 XT, Radeon AI PRO R9700).
-# The other RDNA3 / RDNA4 wave32 chips have the same LDS limit and dot4 instruction and build the same code, but
-# the maintainers have not run them (community reports: gfx1102 #192, gfx1200 #176).
+# Validated on real cards: gfx1100 (RX 7900 XT / XTX) and gfx1201 (RX 9070 / 9070 XT, Radeon AI PRO R9700) by the
+# maintainers; gfx1101 (RX 7800 XT, #254) and gfx1200 (RX 9060 XT, #256) by their owners. gfx1102 (RX 7600) has the
+# same LDS limit and dot4 instruction and passed ctest (#192), but no model run has been reported yet. RDNA2 gfx1030 (RX 6800 / 6900) has the
+# same LDS limit and wave32 but an older dot4 instruction (v_dot4_i32_i8, hip_compat/intrinsics.hpp); a community
+# report ran it (#311), the maintainers have not.
 set(_strata_hip_validated gfx1100 gfx1201)
-set(_strata_hip_unvalidated gfx1101 gfx1102 gfx1200)
+set(_strata_hip_community gfx1101 gfx1200)
+set(_strata_hip_unvalidated gfx1102 gfx1030)
+# CMake hands HIP a ';' list, but a -DCMAKE_HIP_ARCHITECTURES typed by hand (or ROCm's own Windows tooling) may use
+# spaces, which foreach(IN LISTS) would otherwise treat as one element.
+string(REPLACE " " ";" _strata_hip_norm "${CMAKE_HIP_ARCHITECTURES}")
 set(STRATA_HIP_ARCH_LIST "")
-foreach(_arch IN LISTS CMAKE_HIP_ARCHITECTURES)
+foreach(_arch IN LISTS _strata_hip_norm)
+  if(_arch STREQUAL "")
+    continue()
+  endif()
   string(REGEX REPLACE ":.*$" "" _base "${_arch}")      # gfx1100:xnack- -> gfx1100
   if(_base IN_LIST _strata_hip_validated)
+  elseif(_base IN_LIST _strata_hip_community)
+    message(STATUS "Strata HIP: ${_base} was validated by community reports (docs/AMD_HIP.md)")
   elseif(_base IN_LIST _strata_hip_unvalidated)
     message(WARNING "Strata HIP: ${_base} builds, but it is not validated on a real card yet; please report results")
   else()
     message(FATAL_ERROR
-      "Strata HIP supports wave32 gfx1100 and gfx1201 (unvalidated: ${_strata_hip_unvalidated}); "
+      "Strata HIP supports wave32 gfx1100, gfx1101, gfx1200 and gfx1201 (unvalidated: ${_strata_hip_unvalidated}); "
       "CMAKE_HIP_ARCHITECTURES is '${CMAKE_HIP_ARCHITECTURES}'")
   endif()
   list(APPEND STRATA_HIP_ARCH_LIST "${_base}")
@@ -57,11 +68,19 @@ target_include_directories(strata_hip_runtime BEFORE INTERFACE
   "${STRATA_HIP_COMPAT_INCLUDE_DIR}" "${CMAKE_CURRENT_SOURCE_DIR}/include")
 target_compile_definitions(strata_hip_runtime INTERFACE STRATA_USE_HIP=1 "STRATA_HIP_ARCHS=\"${STRATA_HIP_ARCHS}\"")
 target_link_libraries(strata_hip_runtime INTERFACE hip::host)
-foreach(_language IN ITEMS CXX HIP)
+# The shim renames the CUDA runtime to HIP, force-included into every host and device source. On Windows the host
+# compiler is ROCm's clang++ too (tools/hip/build_windows.bat: CMake refuses to mix cl.exe with Clang HIP), which takes
+# -include like it does on Linux; an MSVC-style front end (cl / clang-cl) takes /FI instead. Forward slashes, so /FI
+# does not read the path's backslashes as escapes.
+file(TO_CMAKE_PATH "${STRATA_HIP_COMPAT_INCLUDE_DIR}/cuda_runtime.h" _strata_hip_force)
+if(MSVC)
+  target_compile_options(strata_hip_runtime INTERFACE "$<$<COMPILE_LANGUAGE:CXX>:/FI${_strata_hip_force}>")
+else()
   target_compile_options(strata_hip_runtime INTERFACE
-    "$<$<COMPILE_LANGUAGE:${_language}>:-include>"
-    "$<$<COMPILE_LANGUAGE:${_language}>:${STRATA_HIP_COMPAT_INCLUDE_DIR}/cuda_runtime.h>")
-endforeach()
+    "$<$<COMPILE_LANGUAGE:CXX>:-include>" "$<$<COMPILE_LANGUAGE:CXX>:${_strata_hip_force}>")
+endif()
+target_compile_options(strata_hip_runtime INTERFACE
+  "$<$<COMPILE_LANGUAGE:HIP>:-include>" "$<$<COMPILE_LANGUAGE:HIP>:${_strata_hip_force}>")
 
 # CMake does not infer HIP from Strata's existing CUDA-shaped .cu suffixes.
 file(GLOB_RECURSE _strata_hip_sources CONFIGURE_DEPENDS

docs/AI_SETUP.md

@@ -0,0 +1,196 @@
+# Installing Strata for a user (instructions for AI agents)
+
+This page is written for an AI coding assistant (Claude Code, Cursor, Codex, GitHub Copilot, ...) that a user asked to
+set up Strata on their PC. Strata runs the Qwen3.8-Flash-Next model locally on one NVIDIA or AMD graphics card plus
+system RAM, and serves an OpenAI- and Anthropic-compatible API on `http://127.0.0.1:8080`. The human-oriented
+version of everything below is [INSTALL.md](INSTALL.md).
+
+Work through the steps in order. Tell the user what you are doing in plain words; they may not be technical.
+
+## 0. Ground rules
+
+- **Never expose the server without an API key.** Keep the default `--host 127.0.0.1`. Only if the user asks for
+  access from other devices, use `--host 0.0.0.0` **together with** `--api-key <a long random secret>`, and give the
+  user the key. Never put a tunnel or port forward in front of a server without a key.
+- Do not change the user's system beyond what setup does (setup installs Python for the user account if needed,
+  and everything else inside the Strata folder and its `Strata-data` folder). Ask before installing drivers.
+- The model download is ~70 GB (up to 111 GB for Unsloth's 4-bit). Confirm the user is fine with that before you
+  start, especially on a metered connection.
+- Setup is long-running (an hour or more on a slow connection). Run it in the background or with a long timeout and
+  poll its output; do not kill it because it is quiet for a while. It is resumable: running the same command again
+  continues where it stopped.
+
+## 1. Check the PC
+
+The quickest way is setup's own check, which prints the GPU(s), driver, RAM, CPU, free disk and whether this PC can run
+Strata, and installs nothing except Python and the `.venv` it needs to run (step 2 gets the repository first):
+
+```
+Windows:  START-HERE.bat --check
+Linux:    ./setup.sh --check
+```
+
+To check by hand:
+
+| What | Windows (PowerShell) | Linux |
+| --- | --- | --- |
+| OS | `[Environment]::OSVersion` | `cat /etc/os-release` |
+| NVIDIA GPU, VRAM, driver | `nvidia-smi --query-gpu=index,name,memory.total,driver_version --format=csv` | the same |
+| AMD GPU | `Get-CimInstance Win32_VideoController \| Select-Object Name, AdapterRAM` (AdapterRAM is capped at 4 GB; trust the model name) | `lspci \| grep -i -E 'vga\|display'`; VRAM: `cat /sys/class/drm/card*/device/mem_info_vram_total` (bytes); `rocm-smi` only if ROCm is installed |
+| RAM | `(Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory / 1GB` | `free -g` |
+| Free disk | `Get-PSDrive -PSProvider FileSystem` | `df -h .` |
+
+Requirements (details: [INSTALL.md](INSTALL.md#what-you-need)):
+
+- **GPU:** NVIDIA RTX 20, 30, 40 or 50 series, or AMD Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT,
+  RX 9070 / 9070 XT, Radeon AI PRO R9700, RX 6800 / 6900 series; 12 GB of VRAM or more (an NVIDIA card with 8 GB runs,
+  slowly). GTX 10 series and older, and integrated GPUs, are not supported.
+- **Driver:** NVIDIA 580 or newer. AMD on Linux: the kernel's amdgpu driver; on Windows: a current AMD Adrenalin
+  driver. If the driver is missing or too old, tell the user to update it (NVIDIA App / nvidia.com/drivers, or AMD
+  Software) and restart; do not install drivers yourself unless they ask.
+- **RAM:** 32 GB or more (see step 3). **Disk:** ~80 GB free, ideally on an NVMe SSD. **CPU:** x86-64 with AVX2.
+- **OS:** Windows 10/11 or Linux (Ubuntu 22.04/24.04 are fully automatic).
+
+If the PC does not meet them, say which part is missing and stop.
+
+## 2. Get Strata
+
+```
+git clone https://github.com/Niko1221/Strata.git
+```
+
+Without git: download https://github.com/Niko1221/Strata/archive/refs/heads/main.zip and unzip it. Pick a drive with
+enough free space: the model files go to a `Strata-data` folder **next to** the Strata folder (or `--data-dir <path>`
+on another drive).
+
+## 3. Pick the model
+
+By the PC's RAM (ask the user whether they mainly want it for code - then the Coder is a good pick at any RAM size):
+
+| RAM | `--family` / `--model` | Notes |
+| --- | --- | --- |
+| 32 GB | `--family coder` (size IQ1_M) | for code; with a 24 GB card Q2_0 and IQ2_XS also run (setup picks the low-RAM mode) |
+| 48 GB | `--family qwen --model IQ2_XS` | or `Q2_0` (fastest) |
+| 64 GB | `--family qwen --model IQ2_XS` (recommended) | `IQ3_XXS` / `IQ3_S` are slower and a bit better |
+| 96 GB+ | `--family qwen --model IQ3_S` | `--family unsloth --model UD-Q4_K_XL` is experimental: NVIDIA only, NVMe SSD, 7-8.5 tokens/s on 64 GB |
+
+`--family swift` (Swift 1.5, a fine-tune that thinks shorter; sizes Q2_0, IQ2_XS, IQ3_XXS) is the alternative to
+`qwen`. With `--yes` and no `--model`, setup picks the recommended size for the RAM itself. More: [MODELS.md](MODELS.md).
+
+## 4. Run setup without questions
+
+From the Strata folder (Windows: in `cmd`, or `cmd /c START-HERE.bat ...` from PowerShell):
+
+```
+Windows:  START-HERE.bat --yes --family qwen --model IQ2_XS --no-start
+Linux:    ./setup.sh --yes --family qwen --model IQ2_XS --no-start
+```
+
+The flags (all of them: `START-HERE.bat --help`):
+
+| Flag | Meaning |
+| --- | --- |
+| `--yes` | take the recommended answer to every question (no prompts) |
+| `--family qwen\|swift\|coder\|unsloth` | the model version |
+| `--model Q2_0\|IQ2_XS\|IQ3_XXS\|IQ3_S\|IQ1_M\|UD-Q4_K_XL` | the size (the Coder is IQ1_M, Unsloth UD-Q4_K_XL) |
+| `--context N` | context in tokens; default by VRAM: 32768 under 14 GB, 65536 under 20 GB, else 131072 |
+| `--vision yes\|no\|gpu\|cpu` | read pictures; `--yes` leaves images off. AMD cards: `cpu` |
+| `--gpu N` / `--gpus 0,1` / `--gpus all` | one card, or several sharing the model (default: the card with the most VRAM) |
+| `--backend cuda\|hip` | NVIDIA or AMD engine; chosen by itself on a PC with only one kind of card |
+| `--data-dir PATH` | where the 70-120 GB of model files go |
+| `--port N` | the server port (default 8080) |
+| `--host H --api-key K` | listen beyond this PC; **only together with a key** |
+| `--no-start` | install only, do not start the server |
+| `--setup` | install another model or change settings of an installed one |
+| `--check` | only check the PC |
+
+`--no-start` is recommended for agents: the server runs in the foreground until its window is closed, which would
+block your shell. Start it separately in step 6.
+
+Notes:
+- **Linux:** setup uses `sudo apt` (or dnf/pacman) to install Python with venv if it is missing, and on AMD may need
+  `build-essential` and `git`. You cannot type the user's password: if a `sudo` step is needed, ask the user to run it
+  (e.g. `sudo apt install python3-venv build-essential git`) and then rerun setup.
+- **AMD on Linux:** setup installs ROCm into `.venv` (~10 GB, no sudo) unless a system ROCm 7 exists, and compiles the
+  engine for the card (10-20 minutes, once).
+- **AMD on Windows (new in 0.1.34):** setup downloads the ready-made AMD engine (~550 MB, ROCm included) - nothing
+  is compiled and only the AMD driver is needed. Before the model download it runs `engine\strata-device.exe
+  --list-devices`; if that does not list the card, the driver is the problem (tell the user to update AMD Software).
+  One card per model and no images on Windows for now. It is new: ask the user to report how it runs (docs/AMD_HIP.md).
+- **NVIDIA with no ready-made engine for the card:** setup offers to install build tools and compile (20-40 minutes);
+  `--yes` accepts.
+
+## 5. While it downloads, tell the user
+
+- It downloads about 70 GB from Hugging Face; how long depends on their connection (at 100 Mbit/s roughly 1.5-2
+  hours). It can be stopped and continues where it left off.
+- Then it prepares the model for their PC (a few minutes) and, on AMD on Linux, compiles the engine.
+- **When the model starts, the PC can be slow or stop responding for 1-3 minutes** (longest the first time): Strata
+  loads 35-55 GB into RAM and locks part of it for the graphics card. That is normal; they should not close the window.
+- Nothing leaves their PC: the model runs locally.
+
+## 6. Start the server
+
+Setup prints the start script it wrote (`start script: run-<model>.bat`). The name is the family tag plus the size,
+lower case: `run-iq2_xs`, `run-swift-iq2_xs`, `run-coder-iq1_m`, `run-unsloth-ud-q4_k_xl`.
+
+```
+Windows (PowerShell):  Start-Process -FilePath ".\run-iq2_xs.bat"            (opens its own window)
+Linux:                 nohup ./run-iq2_xs.sh > strata-server.out 2>&1 &
+```
+
+Or `START-HERE.bat` / `./setup.sh` without flags, which starts the installed model (it asks which one when several are
+installed; add `--yes` to take the first). The server opens the user's browser on `http://127.0.0.1:8080` when it is
+ready. Closing its window (or stopping the process) stops the model.
+
+## 7. Verify
+
+The HTTP server answers once the model is loaded (30-90 s on later starts, a few minutes the first time). Poll:
+
+```
+curl http://127.0.0.1:8080/health
+```
+
+It returns `{"status": "ok", "model": ..., "max_context": ..., "images": ..., "api_key": ..., "loaded": true, ...}`.
+Then:
+
+```
+curl http://127.0.0.1:8080/v1/models
+curl http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d "{\"model\": \"strata\", \"messages\": [{\"role\": \"user\", \"content\": \"Say hello in five words.\"}], \"max_tokens\": 64, \"reasoning_effort\": \"none\"}"
+```
+
+With an API key, add `-H "Authorization: Bearer <key>"`. The server window prints a `ready: http://127.0.0.1:8080/v1`
+line when it is ready; the engine log is `strata-<model>.log` in the Strata folder.
+
+## 8. Connect the user's apps
+
+- **Browser:** `http://127.0.0.1:8080` - Chat, a live Monitor, and About (settings and addresses).
+- **Any OpenAI-compatible app or agent:** base URL `http://127.0.0.1:8080/v1`, any API key (or the configured one),
+  any model name.
+- **Anthropic-compatible apps:** `http://127.0.0.1:8080/v1/messages`.
+- **Claude Code:** `ANTHROPIC_BASE_URL=http://127.0.0.1:8080`, `ANTHROPIC_MODEL` set to a Claude model name it knows
+  (Strata ignores the name), and any `ANTHROPIC_AUTH_TOKEN` (or the configured key).
+- **Thinking level:** `"reasoning_effort": "none" | "low" | "medium" | "high"` (default high).
+- Strata answers one request at a time. API details: [DETAILS.md](DETAILS.md#using-it).
+
+## 9. When something fails
+
+| Symptom | What to do |
+| --- | --- |
+| `the NVIDIA driver is too old` | The user updates the driver (NVIDIA App or nvidia.com/drivers) and restarts; rerun setup. |
+| Download or install stopped | Rerun the same setup command; it continues. |
+| `port 8080 is already in use` | Strata is already running (check `/health`), or use `--port 8081`. |
+| Very slow, disk busy, or "the engine stopped unexpectedly" | Not enough free RAM: close programs, or set up a smaller size (`--setup --model Q2_0` or IQ2_XS). |
+| `ExpertCache: cudaMalloc(...) failed: out of memory` with free VRAM (Windows) | The page file is off or tiny: set it to "System managed" and restart. |
+| `prompt ... exceeds the context` | Rerun setup with `--setup --context <bigger>`. |
+| No AMD GPU found (Linux) | The amdgpu driver lists no GPU; check the card and driver. Integrated GPUs are not supported. |
+| Python or build tools could not be installed | Install what the message names (links are printed), then rerun. |
+
+More: [TROUBLESHOOTING.md](TROUBLESHOOTING.md) and the [full table](DETAILS.md#troubleshooting). If it still fails,
+collect `strata-<model>.log` and the setup output, and suggest an issue at https://github.com/Niko1221/Strata/issues.
+
+## Alternative: the MCP server
+
+Strata also has an MCP server, so an AI tool can check, install, start and stop Strata through tool calls
+(`strata_status`, `strata_models`, `strata_install`, `strata_start`, `strata_stop`, `strata_logs`) instead of the
+shell commands above. See [MCP_SERVER.md](MCP_SERVER.md).

docs/AMD_HIP.md

@@ -1,12 +1,15 @@
-# Experimental AMD HIP backend (gfx1100, gfx1201)
+# AMD Radeon: the HIP backend (gfx1100, gfx1101, gfx1200, gfx1201, gfx1030)
 
-This is a Linux source build for the RX 7900 XT / XTX (RDNA3, gfx1100) and the
-RX 9070 / 9070 XT / Radeon AI PRO R9700 (RDNA4, gfx1201; see [RDNA4](#rdna4-gfx1201)).
-It is opt-in; the NVIDIA installer and CUDA build remain the default. Other AMD
-architectures, wave64, Windows HIP, and mixed AMD/NVIDIA execution are outside this contribution.
+Strata runs on AMD Radeon cards through its HIP backend, the same engine as on NVIDIA compiled for AMD. This page
+covers the build on Linux (on Windows a ready-made engine, see [Windows](#windows)) for the RX 7900 XT / XTX (RDNA3, gfx1100) and the
+RX 9070 / 9070 XT / Radeon AI PRO R9700 (RDNA4, gfx1201; see [RDNA4](#rdna4-gfx1201)). The RX 7800 XT / 7700 XT
+(gfx1101) and the RX 9060 XT (gfx1200) were validated by their owners (see [Community-validated
+cards](#community-validated-cards)); the RX 6800 / 6900 series (RDNA2, gfx1030) builds and runs too, reported by a community machine and not yet validated by the maintainers (see [RDNA2](#rdna2-gfx1030)). Setup chooses it by itself on a PC with no NVIDIA card Strata can use (`--backend hip` on a PC with both); the
+install steps for users are in [INSTALL.md](INSTALL.md#amd-cards). Other AMD architectures, wave64, and mixed
+AMD/NVIDIA execution in one run are not supported.
 
 The backend maps the CUDA-shaped runtime and BLAS calls to HIP/hipBLAS, uses
-RDNA3/RDNA4's signed integer dot instruction for quantized kernels, and supplies
+RDNA2/RDNA3/RDNA4's signed integer dot instruction for quantized kernels, and supplies
 wave32 shuffle/packed-byte operations. CUDA-only QSA matrix instructions have
 an ordered FP32 fallback. Prefill supports both dequantization plus hipBLAS GEMM and opt-in HIP ggml MMQ.
 An optional, calibrated hipBLASLt path accelerates dense projections (per-architecture tables in `tools/hip`).
@@ -15,8 +18,8 @@ This does not claim bit-identical model answers across backends. See
 
 ## Install with setup (recommended)
 
-On Linux with an RX 7900 XT / XTX or an RX 9070 / 9070 XT / Radeon AI PRO R9700 and the kernel's amdgpu driver
-(no ROCm install needed):
+On Linux with an RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT or Radeon AI PRO R9700 and
+the kernel's amdgpu driver (no ROCm install needed):
 
 ```sh
 ./setup.sh --backend hip
@@ -26,8 +29,10 @@ On Linux with an RX 7900 XT / XTX or an RX 9070 / 9070 XT / Radeon AI PRO R9700
   supported. On a PC without an NVIDIA card Strata can use, `--backend hip` is chosen automatically.
 - **ROCm:** a system ROCm 7 in `/opt/rocm` (or `$ROCM_PATH`) with hipcc and hipBLAS is used when present. Otherwise
   (or when it is older than 7.0) ROCm is installed into `.venv` from AMD's TheRock wheels (~10 GB, no sudo), pinned
-  to the version this backend was tested with, from the card family's index: `gfx110X-dgpu` for gfx1100,
-  `gfx120X-all` for gfx1201 (`STRATA_ROCM_VERSION` / `STRATA_ROCM_INDEX` override them).
+  to the version this backend was tested with, from the card family's index: `gfx110X-dgpu` for gfx1100 / gfx1101,
+  `gfx120X-all` for gfx1200 / gfx1201, `gfx103X-all` for gfx1030 (`STRATA_ROCM_VERSION` /
+  `STRATA_ROCM_INDEX` override them; the gfx1030 index is not checked to carry the pinned version: a system
+  ROCm 7 is the tested path there).
 - **Engine:** compiled on your PC for the card's architecture (10-20 minutes, once; again after a `git pull` that
   changes it, or when you pick a card of another architecture). This needs a C++ compiler and git
   (`sudo apt install build-essential git`).
@@ -35,10 +40,75 @@ On Linux with an RX 7900 XT / XTX or an RX 9070 / 9070 XT / Radeon AI PRO R9700
   card's architecture and the installed hipBLASLt version (read from `hipblaslt-version.h`; 1.2.0 is `100200`).
   Otherwise it says so and the prompt's dense matrix products use plain hipBLAS (slower prompts, same answers).
   A table's solution ids are valid only for that pair, and the engine refuses any other table.
-- **Limits for now:** one GPU, no images, no calibration. The Monitor shows no GPU statistics.
+- **Several cards:** setup takes one card (the one with the most VRAM, or `--gpu N`) unless you name more:
+  `./setup.sh --backend hip --gpus 1,0` splits the model's layers across them, the first one the main card (numbers
+  as setup lists them; `--gpus all` = every supported card, the most VRAM first). Every chosen card must be one of the
+  architectures above; the engine is compiled for each of them (cards of two families, e.g. gfx1100 + gfx1201, need
+  a system ROCm 7: AMD's wheels hold one family). A split pays only when no single card holds the model's experts
+  (see RDNA4 below).
+- **Limits for now:** images only through the CPU encoder (`--vision cpu`, 0.1.32), no calibration. The Monitor
+  shows the card's load, VRAM, temperature and power from Linux sysfs (0.1.32).
 
 The rest of setup is the same as on NVIDIA: the model download, the start script, the server.
 
+## Windows
+
+Since 0.1.34 an AMD card on Windows is set up like an NVIDIA one: download Strata, double-click `START-HERE.bat`.
+On a PC with no NVIDIA card Strata can use, the AMD card is chosen by itself; with both, setup asks
+(`START-HERE.bat --backend hip` picks AMD directly).
+
+- **You need:** Windows 10 or 11 (64-bit), one of the cards above, and a current AMD driver ([AMD Software:
+  Adrenalin Edition](https://www.amd.com/en/support/download/drivers.html)). Nothing else: no ROCm or HIP SDK
+  install, no compiler, no admin rights.
+- **Detection:** setup reads the display adapters Windows lists (their PCI ids; the VRAM size from the display
+  driver's registry entry). An integrated Radeon is listed as not supported.
+- **Engine:** the ready-made `strata-windows-x64-hip.zip` from the release (built by `tools\hip\build_windows.bat`
+  for gfx1100, gfx1101, gfx1102, gfx1200, gfx1201 and gfx1030) goes into `engine\`. It carries the ROCm libraries the
+  engine loads (`engine\rocm\bin`: the HIP runtime, hipBLAS / rocBLAS / hipBLASLt with their kernels for these cards,
+  amd_comgr and the Microsoft C++ runtime; ROCm 10.2.0a20260930 from AMD's TheRock builds, licenses in
+  `engine\rocm\licenses`). The HIP runtime works through the AMD driver's own components, so the driver is the one
+  thing it needs from the PC.
+- **The HIP runtime next to `strata.exe` (0.1.35, #468 #461):** `amdhip64_7.dll` and `amd_comgr.dll` are also put in
+  `engine\` (setup copies them there on every start). Windows looks in the program's folder before System32, where
+  some AMD drivers install their own `amdhip64_7.dll`; with that one, the bundled libraries crashed on the first
+  prompt (an access violation, or `hipErrorInvalidDeviceFunction`). The engine's log names the runtime it loaded
+  (`strata generate: HIP runtime ...`).
+- **Before the ~60 GB model download** setup runs `engine\strata-device.exe --list-devices` (with `engine\rocm\bin` on
+  the PATH): if the HIP runtime does not see the card, setup stops there and points to the driver. It also gives the
+  card's HIP number: with an integrated Radeon that is device 1, not 0 (#325). From then on setup lists the AMD cards
+  as HIP numbers them, so `--gpu N` and the config's `"gpu"` are HIP numbers.
+- **Differences from Windows-on-NVIDIA and Linux-on-AMD:** no images yet (the CPU image encoder is Linux-only for
+  now), one card per model (`--gpus` is Linux-only for now), no calibration.
+- Two Windows-only engine details (#247, #325): hipBLAS can return success and still leave `hipErrorInvalidValue`
+  set after some BF16/FP16 GEMMs (seen on gfx1201); the engine clears that one stale error after a GEMM that
+  succeeded, on Windows only. `hipHostGetDevicePointer` returns the host pointer itself on Windows: kernels read
+  mapped memory through it correctly, but a device-to-device copy into it does not land, which is why
+  `tests/hip/handoff` times out there (the engine does not use that copy; `tests/hip/mapped_alias` reports it).
+
+**What is validated (0.1.34):** #325's author ran the engine of this port on an RX 9070 XT (Windows 11, ROCm
+10.2.0a20260930 in `.venv`, compiled on the PC): Coder IQ1_M at 32K, 29.2 tok/s decode, ~181 tok/s prefill,
+correct answers; ctest 42 of 46. The maintainers have no Windows AMD card: the release zip was built on an NVIDIA PC,
+and checked there on the Ryzen CPU's integrated Radeon (gfx1036, a test build of the same tree): with only the zip's
+libraries on the PATH, `strata-device` lists the card and a hipBLAS BF16 GEMM matches the CPU; the HIP ctest passes
+52 of 56, the 4 failures the same as on the RX 9070 XT (`hip_handoff`, and three tests that need a pack fixture).
+The ready-made zip itself has not run a model on a discrete card yet - please report.
+
+**Reporting a Windows AMD run** (an issue, or on #325): your card and driver version (AMD Software > System), then
+
+```bat
+engine\strata-device.exe --list-devices
+engine\strata-device.exe --selftest
+```
+
+(from the Strata folder, after `set PATH=%CD%\engine\rocm\bin;%PATH%`), the end of `strata-<model>.log`, and the
+speed lines the server window prints for a first answer.
+
+**Building it yourself:** `tools\hip\build_windows.bat` (Visual Studio 2022 Build Tools with the C++ workload, Python,
+git; no admin, no AMD GPU) installs ROCm from AMD's TheRock wheels into `.rocm-win`, builds, and packages
+`dist\strata-windows-x64-hip.zip`; `START-HERE.bat --backend hip --prebuilt dist\` installs that one.
+`tools\hip\build_windows.bat tests` also builds the HIP tests (`ctest` in `build-hip-win`, with
+`.rocm-win\Lib\site-packages\_rocm_sdk_devel\bin` on the PATH). `STRATA_HIP_ARCHS` picks other architectures.
+
 ## Build
 
 Requirements: a working ROCm driver/runtime, HIP development headers and
@@ -54,9 +124,9 @@ cmake -S . -B build-hip \
 cmake --build build-hip --target strata -j2
 ```
 
-`CMAKE_HIP_ARCHITECTURES` is `gfx1100`, `gfx1201`, or a list such as `"gfx1100;gfx1201"` (one binary for both).
-gfx1101, gfx1102 and gfx1200 (the same wave32, 64 KiB LDS and dot4 instruction) build with a warning: they have
-not been validated on a real card here. At startup the engine and `strata-device` compare each GPU they use
+`CMAKE_HIP_ARCHITECTURES` is `gfx1100`, `gfx1101`, `gfx1200`, `gfx1201`, or a list such as `"gfx1100;gfx1201"`
+(one binary for both). gfx1102 (the same wave32, 64 KiB LDS and dot4 instruction) builds with a warning: it passed
+ctest (#192) but no model run has been reported; so does gfx1030 (RDNA2: the older `v_dot4_i32_i8`, a community run in #311). At startup the engine and `strata-device` compare each GPU they use
 (`gcnArchName` up to the `:` feature suffix) with the architectures the binary was compiled for, and require
 wave32. A binary carried to another card stops with the card's name, its architecture and the build's list,
 instead of failing later with "invalid device function".
@@ -113,9 +183,8 @@ The worker count above was used on a 16-core CPU; measure it for your CPU.
 The 4K context is a smoke-test starting point, not a model limit. The expert cache
 sizes itself automatically and leaves 1 GiB of VRAM headroom.
 
-The installer supports this backend (see "Install with setup" above). The vision helper is NVIDIA-only for now.
-Setup installs one AMD card; the engine's layer split also runs on two AMD cards when the config is written by hand
-(see RDNA4 below).
+The installer supports this backend (see "Install with setup" above). Images run through the CPU encoder for now (`--vision cpu`).
+Setup installs one AMD card, or several with `--gpus` (the engine's layer split; see RDNA4 below).
 
 ## RDNA4 (gfx1201)
 
@@ -145,12 +214,16 @@ an RX 9070 XT 16 GB and a Radeon AI PRO R9700 32 GB (both gfx1201), a Ryzen 9 39
   | RX 9070 XT 16 GB | 4,931 slots, 9.4 GiB | 15.6 GiB | 782 tok/s | 30.8 tok/s | 1,235 tok/s | 35.4 tok/s |
   | R9700, TheRock 7.10 wheels | 12,288 slots | | 957 tok/s | 44.2 tok/s | 1,284 tok/s | 43.9 tok/s |
 
-  The engine's resident memory was about 26 GB in every run.
-- **hipBLASLt:** there is no gfx1201 table in `tools/hip`. A table calibrated on the R9700 at the engine's shapes
-  (hipBLASLt 1.4.1; 0.98-1.76x per GEMM over hipBLAS) changed the end-to-end prompt speed by 0-3%, within noise,
-  so none is shipped: on gfx1201 the plain hipBLAS path is already close.
+  The engine's resident memory was about 26 GB in every run. Since 0.1.31 `__byte_perm` is one `v_perm_b32` and the
+  packed byte subtracts/compare work on four lanes at once (#262, ttio2tech): decode +15% on the R9700 (46.0 -> 53.0
+  tok/s on a 4K prompt, 52.0 -> 60.5 warm) and +5-7% on the 9070 XT, prompts unchanged, the same tokens.
+- **hipBLASLt:** the validation above used hipBLASLt 1.4.1. A table calibrated on the R9700 at the engine's shapes
+  with that version (0.98-1.76x per GEMM over hipBLAS) changed the end-to-end prompt speed by 0-3%, within noise,
+  so none was shipped for it: there the plain hipBLAS path is already close. For hipBLASLt 1.5.0 (ROCm
+  10.2.0a nightly) `tools/hip/gfx1201-hipblaslt-100500.txt` is shipped (see "Tuning table" below); on one R9700 it
+  measured +3.9% prompt speed on 4,210-token prompts (1,590 vs 1,531 tok/s), a modest gain.
 - **Both cards in one run (layer split, engine 0.1.30):** the config's `"backend": "hip", "gpu": [1, 0]` (R9700
-  first) runs through `serve/server.py`; setup does not offer it yet. Auto split put layers 0-27 on the R9700 and
+  first) runs through `serve/server.py` (setup writes it with `--gpus 1,0` since 0.1.31). Auto split put layers 0-27 on the R9700 and
   28-47 on the 9070 XT. With every expert on the GPUs the split gives exactly the tokens of the R9700 alone (4K and
   16K prompts); checkpoints on the split (second turn, rewind, a prompt sharing a prefix, a cancelled prompt
   retried) give exactly the tokens of a fresh read. The conversation cache refuses a split at start (exit 2).
@@ -162,12 +235,90 @@ an RX 9070 XT 16 GB and a Radeon AI PRO R9700 32 GB (both gfx1201), a Ryzen 9 39
   | R9700 + 9070 XT, auto split | 1,384 tok/s | 1,906 tok/s | 42-43 tok/s |
   | RX 9070 XT alone | 1,016 tok/s | 1,519 tok/s | 38-40 tok/s |
 
-  A split is worth it when no single card holds the model's experts. On Linux the split pins at most 8 GiB of the
-  expert arena (a Windows limit that also applies here).
+  A split is worth it when no single card holds the model's experts. Since 0.1.31 a split pins the whole expert
+  arena on Linux (the 8 GiB cap is for Windows/WSL2 only, #253): with an expert cache of 1,500 on the R9700 (most
+  experts streamed) the split read a 16K prompt at 1,803 tok/s instead of 1,287, with the same tokens in 5 + 5
+  starts. `"split_skip_if_fits": true` in the config (0.1.31, opt-in) runs such a pair on the first card alone when it
+  holds every profiled expert: with the R9700 first, 4K prompts 1,776 tok/s (split: 1,244) and decode ~60 tok/s
+  (split: ~51), the tokens of the R9700 alone (docs/MULTI_GPU.md).
+- **Speed switches (engine 0.1.32, measured on the R9700 / 9070 XT with the Coder IQ1_M pack):**
+  - the MoE router (`router_top10`) runs a HIP kernel without its serial FP64 sum and block barriers by default: the
+    same ids and weights bit for bit (`hip_router_fast` checks 65,536 rows), 39 -> 9-12 us per call, the same greedy
+    tokens (5 + 5 starts). Decode: 62.4 -> 70.0 tok/s on the R9700 and +4% on the 9070 XT in one A/B here (ROCm 10.2
+    nightly); a user's repeated A/B/A/B with setup's TheRock 7.10 wheels and an i5-12600K measured +1-4%, inside a
+    12-20% run-to-run spread (#432) - how much it gains depends on how much of decode the router is on that setup.
+    `STRATA_HIP_ROUTER_OLD=1` runs the portable kernel.
+  - `STRATA_HIP_WMMA=1` (opt-in, gfx12 only, int8 KV): the prompt path's QSA attention on RDNA4 matrix cores
+    (`v_wmma_f32_16x16x16_f16`, FP16 hi + lo halves like the CUDA tensor-core kernel; `hip_prompt_attn_wmma` bounds it
+    against the FP32 kernel and FP64). 7.2-7.5x the portable kernel; R9700 prompts 4K 1,784 -> 2,427 tok/s, 16K
+    1,797 -> 2,700 (a user with TheRock 7.10: +29-34%, #432). Not bitwise: greedy text differs from token ~50 on, as with the CUDA tensor-core attention. With it
+    the prompt path's expert ring is 96 slots (as STRATA_PREFILL_RING=96): with the default 384 the 9070 XT's 4K
+    prompts fell to 718 tok/s; with 96 they gain (1,017 -> 1,211; 16K 1,518 -> 2,032). PR #329
+    (bsorensen110) contributed an equivalent gfx12 WMMA kernel of the same speed (within 1%); this one also masks KV
+    pages that KV streaming has not made resident, as the decode kernel does.
+  - The prompt path's QSA top-k picks its kernel by the blocks a query actually has, not the cache's capacity (#337,
+    bsorensen110): the same ids, on by default with AMD (NVIDIA keeps its capacity rule: there the 64K prompts read
+    1-3% slower with it). `STRATA_SELECT_WMMA=1` (opt-in, gfx12) adds #337's
+    matrix-core block scorer; it selects slightly differently (254 of 256 queries the same) and gained +1.5% on 16K
+    prompts at a 262K context on the R9700.
 - **Known:** rarely (about 1 start in 10) a HIP run's greedy output differs from another start's at some token, on
   one card or two and on engine 0.1.29 as well; not yet explained.
-- **Not validated:** gfx1200 (RX 9060 XT; a community report is #176), images, long contexts beyond 16K,
-  answer-quality benchmarks.
+- **Not validated:** images, long contexts beyond 16K, answer-quality benchmarks.
+
+## Community-validated cards
+
+Run by their owners, not on the maintainers' machines; setup accepts them like gfx1100 / gfx1201. No hipBLASLt table
+is shipped for them (make one with [Tuning table](#tuning-table) and compare the prompt speed with and without it).
+
+- **gfx1101, RX 7800 XT 16 GB** (jhohertz, #254; engine 0.1.29, Ryzen 9 5950X, 121 GiB RAM, system ROCm with
+  hipBLASLt 1.4.1): `./setup.sh --backend hip` detected the card and compiled the engine; `strata-device --selftest`
+  passed; ctest 30/32 (`ple_parity` needs the Q2_0 fixture, `platform_memory_test` the memlock limit). Coder IQ1_M,
+  64K context, MTP, with a table the owner calibrated: fresh prompts of 4K-9K tokens at 898-953 tok/s, decode
+  38-44 tok/s (128 tokens).
+- **gfx1200, RX 9060 XT 16 GB** (Efeisot, #256 after #176; engine 0.1.29, Ryzen 9 7950X, 64 GiB RAM, ROCm 7.2 with
+  hipBLASLt 1.2.2): ctest 32/32 (without `ple_parity` and `platform_memory_test`). Coder IQ1_M, greedy, MTP
+  `--spec 4 --spec-min-p 0.5`, with a table the owner calibrated:
+
+  | prompt | prompt speed | decode |
+  |---|---|---|
+  | 2,374 tokens | 540 tok/s | 27.1 tok/s (0.69 draft acceptance; 31.0 at 1.00) |
+  | 65K (`--kv int8 --kv-resident 65536 --prefill 16384`) | 748-753 tok/s | 26-40 tok/s by acceptance |
+  | 130K (the same flags) | 725 tok/s | - |
+
+  Greedy output was the same across runs. For comparison, llama.cpp's HIP build measured 20 tok/s decode and
+  450 tok/s prompt on that card.
+
+## RDNA2 (gfx1030)
+
+The RX 6800 / 6800 XT / 6900 XT / 6950 XT run the same kernels: wave32 and 64 KiB of LDS per workgroup. The one
+difference is the dot instruction: RDNA2 has no `v_dot4_i32_iu8` (gfx11 and newer), so
+`dp4a` uses the plain signed `v_dot4_i32_i8` through `__builtin_amdgcn_sdot4`, which compiles to a single
+`v_dot4c_i32_i8` (same signed x signed byte products, modulo 2^32). There is no WMMA; the QSA scorer takes the
+same ordered FP32 fallback as gfx1100. CMake lists gfx1030 as unvalidated (the build warns) until a maintainer has
+run it; the report below is from a community machine: an RX 6900 XT 16 GB (gfx1030), an i7-13700KF (8 P-cores and
+8 E-cores, AVX2, no AVX-512), 63 GB RAM, NixOS, ROCm 7.2.3 from nixpkgs (clang 22, hipBLAS 3.2).
+
+- **Build and tests** (engine 0.1.26): a complete HIP build for gfx1030, made by hand with cmake and ROCm's own
+  `clang++` (the nixpkgs ROCm is not an `/opt/rocm` tree, so setup's `build_engine_hip` was not exercised; the
+  binary was placed in `engine/` for setup to use). All 28 registered ctest tests pass on the card, including
+  `hip_device_selftest`. An engine 0.1.30 build of this branch configures, builds and runs clean on the same card;
+  the speeds below are measured on it.
+- **End to end** (Swift 1.5 IQ3_XXS, `--context 131072` with `--kv-resident 32768 --adapt-every 1
+  --vram-reserve-mib 1024`, 200 greedy tokens): 38-42 tok/s decode with the default 15 CPU pool workers (one per
+  physical core except the host thread; 8 workers: 36; 24 = every logical core: 27), consistent even with the
+  131,072-token context full.
+- **Prefill** (the same 15-worker configuration): 246 tok/s on a 2,000-token prompt, 330-339 tok/s at the auto
+  8,192-token chunk (7,997 and 15,967 tokens; time to first token 8.9 and 48.3 s), 133 tok/s on a 522-token
+  prompt - short prompts are fixed overhead (19-25 tok/s on 34 tokens). Decode after a 16K prefill holds at
+  45.5 tok/s.
+- **16 GB card:** the expert cache holds 6,310 slots (10.2 GiB) at 16K context and 5,247 slots at 131,072, where
+  setup keeps the KV cache in VRAM because IQ3_XXS needs about 60 GB of RAM plus the cache to stream it. The
+  `--pcie-frac 0` and `--adapt-every 0` of the 7900 XTX configuration in
+  [AMD_HIP_PERFORMANCE.md](AMD_HIP_PERFORMANCE.md) cost 8.6 and 13.6 tok/s here (30 with the defaults): keep the
+  defaults on a 16 GB card.
+- **hipBLASLt:** ROCm's hipBLASLt ships no gfx1030 kernels, so there is no table and the plain hipBLAS path runs.
+- **Not validated:** gfx1031 / gfx1032 (the same `dp4a` path, no hardware report), setup's own build path and the
+  `gfx103X-all` wheels on gfx1030, images, answer-quality benchmarks.
 
 ## Tuning table
 
@@ -185,6 +336,20 @@ The file's second line names the architecture and version (`STRATA_HIPBLASLT_TUN
 as `tools/hip/<arch>-hipblaslt-<version>.txt` for setup, or point `STRATA_HIPBLASLT_TUNING` at it. Compare the
 prompt speed with and without it before keeping it.
 
+Shipped tables:
+
+- `gfx1100-hipblaslt-100100.txt`, `gfx1100-hipblaslt-100200.txt`: RX 7900 XTX.
+- `gfx1201-hipblaslt-100500.txt`: Radeon AI PRO R9700 (gfx1201, 32 GB), calibrated with ROCm 10.2.0a20260914
+  (AMD's `gfx120X-all` nightly, hipBLASLt 1.5.0, library build `d3164197`). 16 dense GEMM geometries at T=4096 and
+  T=8192, 32 rows. setup uses it only when the installed hipBLASLt reports 1.5.0 (it is found in `/opt/rocm`
+  when that is a system ROCm 7 or newer). The version number is the only thing the engine can check, so another
+  1.5.0 build could number its solutions differently. `hip_prefill_hipblaslt_gemm` (with `STRATA_HIPBLASLT_TUNING`
+  set) is a smoke test: it refuses a table for another architecture or version and checks that one BF16 and one
+  F16 row exist and agree with hipBLASEx, which covers 2 of the 32 rows. It does not prove that a solution id is
+  valid: the engine falls back to hipBLASEx for an id the library rejects, and the test still passes. Run it with
+  `STRATA_HIPBLASLT_VERBOSE=1` and look for `fallbacks=0` in its summary line, and recalibrate with
+  `tune_hipblaslt` before using this table with a different 1.5.0 build.
+
 ## Original backend validation (PR #94)
 
 The following is historical validation of the original backend, not a fresh

docs/COMMUNITY_BENCHMARKS.md

@@ -0,0 +1,167 @@
+# Community benchmark results
+
+Share measurements from your own hardware so other users can judge how Strata
+might run on a similar machine. This guide suggests a report format and a pull
+request workflow; a single model, GPU, or context length is enough to contribute.
+
+Existing [benchmark reports](../bench/results/) provide examples. The
+[engine 0.1.26 speed report](../bench/results/2026-09-29-speed-0126/README.md)
+includes hardware, settings, and per-run JSON. The
+[technical details](DETAILS.md#speed-measured) explain the published measurements
+and their limits. Report what you actually measured and label estimates separately.
+
+## Community reports
+
+- [2026-09-30: RTX 5090, Core Ultra 9 285K, 64 GB RAM](../bench/results/2026-09-30-community-rtx-5090/README.md):
+  Strata 0.1.29, original Flash-Next IQ2_XS, 131,072-token context; three runs
+  each at 4,096, 32,768, and 128,000 prompt tokens, plus six recall checks.
+
+## What to record
+
+Include enough information for someone else to repeat your run:
+
+- **Hardware:** GPU model, VRAM, selected GPUs for a multi-GPU run, CPU, installed
+  RAM, and storage type. Include PCIe link speed and width, GPU power limits, and
+  other workloads when known or relevant.
+- **Software:** OS, Strata commit and engine version, driver and CUDA version
+  (or ROCm for HIP), and release binary or source build. Include changed build options.
+- **Model:** exact repository and revision when available, quantization, GGUF
+  filenames, and vision encoder if enabled. Identify custom packs, expert
+  profiles, or draft vocabularies, with hashes or reproducible preparation steps.
+- **Settings:** launch command and relevant run configuration, context limit, KV
+  type and streaming window, expert cache, low-RAM mode, prefill size, CPU workers,
+  MTP, reasoning effort, sampling, and vision. State whether calibration or
+  experimental speed projection was enabled; attach the resulting settings.
+- **Workload:** benchmark command or script, shareable prompts or instructions to
+  build them, actual prompt and generated token counts, output cap, repetitions,
+  and cache state for each run.
+
+Use GB or GiB consistently and name the unit. Mark unavailable values explicitly
+instead of guessing. Remove credentials and private content from configurations,
+prompts, and logs before publishing them.
+
+## How to measure
+
+Start with a configuration that already works on your machine. You do not need to
+download every model or test the largest context. A useful initial speed report
+tests one short prompt and one longer prompt that fits, with a fixed output cap
+(for example, 256 tokens). Record the actual output length if the model ends early.
+
+For each configuration, try at least three measured runs and publish all of them
+with a median and range. If you only have one run, say so. Explain any warm-up and
+whether model loading is included in the timing.
+
+Strata reuses conversation prefixes and adapts its expert cache. Separate prompts
+that are fully processed from follow-ups that reuse tokens, and report the number
+of reused and freshly read tokens from the engine log. Describe how you reset or
+retained state between runs. A warmed expert cache and a reused prompt prefix are
+different conditions; record both when possible.
+
+Keep these measurements separate:
+
+- **Prompt throughput:** freshly processed prompt tokens per second, from the
+  engine's timing output. Keep the corresponding token count and duration.
+- **Decode throughput:** generated tokens per second, from the engine's timing
+  output. Count reasoning tokens too, or explain any different convention.
+- **Time to first token:** seconds from sending the request to its first generated
+  token. For a streaming API measurement, ignore keep-alives and empty deltas and
+  state whether the first token is reasoning or answer text.
+- **Total latency:** elapsed seconds for the whole request. State whether measured
+  at the engine or client, and whether loading, vision encoding, or queueing is included.
+- **Memory:** observed RAM and VRAM usage, including whether the value is a startup
+  snapshot or a peak during inference. Report paging or out-of-memory failures.
+
+Attach the engine timing lines and your measurement script if you used one. Avoid
+calculating decode throughput by dividing generated tokens by total request time,
+which also includes prompt processing. Cancelled or failed requests belong in the
+report as failures, outside the successful throughput summary.
+
+When comparing versions or settings, run both on the same machine and workload
+and list every changed setting. Preserve the per-run results: output text and
+draft acceptance can change speed even when the input is identical.
+
+## Optional correctness checks
+
+Speed measurements alone do not establish answer quality. Include a small check
+relevant to your workload, such as a completed coding task with test results, a
+tool call followed by its result, or an image question with an expected answer.
+
+For long-context recall, the repository includes
+[`tools/needle_bench.py`](../tools/needle_bench.py). With a running local server,
+from the repository root:
+
+```bash
+python tools/needle_bench.py --help
+python tools/needle_bench.py --url http://127.0.0.1:8080 --lengths 32k,128k --depths 10,50,90 --out needles.json
+```
+
+Choose lengths your configured context can hold. The script supports `--api-key`
+for an authenticated server; keep that key out of published commands. Save the
+reported actual prompt lengths, misses, errors, and skipped cases as well as
+successes. A needle test measures recall on those inputs, not overall model quality.
+
+## Submit a report
+
+1. Fork Strata and create a branch for your report.
+2. Add a folder such as `bench/results/YYYY-MM-DD-community-rtx-5090/`. Use the
+   measurement date and a short hardware label; add a suffix if the name exists.
+3. Put a `README.md` based on the template below in that folder, together with
+   per-run JSON or CSV and small scripts or prompts needed to repeat it. Use your
+   own data format and document its fields and units. Keep large model files and
+   generated packs out of the PR.
+4. Open a pull request against `Niko1221/Strata:main`. Summarize the hardware,
+   model, configurations tested, and limitations. Keep a results-only submission
+   separate from engine changes so reviewers can assess the measurements directly.
+
+Maintainers can review the report and decide where to include it. Adding a report
+does not require changing the README's headline performance claims. For a failed
+run or a suspected bug, a GitHub issue with the configuration and relevant logs
+may be more useful than a benchmark PR.
+
+## Report template
+
+Copy this into your report's `README.md`, replace the placeholders, and remove
+sections that do not apply. Use `not measured` for missing measurements; the table
+contains no example performance numbers.
+
+````markdown
+# Community benchmark on GPU name
+
+Measured on DATE by HANDLE. State what was tested and the main limitation.
+
+## Hardware and software
+
+- GPU and VRAM; CPU; installed RAM; storage; PCIe link if known:
+- OS; driver; CUDA or ROCm:
+- Strata commit; engine version; release binary or source build:
+- Background workloads and any power limits:
+
+## Model and configuration
+
+- Model repository and revision; quantization; GGUF filenames:
+- Vision encoder; custom packs or profiles:
+- Context; KV type and streaming; cache; prefill; low-RAM mode:
+- MTP; reasoning; sampling; calibration; experimental speed projection:
+
+```text
+Exact launch command and relevant configuration, with credentials removed.
+```
+
+## Method
+
+Link prompts and scripts. Describe output cap, repetitions, warm-up, loading,
+prompt reuse, expert-cache state, timing boundaries, and memory measurement.
+
+## Results
+
+| Configuration | Actual prompt tokens | Reused tokens | Generated tokens | Runs | Prompt tok/s median and range | Decode tok/s median and range | TTFT seconds median and range |
+| --- | ---: | ---: | ---: | ---: | --- | --- | --- |
+| Fill in measured values | | | | | | | |
+
+Link per-run data and logs. Include total latency and memory observations when
+measured, with units and timing boundaries. List failures and skipped cases.
+
+## Correctness and limitations
+
+Describe checks, expected answers, observed results, and untested behavior.
+````

docs/DETAILS.md

@@ -1,11 +1,13 @@
 # Strata - the details
 
 The technical side of Strata: every measured number, the API, images, all settings and how the engine works.
-New here? Start with the [README](../README.md) - it has everything you need to install and use it.
+New here? Start with the [README](../README.md); installing step by step is in [INSTALL.md](INSTALL.md), the models in
+[MODELS.md](MODELS.md), common problems in [TROUBLESHOOTING.md](TROUBLESHOOTING.md).
 
 > **On this page:** [Speed](#speed-measured) · [Other GPUs](#other-gpus-estimated) · [Which model?](#which-model) ·
 > [Requirements](#before-you-start) · [Windows](#windows) · [Linux](#linux) · [API](#using-it) ·
-> [MCP tools](#tools-from-mcp-servers) · [Images](#images-vision) ·
+> [MCP tools](#tools-from-mcp-servers) · [MCP server](#manage-strata-from-your-ai-assistant-mcp-server) ·
+> [Images](#images-vision) ·
 > [Troubleshooting](#troubleshooting) · [How it works](#how-it-works)
 
 ---
@@ -50,14 +52,17 @@ accepted, so a different answer to the same prompt moves it by several percent.
 [`bench/results/2026-09-28-speed-0114`](../bench/results/2026-09-28-speed-0114/README.md).
 
 IQ3_XXS and IQ3_S at 262K are not measured: with their 43 / 50 GB of experts, a 260K-token context brings a 64 GB PC
-to its memory limit. Use up to 128K with them on 64 GB (setup caps it). IQ3_S (engine 0.1.4 or newer) is only published
-for the original model, not for Swift 1.5.
+to its memory limit by setup's estimate (the experts + the context's KV cache + 24 GB), so setup recommends up to 128K
+with them on 64 GB. A longer context you choose (`--context 262144`, or a pick in its list) is kept, with a note: users
+ran IQ3_S at 256K on 64 GB with RAM to spare (#406). In the low-RAM mode the KV cache stays in VRAM and the context
+does not count against RAM. IQ3_S (engine 0.1.4 or newer) is only published for the original model, not for Swift 1.5.
 
 **KV streaming (engine 0.1.5):** at 64K and more, setup keeps the context's KV cache in RAM and only the part the
 attention reads in VRAM (`--kv-resident 32768`), so more experts fit on the GPU. Q2_0 at 262K: 50.9 -> 62.6 tokens/s
 (1,589 -> 3,872 experts in VRAM); at 128K about +6%. The attention reads exactly the same values (only where the KV lives changes); it
 costs ~13.7 KB of RAM per context token (1.7 GB at 128K). Existing installs: run `START-HERE.bat --setup` once to turn
-it on.
+it on. Setup turns it on when the RAM has room for it; `--kv-streaming on|off` overrides that (on past the RAM test with a
+note; never with `--kv k8v4` or under WSL, which cannot stream).
 
 **4-bit KV cache (engine 0.1.8, optional):** `START-HERE.bat --setup` asks above 8K context (or pass `--kv q4_0`). It
 halves the KV cache's memory with a Hadamard rotation before 4-bit rounding (PR #21), about 4% faster at 128K, but it
@@ -76,14 +81,24 @@ slightly differently. How many tokens share an expert depends on the drafts in a
 at temperature 0 can end in a different (equally good) answer when the drafting, the cache state or a resumed
 conversation differ (issue #152). `STRATA_IQ_MT_MIN=1` (in the config's `env`) uses the multi-token kernels for
 every group: the answer then no longer depends on the drafting. Measured on a Ryzen 7600 (AVX-512): IQ3_S decode
--1..-3%, the other models the same; the default stays the fastest rule.
+-1..-3%, the other models the same; the default stays the fastest rule. Through the server, two more things carry
+over from one request to the next (#410): the adaptive tier moves experts between RAM and VRAM (the GPU and the CPU
+round an expert differently), and the prompt cache resumes a repeated prompt and reads only its tail through the
+decode path. For byte-identical repeats add `--prompt-cache 0 --adapt-swaps 0 --pcie-frac 0` to the engine's args
+as well (#410): the PCIe share of the missed experts (computed on the GPU instead of the CPU) still made the first
+answer after a start differ from the next ones. Measured here (IQ3_XXS, a 3.6K-token prompt, 4 repeats): with all
+three switches 1 answer of 4, without `--pcie-frac 0` 2 of 4 (the first one differs), with the defaults 2 of 4.
+`--pcie-frac 0` costs decode speed (the missed experts all run on the CPU), so keep it for A/B runs.
 
 **The draft layer's tokens (0.1.27, `--draft-vocab`):** the MTP draft layer can only propose tokens from a subset
 of the vocabulary (`mtp/rt/draft_vocab.bin`). Since 0.1.27 the subset includes every Chinese, Japanese and Korean
 token (106,299 ids), so answers in those languages are 15-38% faster (Q2_0, RTX 5070). Its head takes ~180 MiB of
 VRAM, which the expert cache leaves free for it (0.1.28). `START-HERE.bat --setup --draft-vocab en` keeps the
 English/code subset from before (40,525 ids, ~110 MiB less VRAM, English answers 1-2% faster; CJK answers get
-almost no drafts). `tools/draft_vocab.py` builds and inspects subsets.
+almost no drafts). `--draft-vocab cyrillic` takes the English/code subset plus the whole Cyrillic script (58,963
+ids): the shipped subsets hold 142 of the vocabulary's 18,580 Cyrillic tokens, so Ukrainian or Russian answers got
+1.4 tokens a round; with it 2.1, and 83 -> 109 tokens/s (RTX 5090, the NVFP4 fork), English unchanged.
+`tools/draft_vocab.py` builds and inspects subsets.
 
 **Low-RAM mode (engine 0.1.26, chosen by setup):** normally all of a model's experts are copied into RAM (23-50 GB,
 pinned) and the GPU holds a copy of the most-used ones. On a PC whose RAM cannot hold them beside the system (the
@@ -111,6 +126,41 @@ other ~18 GB), a 32 GB PC with a 12-16 GB GPU the Coder; IQ3_XXS on a 32 GB PC s
   not hold do not fit, it says so and runs the plain mapped mode. The server log shows, per request, how many expert
   reads went to the file (`resident RAM: ... blob reads from the file`: 0 in steady use).
 - `--low-ram resident|mmap` forces one variant (also on a PC with enough RAM, e.g. to try it).
+- Several GPUs (#364, #384): setup recommends one GPU in the low-RAM mode (the resident variant has no layer split
+  yet), and asks; `--gpus 0,1` (or answering 2) shares the model across them with the mapped variant
+  (`--mmap-experts`): the cards together hold more of the experts, and two users measured it 1.3-1.6x faster than
+  one card, but the OS file cache can fill the RAM to 0 free during long prompts. `--yes` keeps one GPU. A config
+  with `--resident-experts` started with `--gpus` switches to `--mmap-experts` with a note, and the engine runs that
+  pair as `--mmap-experts` with a warning instead of refusing it.
+
+**Low-RAM mode without `experts.bin` (engine 0.1.31):** for the native packs (IQ2_XS, IQ3_XXS, IQ3_S, the Coder, Swift,
+Q2_0 packed by `tools/iq_pack.py`; not the canonical Q2_0 pack setup makes for AVX-512 CPUs) the mapped mode no longer
+needs the pack's `experts.bin`: when the pack has none, the engine
+maps the model's GGUF files themselves and reads each expert's gate, up and down rows from where `native_experts.txt`
+says they are (the files are checked against it first: every tensor's name, type, shape, offset and bounds). That
+saves the 23-50 GB copy on the disk. The answers are the same: on the Coder, 64 greedy tokens from `experts.bin` and
+from the GGUF gave identical tokens and logits. An expert read from the GGUF is three reads instead of one, so the
+engine fetches a layer's missing experts on 8 threads (`STRATA_FETCH_THREADS`) with one batched page request
+(Windows `PrefetchVirtualMemory`). With an `experts.bin` in the pack, nothing changes. Setup does not use this yet.
+
+**A RAM budget (engine 0.1.31, `--resident-budget-gib N`):** the resident variant for a model whose experts do not all
+fit: the N GiB of experts the GPU cache does not hold that the expert profile ranks hottest are copied into RAM at
+start (locked; page-locked when the driver allows the whole budget), and the rest are read from the files through the
+OS file cache. It implies `--mmap-experts` and leaves 4 GB of free RAM (a larger N is clamped to that less 256 MiB,
+with a message; #403: a clamped budget no longer fails the safety check that follows, and a budget that cannot be
+kept at all is a warning, with every expert read from the files). Setup sets N with `--resident-budget-gib N`. With
+the GGUF read in place it also warms the next layer's likely experts: while the CPU works on a layer, a thread applies
+the next layer's router to this layer's input and asks the OS for the pages of the predicted experts that neither the
+GPU nor the RAM budget holds (only pages - the experts computed are the same; `STRATA_LOOKAHEAD=0` turns it off). This
+is what runs [Unsloth's UD-Q4_K_XL](UNSLOTH_Q4.md) (72 GiB of experts) on a 64 GB PC: 7-8.5 tokens/s at N = 40 on an
+RTX 5070, against ~3 tokens/s before these changes.
+
+**How much came from where:** with `--stats` the engine prints the tiers of the decode (`expert tiers`: blobs from the
+RAM copy, blobs and MB from the files, the time spent reading them; `routing prefetch`: how many of the file reads had
+been warmed). The server log has the same per request (`expert tiers: GPU ... hits ...; RAM ... blobs, files ...
+blobs ... MB read`), and `GET /metrics` lists `ram_blobs`, `file_blobs` and `file_mb` for each recent request (with
+engine 0.1.31 or newer). It also lists each request's speculative drafts, `drafts_offered` and `drafts_accepted`
+(`null` when the engine did not report them), and their sums since the server started in `totals` (#457).
 
 Time to first token is prompt length / prompt speed: with Q2_0 about 4 s at 4K, 25 s at 32K, under 2 minutes at 128K
 and 4.5 minutes at 262K (engine 0.1.13 made long prompts about twice as fast, below).
@@ -195,14 +245,24 @@ thought about for 1,524 tokens. Not a benchmark, but consistent with the claim.
 START-HERE.bat --setup --family swift --model IQ2_XS
 ```
 
+### Experimental: Unsloth's UD-Q4_K_XL
+
+A 4-bit quantization of the same model (111 GB, 72 GiB of experts). Setup offers it from engine 0.1.32
+(`--family unsloth --model UD-Q4_K_XL`: a RAM budget of your RAM less 24 GB, the rest read from the SSD); what it
+does and the manual workflow are in **[docs/UNSLOTH_Q4.md](UNSLOTH_Q4.md)**. On a 64 GB PC with a 12 GB RTX 5070 it writes 7-8.5
+tokens/s, most experts read from the SSD; it picks the same tokens as llama.cpp on the same file at 97.5-99% of
+the positions of short greedy answers, 90-91% after a 16K prompt, differing mostly at near-ties
+([measured](UNSLOTH_Q4.md#quality-against-llamacpp-on-the-same-file)).
+
 ## Before you start
 
-You need **only an NVIDIA driver** (version 580 or newer; update it with the NVIDIA App or from
-[nvidia.com/drivers](https://www.nvidia.com/drivers)). Everything else is installed for you the first time.
+You need **only a graphics driver**: NVIDIA 580 or newer (update it with the NVIDIA App or from
+[nvidia.com/drivers](https://www.nvidia.com/drivers)), or for AMD the one in [INSTALL.md](INSTALL.md#what-you-need).
+Everything else is installed for you the first time.
 
 | | |
 | --- | --- |
-| GPU | NVIDIA **RTX 20, 30, 40 or 50 series**, **12 GB VRAM or more** (8 GB runs, slowly). Measured on an RTX 5070 and an RTX 3090; RTX 20 (Turing, since 0.1.27) was tested by a contributor on an RTX 2070. |
+| GPU | NVIDIA **RTX 20, 30, 40 or 50 series**, **12 GB VRAM or more** (8 GB runs, slowly). Measured on an RTX 5070 and an RTX 3090; RTX 20 (Turing, since 0.1.27) was tested by a contributor on an RTX 2070. Or AMD **Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700, RX 6800 / 6900 series**: [AMD_HIP.md](AMD_HIP.md). |
 | RAM | **64 GB** recommended (see the table above). |
 | CPU | x86-64 with AVX2 (any Intel/AMD desktop CPU from the last ~8 years). AVX-512 (Ryzen 7000/9000) is a bit faster. |
 | Disk | ~70-80 GB free for the model, ~6 GB for the MTP layer (+1 GB with images). **Q2_0 on an AVX-512 CPU** also writes a one-time ~40 GB copy of its experts for the fast CPU kernel. An NVMe SSD is strongly recommended. |
@@ -383,6 +443,16 @@ print(r.choices[0].message.content)
   Without a setting the model uses its own default, **high**. `none` answers at once (fastest); `low` keeps the thinking
   short. The levels are instructions the model was trained with, not a hard token limit: on easy questions all three
   think briefly, on hard ones `high` thinks longest and is most accurate.
+- **A hard thinking budget (opt-in).** `"reasoning_budget_tokens": N` in a request (OpenAI or Anthropic) caps the
+  thinking at N tokens: when it gets there the server ends it with a short wrap-up line and `</think>`, and the model
+  answers from there (the engine continues from what it already holds, so nothing is read again). The wrap-up is
+  part of the thinking the client sees and counts as output tokens. `"reasoning_budget_tokens": N` in
+  `strata-<model>.json` sets it for every request; a request's own value wins, and `0` means no budget. Off by default;
+  Anthropic's `"thinking": {"budget_tokens": N}` still only chooses the level, as above.
+- **Anthropic requests that don't ask for thinking (opt-in, 0.1.32, #278).** By default a `/v1/messages` request
+  with no `"thinking"`, effort or budget thinks as the model's template does. `"anthropic_thinking": "on_request"` in
+  `strata-<model>.json` renders such a request without thinking - Anthropic's own rule, and what Claude Code's short
+  helper calls (a session title in a few dozen tokens) need; its real turns ask for thinking when it is on there.
 - **Streaming.** With `"stream": true` everything arrives as it is made: the thinking, the answer, and tool calls
   (the tool's name first, then its arguments piece by piece, like OpenAI and Anthropic do). While the model reads a
   long prompt the stream sends keep-alives, so agents do not time out; the server window prints progress every
@@ -396,6 +466,9 @@ print(r.choices[0].message.content)
   `max_tokens` would run past the context is refused too (400); agents that always ask for their full output cap
   can instead get it shortened to the room left: add `"fit_max_tokens": true` to `strata-<model>.json` (or pass
   `--fit-max-tokens` to `serve/server.py`). A prompt that leaves no room at all is still refused.
+- **Model aliases** (0.1.32). `"aliases": ["qwen", "local-model"]` in `strata-<model>.json` lists the model under
+  those names too in `/v1/models` (each with its own `id`, and in the model's `aliases`), like llama-server's
+  `--alias`; a request naming one is answered under that name. Any other name is still served, as before.
 - **From other devices on your network.** The server listens on your PC only (`127.0.0.1`) unless you say otherwise:
   run setup with `START-HERE.bat --setup --host 0.0.0.0 --api-key some-long-secret` (or add `"host": "0.0.0.0"` and
   `"api_key": "..."` to `strata-<model>.json`). The server window then prints this PC's addresses
@@ -406,7 +479,12 @@ print(r.choices[0].message.content)
 - **From the internet.** Put a tunnel in front of it, for example [cloudflared](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/do-more-with-tunnels/trycloudflare/):
   `cloudflared tunnel --url http://127.0.0.1:8080`. **Set a key first**, or anyone with the link can use your PC:
   add `"api_key": "some-long-secret"` to `strata-<model>.json` (or set the `STRATA_API_KEY` environment variable);
-  clients then send it as their API key.
+  clients then send it as their API key. Streamed answers carry `X-Accel-Buffering: no`, so nginx-style proxies pass
+  each token on at once. The web app's settings and MCP tools only answer Strata's own page: when you open it through
+  a proxy or tunnel whose address differs, add that address, e.g. `"trusted_origins": ["https://strata.example.com"]`.
+- **From web apps in a browser (CORS).** Off by default. `"cors_origins": ["https://chat.example.com"]` lets pages of
+  those origins call `/v1/*` from the browser (Open WebUI's direct connections, browser extensions); `["*"]` lets any
+  page do it - only sensible with an API key. It never opens `/settings`, `/unload` or the MCP tools.
 
 **Conversation cache.** A request that continues a chat reads only the part after what the engine already holds: the
 live session, or one of the checkpoints it keeps in RAM (up to 6, ~118 MB each, taken at the start of each new
@@ -553,6 +631,29 @@ unmeasured: all the runs above are text.
 
 ---
 
+## Manage Strata from your AI assistant (MCP server)
+
+`tools/strata_mcp.py` is an MCP server for Claude Code, Claude Desktop, Cursor, VS Code, Codex and other assistants.
+Once it is added, you can ask your assistant "install Strata for this PC", "start Strata" or "is Strata running?".
+In Claude Code, add it with:
+
+```bash
+claude mcp add strata -- python C:\Users\you\Strata\tools\strata_mcp.py
+```
+
+It has eight tools: status (the running model, what is installed, the hardware, a recommended size), the model
+list, install, start, stop, logs, a speed test, and connection settings for other apps.
+
+Install runs `setup.py` with `--yes` in the background. Before it downloads anything, it shows the plan and waits
+for your OK. Start and stop work like the run scripts and the server's own unload. The MCP server only ends
+processes it started itself. It uses only Python's standard library, so it works before `.venv` exists.
+
+The config snippets for every client, the tool arguments and the safety rules are in
+[docs/MCP_SERVER.md](MCP_SERVER.md). This is the opposite direction from
+[Tools from MCP servers](#tools-from-mcp-servers) above, where the Strata model calls *your* MCP tools.
+
+---
+
 ## Images (vision)
 
 The model has a vision encoder: [`mmproj-Qwen3.8-Flash-Next-BF16.gguf`](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF)
@@ -568,6 +669,11 @@ helper (`strata-vision`, from llama.cpp's `mtmd` library) and adds it to your st
 A picture becomes up to 1,024 tokens of the context (a 640x480 photo: 300). The same picture sent again, as chat apps
 do on every turn, is encoded only once.
 
+**A spare GPU for the encoder (0.1.33, #408):** with a card the engine doesn't use, add `"cuda_device": 2` (numbered
+like `nvidia-smi`) to the `"vision"` section of `strata-<model>.json`: the encoder then runs on that card alone. Lower
+`--vram-reserve-mib` in `"args"` to 700 as well, so the engine's cards keep that VRAM for the expert cache. The
+encoder's card needs code in the ready-made encoder (RTX 20/30/40/50).
+
 ### Sending a picture
 
 **Terminal chat:** type `/image <path to a picture>`, press Enter, then type your question.
@@ -732,3 +838,56 @@ Strata itself: [MIT](../LICENSE). The model files are not part of it; their lice
   `serve/web/fonts/OFL.txt`). Its Monitor tab started from @code-martin's dashboard idea (PR #22).
 - The experimental speed projection's vector (`data/experimental-speed-projection/`): Qwen Community License 1.0,
   made from the model's activations (see its README).
+
+### Start the text API without occupying the GPU
+
+`serve/server.py --engine strata --config strata-<model>.json --lazy` (or `"lazy_load": true` in that
+config) starts the lightweight HTTP API without spawning the native engine. The first generation request
+loads it through the existing reload path, including `before_load` and `min_free_vram_mib`. Eager startup
+remains the default. This option is text-only: a vision configuration with lazy startup is rejected explicitly.
+
+`POST /v1/load` and `/v1/unload` are JSON control aliases for integrations, accepting `{}` or
+`{"model":"<configured model>"}` and returning model status. They require the configured API key,
+`application/json`, and no foreign browser Origin. They return **409** while a request is active or queued,
+and **404** for an unknown model. Existing `/load` and `/unload` behavior is preserved. `/api/health` aliases
+`/health`; `/v1/status` exposes `loaded` and `auto_load`. The unloaded model remains discoverable.
+
+Unloading and shutdown close the native engine's stdin after sending `QUIT`, allowing Windows' detached
+stdin reader to see EOF. Cleanup waits for process exit before releasing handles; if forced shutdown still
+times out, the server keeps ownership and reports an error rather than claiming the model was unloaded.
+
+### JSON response formats
+
+`POST /v1/chat/completions` accepts `response_format: {"type":"json_object"}` or
+`{"type":"json_schema","json_schema":{"name":"answer","strict":true,"schema":{"type":"object","properties":{"answer":{"type":"integer"}},"required":["answer"],"additionalProperties":false}}}`.
+The schema must describe an object at its root. Local `#` references work; remote references are refused.
+`json_schema` is checked with the Python package `jsonschema` when it is installed (`python -m pip install
+"jsonschema>=4.23,<5"`; setup does not add it); without it the answer is only checked to be one JSON object, and the
+server says so once.
+
+This is **schema prompting followed by server validation**, not grammar-constrained decoding. One generation
+is made per request, with no hidden retry. Successful responses contain a validated JSON object. Malformed JSON,
+duplicate keys, non-finite numbers, schema violations and incomplete generations return **502** with
+`error.code: structured_output_failed`; invalid request schemas return **400**. JSON formats combined with
+tools/MCP are refused explicitly. Without `response_format`, ordinary text and tool behavior stays the same.
+
+Structured SSE buffers the answer while sending keep-alive comments. It emits content only after validation,
+then usage/timings and `[DONE]`; failures emit an SSE error and `[DONE]` without invalid content deltas.
+`/v1/status.structured_output` advertises the formats, validation method and buffered streaming behavior.
+
+### API request monitor
+
+Off by default, since it keeps prompts and answers in memory: turn it on with `"api_monitor": true` in
+`strata-<model>.json` (or `serve/server.py --api-monitor`); otherwise nothing is recorded and the two endpoints below
+answer 404.
+Open `/api-monitor` to inspect API traffic without opening a chat. It shows the model state, safe
+load/unload controls, active/queued requests, original request bodies, output, separate reasoning and
+non-stream response bodies. Total wall-clock includes FIFO waits and automatic loading; load, queue,
+first-token, prompt/output tokens and engine decode timing are shown separately.
+
+`GET /api/requests` returns compact summaries; `GET /api/requests?id=<id>` returns one retained request.
+Both use the existing API-key check. The monitor retains the newest **100 requests in memory** until restart,
+with **262,144 characters per input/output/reasoning/response field** and visible truncation flags. The actual API
+responses are unaffected. Headers are not recorded, and the monitor key is kept in this tab's session storage.
+Treat request history as sensitive input/output when exposing Strata on a network: set an API key as above.
+The page uses relative URLs and works through the existing host binding or a reverse proxy.

docs/HOW_IT_WORKS.md

@@ -0,0 +1,77 @@
+# How does Strata work?
+
+Models like Qwen3.8-Flash-Next normally run on servers with hundreds of gigabytes of graphics memory. Your graphics
+card has 12-24 GB. Strata makes it fit by **sharing the work across your whole PC** - the same idea as a kitchen, where
+the things you use all the time stay on the counter and the rest waits in the pantry. Back to the
+[README](../README.md#how-does-it-work).
+
+## Who does what
+
+<p align="center"><img src="media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>
+
+- **The model is a team of 24,576 small specialists ("experts"),** and each word it writes needs only 10 of them.
+  So it doesn't have to have all of them on the graphics card at once.
+- **Your graphics card** does the part of the work needed for every word, and keeps the few thousand experts that
+  are asked most often. It keeps learning which ones those are while you use it.
+- **Your RAM** holds every expert. When a word needs one the card doesn't have, **your processor** works on it -
+  at the same time as the graphics card, so neither waits for the other.
+- **Your SSD** holds a big lookup table; the model only reads a few small rows of it per word.
+
+The same in technical terms:
+
+- **GPU (VRAM):** attention and DeltaNet mixers, the gated-residual weights, routers, shared experts, output head, the
+  MTP draft layer, the KV cache (from 64K: only its most-read part, the rest streams from RAM), and an **expert
+  cache** that fills the rest of VRAM with the most-used experts (it adapts to the conversation while you chat).
+  More VRAM matters more than a faster GPU: every extra GB holds ~700 more experts, and every expert on the GPU is one
+  the CPU does not have to compute.
+- **RAM:** all 24,576 experts, pinned. The CPU computes the experts that are not on the GPU **in place**, at the same
+  time as the GPU works on the cached ones (AVX-512 / AVX2 kernels, ggml's for the i-quants). On a PC with little RAM
+  and a big card, the [low-RAM mode](MODELS.md#a-big-graphics-card-and-little-ram) keeps in RAM only what the GPU does
+  not hold.
+- **SSD:** the 28.8 GB n-gram table, read a few rows per token through the OS cache.
+
+This works the same on NVIDIA (CUDA) and AMD (HIP) cards: the same engine, compiled for each.
+
+## Guess, then check
+
+<p align="center"><img src="media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>
+
+- **Guess, then check.** A small, fast helper built into the model guesses the next few words, and the big model
+  checks all the guesses in one go. It keeps the ones it agrees with and writes the next word itself - so one step
+  often produces several words. The helper only guesses - the big model decides every word - so you get the same
+  quality answer, 1.6-1.8x sooner.
+- In technical terms: the model's own MTP layer drafts up to 3 tokens; one pass over all 48 layers checks them, 2.4-3.2
+  tokens per pass on average. When the reply repeats the context (code edits, quoted text), **prompt lookup** drafts
+  up to 5 tokens from the earlier copy, but only where its measured acceptance and cost say it pays: code edits 6-11%
+  faster, other text unchanged. The drafts are checked like the MTP's, so the output is the same.
+
+## Reading long texts
+
+**Long texts are read in big pieces** (up to 8,192 tokens - pieces of words - at a time), which is why a long
+document or code base is read at over 1,000 tokens per second. The experts of the next layer stream to the GPU over
+PCIe while the current layer's attention runs. After the first message, Strata keeps the conversation and reads only
+what is new, so follow-ups start in seconds.
+
+## More
+
+The [details](DETAILS.md#how-it-works) explain every part and its numbers, and the
+[paper](paper/Strata-Paper.pdf) tells the whole story, with the measurements behind it.
+
+## Credits
+
+- Model: [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) by the Qwen team; compressed versions by
+  [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF);
+  [Swift 1.5](https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF) by UkisAI; the experimental
+  [UD-Q4_K_XL](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF) by Unsloth (its support follows
+  [eddoursul/Strata](https://github.com/eddoursul/Strata)). Their licenses apply to the model files.
+- Built with parts of [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp) (MIT). Ideas from
+  [Splash](https://github.com/incoai/splash), [ninfer](https://github.com/Neroued/ninfer) and
+  [HyperQwen](https://github.com/syv-ai/HyperQwen). More in the [details](DETAILS.md#credits-and-licenses).
+- The AMD (HIP) backend: [AMD_HIP.md](AMD_HIP.md) names the contributors and the machines it was validated on.
+
+## License
+
+Strata is open source under the [MIT License](../LICENSE). A few parts carry their own licenses: `third_party/ggml`
+(MIT, llama.cpp / ggml), the web app's font (SIL Open Font License 1.1) and the experimental speed projection's
+vector in `data/experimental-speed-projection` (Qwen Community License 1.0, from the model's activations). The
+models are not part of this repository; each model's own license applies to its files.

docs/INSTALL.md

@@ -0,0 +1,187 @@
+# Installing Strata
+
+Everything about installing, starting, updating and configuring Strata, on Windows and Linux, with an NVIDIA or an
+AMD graphics card. The short version is in the [README](../README.md#install); an AI coding assistant can do all of
+this for you with [AI_SETUP.md](AI_SETUP.md).
+
+> **On this page:** [What you need](#what-you-need) · [Windows](#windows) · [Linux](#linux) ·
+> [AMD cards](#amd-cards) · [Several cards](#two-or-three-cards) · [Docker](#docker-linux) · [Updating](#updating) ·
+> [Where things are stored](#where-things-are-stored) · [Setup's questions](#setups-questions) ·
+> [Tuning](#tuning-for-your-pc) · [All options](#options-without-questions)
+
+## What you need
+
+| | |
+| --- | --- |
+| **GPU** | **NVIDIA** RTX 20, 30, 40 or 50 series, **12 GB VRAM or more** (8 GB runs, slowly). Measured on an RTX 5070 and an RTX 3090; RTX 20 (Turing, since 0.1.27) was tested by a contributor on an RTX 2070. **AMD** Radeon RX 7900 XT / XTX, RX 9070 / 9070 XT and Radeon AI PRO R9700 (validated), RX 7800 XT / 7700 XT and RX 9060 XT (validated by their owners), RX 6800 / 6900 series (community-reported), with 12 GB of VRAM or more. See [AMD cards](#amd-cards). |
+| **RAM** | Enough for the size you pick ([which model](MODELS.md#pick-by-ram)); **64 GB** runs every size. A big GPU makes up for less RAM - the [low-RAM mode](MODELS.md#a-big-graphics-card-and-little-ram). |
+| **CPU** | x86-64 with AVX2 (any Intel/AMD desktop CPU from the last ~8 years). AVX-512 (Ryzen 7000/9000) is a bit faster. |
+| **Disk** | ~70-80 GB free for the model, ~6 GB for the MTP layer (+1 GB with images). **Q2_0 on an AVX-512 CPU** also writes a one-time ~40 GB copy of its experts for the fast CPU kernel. On Linux with an AMD card, ROCm takes ~10 GB more when setup installs it. An NVMe SSD is strongly recommended: it makes the first start much faster. |
+| **OS** | Windows 10/11, or Linux (Ubuntu 22.04/24.04 get everything installed automatically). |
+| **Driver** | **NVIDIA:** a current driver, version 580 or newer ([nvidia.com/drivers](https://www.nvidia.com/drivers) or the NVIDIA App). **AMD:** on Linux the kernel's amdgpu driver (no ROCm install needed); on Windows a current AMD Software: Adrenalin Edition driver ([amd.com/support](https://www.amd.com/en/support)). |
+
+The driver is the only thing you install yourself. Everything else - Python, the engine, the model - is set up for
+you the first time: Python 3.12 if you have none (for your user account, no admin), a private Python environment in
+`.venv/`, the Strata engine, the model and the MTP draft layer. On NVIDIA it uses the ready-made engine for RTX
+20/30/40/50 and NVIDIA's CUDA libraries from pip (~0.4 GB); if no ready-made engine fits your PC, it offers to install
+the build tools (Visual Studio Build Tools + CUDA Toolkit on Windows, `build-essential` + CUDA on Ubuntu) and compiles
+the engine for your GPU (asks first; 20-40 minutes once). More: [details](DETAILS.md#before-you-start).
+
+## Windows
+
+1. [Download this project](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) and unzip it (or
+   `git clone` it).
+2. Double-click **`START-HERE.bat`**.
+3. Answer a few questions - or just press Enter each time for the recommended choice ([the questions](#setups-questions)).
+
+Then it downloads everything (the model is ~70 GB, so the first time takes a while - you can stop and it picks up
+where it left off) and **starts the model**. Your browser opens the Strata app at `http://127.0.0.1:8080`.
+
+> **While the model starts, your PC can be slow or stop responding for 1-3 minutes** (longest the first time): Strata
+> loads 35-55 GB into your RAM and locks part of it for the graphics card. That's normal - wait, and don't close the
+> window. The window tells you what it is doing.
+
+**Next time**, just double-click `START-HERE.bat` again: it starts right away (30-90 s to load the model), nothing is
+downloaded twice. Close its window to stop the model. `SETUP.bat` (the same as `START-HERE.bat --setup`) installs
+another model or changes the settings. Starting Strata from Task Scheduler at logon needs two task settings, or the
+start is 24x slower: [Running it at startup](DETAILS.md#running-it-at-startup-task-scheduler).
+
+## Linux
+
+```bash
+./setup.sh
+```
+
+The same questions, the same automatic install (it uses `sudo apt` for Python and, only if it has to compile, for
+the build tools), and the same start: `http://127.0.0.1:8080`. Later runs of `./setup.sh` (or `./run-<model>.sh`)
+start the model directly; `./setup.sh --setup` installs another model or changes the settings. Other distributions,
+WSL and compiling: [details](DETAILS.md#linux).
+
+## AMD cards
+
+The steps are the same as with NVIDIA: `START-HERE.bat` on Windows, `./setup.sh` on Linux. Setup finds the Radeon
+card and chooses the AMD (HIP) engine by itself on a PC with no NVIDIA card Strata can use; `--backend hip` chooses
+it on a PC that has both. Supported cards: RX 7900 XT / XTX (gfx1100), RX 7800 XT / 7700 XT (gfx1101), RX 9060 XT
+(gfx1200), RX 9070 / 9070 XT and Radeon AI PRO R9700 (gfx1201), and the RX 6800 / 6900 series (gfx1030). Integrated
+Radeon GPUs are listed as not supported.
+
+On Linux setup uses a system ROCm 7 when there is one, or installs ROCm into `.venv` from AMD's wheels (~10 GB, no
+sudo), and compiles the engine on your PC for the card (10-20 minutes, once; it needs a C++ compiler and git:
+`sudo apt install build-essential git`). Several AMD cards share the model with `--gpus`, as on NVIDIA.
+
+What differs from NVIDIA for now: pictures are read by the image encoder on the CPU (`--vision cpu`, 10-30 s per
+picture), `--calibrate` is NVIDIA-only, and Unsloth's 4-bit model needs an NVIDIA card. Measurements per card, the
+build by hand and the tuning tables: [AMD_HIP.md](AMD_HIP.md).
+
+## Two or three cards
+
+**Two or three NVIDIA cards?** Just run `START-HERE.bat`: it lists your cards, says which ones Strata can use, and
+asks whether to share the model across them (recommended when two can). An install made on one card asks once at
+its next start. Or choose yourself: `START-HERE.bat --gpus 0,2` (both, remembered), `--gpus all`, or `--gpu 0` (one
+card, this start only). Each card keeps the experts of its own layers, and prompts flow through the cards in a
+pipeline: on an RTX 5080 + RTX 3090 prompts were read 18-20% faster than on the 5080 alone, decoding on par.
+Every card must be an RTX 20 series or newer with 8 GB or more. See [MULTI_GPU.md](MULTI_GPU.md).
+
+**Several AMD cards:** setup takes one card (the one with the most VRAM, or `--gpu N`) unless you name more:
+`--gpus 1,0` splits the model's layers across them, the first one the main card (numbers as setup lists them). A
+split pays only when no single card holds the model's experts ([AMD_HIP.md](AMD_HIP.md#rdna4-gfx1201)).
+
+## Docker (Linux)
+
+The same idea, in a container (NVIDIA cards).
+
+1. Host: Docker with the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
+   and a driver **580 or newer** (CUDA 13.0).
+2. Build (this compiles the engine into the image, so the container never compiles):
+   `docker build -t strata .`
+   `docker build -t strata --build-arg CUDA_ARCHITECTURES=89 .` builds for one card only (faster).
+   The default covers RTX 30 (86), RTX 40 (89), RTX 50 (120) and A-series (80); a card outside that
+   set needs a rebuild with its own arch. Add `--build-arg BUILD_VISION=0` to skip the image encoder.
+3. Run (the first start downloads the ~70 GB model, then starts; later starts go straight to serving):
+   `docker run --rm --gpus all -p 8080:8080 --ulimit memlock=-1 -v strata-data:/data strata`
+
+   The setup choices are env vars: `-e MODEL=IQ2_XS -e FAMILY=qwen -e CONTEXT=32768 -e VISION=no`
+   (or `MODEL=Q2_0|IQ3_XXS|IQ3_S`, `FAMILY=swift|coder`; the defaults above are the recommended ones).
+   `-e VISION=cpu` keeps the image encoder on the CPU. `-e KV=int8|q4_0|k8v4` picks the KV cache
+   precision; `k8v4` is INT8 K with 4-bit V and keeps its KV in VRAM from 64K up.
+   Only the model files, the prepared pack, the MTP layer and the install config live in the
+   `strata-data` volume; the engine is part of the image. Switching between models already on the
+   volume needs no setup pass: `-e MODEL=Q2_0 -e FAMILY=coder` picks that model's config. Add
+   `-e REINSTALL=1` only to change settings for a model already set up (context, vision, KV, host,
+   api_key, LOW_RAM), since those are recorded in its config.
+   Strata loads 32-62 GB into RAM. `--gpus all` on a host with two usable cards takes both: the
+   layer split is setup's recommended default ([MULTI_GPU.md](MULTI_GPU.md)), and a volume
+   set up for one card switches to the pair on its first start there. Pin one card with `-e GPU=0`,
+   or name them with `-e GPUS=0,2` and where the later card's layers start with `-e LAYER_SPLIT=18`.
+   A memory limit needs `-e LOW_RAM=on`, which maps the model's experts from the pack instead of
+   keeping them in RAM: setup.py measures the host's RAM, not the container's limit, so it cannot
+   see a cap. LOW_RAM runs on one card unless `GPUS` names several (then the experts the cards do not hold are
+   read through the OS file cache, which can fill the RAM during long prompts).
+   The server listens on `0.0.0.0:8080` by default; set `-e API_KEY=<secret>` before exposing the port
+   to a network. The image has a `HEALTHCHECK` on `/health`, so `docker ps` shows the container
+   healthy once the model is loaded, and `GET /v1/status` says what it is running.
+
+## Updating
+
+Download the new version and unzip it anywhere (or `git pull`), then run `START-HERE.bat` (Linux: `./setup.sh`) in
+it. The model files are kept in a `Strata-data` folder next to your Strata folder, so a new copy finds them and sets
+itself up the same way - nothing big is downloaded again. On Linux after a `git pull`, setup compiles the engine
+again when its source changed (a minute or two for the changed files); if that compile fails, it says so and starts
+the engine you had.
+
+## Where things are stored
+
+- **Your chats: only in your browser.** The Chat tab keeps the conversation, its settings and the API key you typed
+  in the browser's local storage (`strata.*` keys) - not on the server and not in the Strata folder. Pictures are not
+  kept, only their names. Another browser or a private window starts empty; clearing the site's data deletes them.
+- **How the model starts:** `strata-<model>.json` in the Strata folder (context, GPUs, host, API key, ...), written
+  by setup; next to it `run-<model>.bat` / `.sh`, the log `strata-<model>.log` and, when you use "Use for other
+  apps too", `strata-<model>.shared-settings.json`.
+- **The model files** (`models/`, `packs/`, `mtp/`, 70-120 GB): in **`Strata-data` next to the Strata folder**, or
+  wherever `--data-dir` put them.
+- **Where that data folder is:** `%APPDATA%\Strata\settings.json` on Windows, `~/.config/strata/settings.json` on
+  Linux ([details](DETAILS.md#before-you-start)).
+- **The program itself:** `.venv/`, `engine/` and `third_party/` in the Strata folder.
+
+## Setup's questions
+
+- **Which model and size?** The original, Swift 1.5, the Coder or Unsloth's 4-bit, and Q2_0, IQ2_XS, IQ3_XXS or
+  IQ3_S - see [which model](MODELS.md).
+- **How much context?** How much text it can keep in mind at once (it suggests one for your card). 384K and
+  512K (experimental) extend the model past its trained 262K by rope scaling - the setup turns it on itself (yarn and a
+  covering factor; `--rope-scaling`/`--rope-scale` override) ([details](DETAILS.md): "Context extension past
+  262K").
+- **Images?** Whether it should also read pictures. The image encoder (0.9 GB) runs on the GPU (0.1-0.5 s per
+  picture, ~1.4 GB of VRAM kept free for it) or on the CPU (10-30 s per picture, nothing on the GPU); with AMD cards
+  on the CPU for now. [Details](DETAILS.md#images-vision).
+- **Experimental speed projection?** Off unless you say yes - [read what it does](DETAILS.md#experimental-speed-projection-experimental-off-by-default)
+  first. It changes how the model answers, and only the original model offers it.
+
+With `--yes` setup takes the recommended answer to every question.
+
+## Tuning for your PC
+
+Every PC is different: `START-HERE.bat --calibrate` (Linux: `./setup.sh --calibrate`) measures a few engine settings
+on yours and keeps the fastest (about 5-10 minutes; on an RTX 5070 with a Ryzen 5 7600 it made the Coder 7% faster).
+It keeps a setting only when it is more than 3% faster, and the result is remembered per PC and model, so updates
+keep it. NVIDIA cards for now. [What it measures](DETAILS.md#double-click-start-herebat).
+
+## Options without questions
+
+```
+START-HERE.bat --setup                          install another model, or change context / images
+SETUP.bat                                       the same (double-click it)
+START-HERE.bat --model IQ2_XS --context 32768 --vision yes --yes     no questions
+START-HERE.bat --gguf-dir D:\models\IQ2_XS       use GGUF files you already have
+START-HERE.bat --data-dir E:\Strata-data         keep the model files somewhere else
+START-HERE.bat --port 8081                      another port
+START-HERE.bat --gpu 1                          another GPU (setup picks the one with the most VRAM)
+START-HERE.bat --gpus 0,2                       several GPUs sharing the model
+START-HERE.bat --setup --backend hip            the AMD engine on a PC that also has an NVIDIA card
+START-HERE.bat --setup --host 0.0.0.0 --api-key <secret>     reachable from other devices, with a key
+START-HERE.bat --calibrate                      tune the engine for this PC (about 5-10 minutes), then start
+START-HERE.bat --check                          only check this PC
+```
+
+On Linux the same options go to `./setup.sh`. `START-HERE.bat --help` lists them all. The server's own settings
+(sharing the GPU with games, MCP tools, CORS, API keys, the API itself) are in the [details](DETAILS.md#using-it).

docs/MCP_SERVER.md

@@ -0,0 +1,205 @@
+# Strata's MCP server: let your AI assistant install and run Strata
+
+`tools/strata_mcp.py` is an [MCP](https://modelcontextprotocol.io) server for the AI assistant you already use
+(Claude Code, Claude Desktop, Cursor, VS Code, Codex ...). Add it once, then ask in plain words:
+
+- "Install Strata for this PC."
+- "Start Strata." / "Stop Strata."
+- "Is Strata running? How much VRAM does it use?"
+- "Which Strata model fits my PC?"
+- "Strata didn't start - what does the log say?"
+- "How fast is Strata on this PC?" (a short speed test)
+- "How do I connect Open WebUI / the OpenAI SDK / Claude Code to Strata?"
+
+The assistant calls the server's tools. The tools run Strata's own setup and start the model the way the start
+scripts do. It is one Python file that uses only the standard library, so it works before setup has created
+`.venv`. It needs Python 3.10 or newer, 64-bit on Windows.
+
+It is not the same thing as [Tools from MCP servers](DETAILS.md#tools-from-mcp-servers). That feature
+(`serve/mcp.py`) lets the *Strata model* call tools from your MCP servers in its chat page. This page is the other
+direction: *your assistant* manages Strata.
+
+## Add it
+
+You need a copy of Strata: the zip from GitHub, or `git clone https://github.com/Niko1221/Strata`. Below,
+`C:\Users\you\Strata` (Windows) and `/home/you/Strata` (Linux) stand for that folder. Use `python3` on Linux. On
+Windows, use `py` if `python` is not on your PATH.
+
+### Claude Code
+
+```bash
+claude mcp add strata -- python C:\Users\you\Strata\tools\strata_mcp.py         # Windows
+claude mcp add strata -- python3 /home/you/Strata/tools/strata_mcp.py           # Linux
+```
+
+Add `--scope user` to have it in every project. `/mcp` in Claude Code shows that it is connected.
+
+### Claude Desktop
+
+Open Settings > Developer > Edit Config (`claude_desktop_config.json`) and add:
+
+```json
+{
+  "mcpServers": {
+    "strata": {
+      "command": "python",
+      "args": ["C:\\Users\\you\\Strata\\tools\\strata_mcp.py"]
+    }
+  }
+}
+```
+
+On Linux, use `"command": "python3"` and `"args": ["/home/you/Strata/tools/strata_mcp.py"]`. Restart Claude
+Desktop after the change.
+
+### Cursor
+
+Use `~/.cursor/mcp.json` for every project, or `.cursor/mcp.json` in one project. The format is the same as
+Claude Desktop's:
+
+```json
+{
+  "mcpServers": {
+    "strata": {
+      "command": "python",
+      "args": ["C:\\Users\\you\\Strata\\tools\\strata_mcp.py"]
+    }
+  }
+}
+```
+
+### VS Code (GitHub Copilot agent mode)
+
+Use `.vscode/mcp.json` in a workspace, or run "MCP: Open User Configuration" from the command palette:
+
+```json
+{
+  "servers": {
+    "strata": {
+      "type": "stdio",
+      "command": "python",
+      "args": ["C:\\Users\\you\\Strata\\tools\\strata_mcp.py"]
+    }
+  }
+}
+```
+
+On Linux, use `"command": "python3"` and `"args": ["/home/you/Strata/tools/strata_mcp.py"]`.
+
+### Codex CLI
+
+In `~/.codex/config.toml`:
+
+```toml
+[mcp_servers.strata]
+command = "python"
+args = ['C:\Users\you\Strata\tools\strata_mcp.py']    # Linux: command = "python3", args = ["/home/you/Strata/tools/strata_mcp.py"]
+tool_timeout_sec = 180                                # strata_stop / strata_benchmark can take longer than 60 s
+```
+
+### Any other MCP client
+
+The command is `python <Strata folder>/tools/strata_mcp.py`. The transport is stdio. It speaks MCP revision
+2025-06-18, and also 2025-03-26 and 2024-11-05.
+
+`--root <folder>` manages a Strata folder other than the one the file is in. For example:
+`python D:\tools\strata_mcp.py --root D:\Strata`.
+
+## The tools
+
+| Tool | What it does | Arguments (all optional) |
+| --- | --- | --- |
+| `strata_status` | Whether a model is running: the model, engine version, context, busy or idle, VRAM and RAM in use. What is installed: models, start scripts, engine version, data folder, downloads. An install in progress. This PC's hardware: OS, CPU, RAM, GPUs, page file, free disk. The model and context setup recommends for this PC. | `port`, `hardware` (true/false) |
+| `strata_models` | Every family and size setup offers, with download size, RAM needed, whether it fits this PC, and whether it is installed. Also the context lengths and the recommendation. | - |
+| `strata_install` | Runs setup without questions, in the background (see below). Without `confirm: true` it only returns the plan. A second call shows the progress. `cancel: true` stops the install. | `family` (qwen, swift, coder, unsloth), `model` (Q2_0, IQ2_XS, IQ3_XXS, IQ3_S, IQ1_M, UD-Q4_K_XL), `context` (8192 ... 524288), `vision` (no, yes, gpu, cpu), `kv` (int8, q4_0, k8v4), `gpu` (a number) or `gpus` ("0,2", "all"), `backend` (auto, cuda, hip), `low_ram`, `port`, `data_dir`, `confirm`, `cancel` |
+| `strata_start` | Starts an installed model in the background and waits until it answers. If it is not ready after `wait_seconds` (default 50), it returns "loading". | `model` (an id from `strata_status`, e.g. `q2_0`, `coder-iq1_m`; default: the most recently used), `port`, `gpu`, `wait_seconds` (0-600) |
+| `strata_stop` | Unloads the model through the server's own API, then ends the server it started. | `port`, `force` |
+| `strata_logs` | The last lines of the server log, the engine log or the setup log. | `source` (server, engine, setup), `model`, `lines` (1-500) |
+| `strata_benchmark` | Sends a short fixed request (greedy, thinking off) and reports output and prompt tokens/s. | `port`, `max_tokens` (16-512) |
+| `strata_connect_info` | The OpenAI and Anthropic base URLs, the model id, whether a key is needed, and settings for Claude Code, Codex, the OpenAI SDK, curl, Cursor, Continue and Open WebUI. | `port` |
+
+Every result is JSON with a one-line `summary`. It is sent both as text and as MCP structured content.
+
+## How it works
+
+**Install.** `strata_install` uses setup's own logic and tables (it imports `setup.py`'s `MODELS`, `FAMILIES`,
+`CONTEXTS` and RAM checks) to fill in what you left out:
+
+- the size setup itself would pick: IQ3_XXS from 60 GB of RAM, Q2_0 from about 44 GB, the Coder from about 28 GB,
+  and the Coder in the low-RAM mode below that when the GPU makes up for the RAM;
+- the context setup recommends for the GPU's VRAM;
+- images off.
+
+The first call returns the plan: the download size (58-111 GB), the free disk space, the RAM check, and the exact
+setup command. The assistant should show you the plan and call again with `confirm: true` once you agree.
+
+It then runs the same steps `START-HERE.bat` / `setup.sh` run:
+
+1. It creates `.venv` with the Python it runs on, if `.venv` is missing.
+2. It runs `setup.py --yes --no-start --family ... --model ... --context ... --vision ...` as a background process.
+   The output goes to `.strata-mcp/setup.log` in the Strata folder.
+
+The tool returns at once. Call it again to see the step (1-7), the download percentage, or setup's error message.
+Everything setup has finished is skipped on the next run, and downloads resume. If the AI app closes, the install
+keeps running. The install is refused while a model this server started is running, because setup may replace the
+engine files that model uses.
+
+**Start.** `strata_start` runs the same command as the `run-<model>` script, `serve/server.py --engine strata
+--config strata-<model>.json --port N`, with the venv's Python, without `--open` (no browser) and without a console
+window. The server's output goes to `.strata-mcp/server.log`. The tool then polls `GET /health` until the model
+answers. Loading takes 1-3 minutes. If the server ends while loading, the tool returns the end of its log. The model
+keeps running after the AI app closes, until `strata_stop`. If the AI app runs its tools inside a Windows job that
+does not allow processes to leave it, the model ends together with the app.
+
+Two cases are refused:
+
+- a model is already running: the tool returns it and starts nothing;
+- another program uses the port: the tool says so.
+
+**Stop.** `strata_stop` first sends `POST /unload`. The server sends the engine `QUIT`, and the engine frees its
+VRAM and pinned RAM itself, as when you close the window. Then the server ends:
+
+- Linux: SIGTERM, which runs the server's own Ctrl+C path.
+- Windows: the process ends. Its job object ends the vision encoder and the MCP servers it started.
+
+While a request is running, the tool refuses unless you pass `force: true`.
+
+**Status.** The tool reads `GET /health`, `/v1/status`, `/status` and, on older servers, `/metrics` on 127.0.0.1.
+If a config of the folder sets an `api_key`, the tool sends that key with its own requests. It never shows the key.
+
+## Safety
+
+- **No shell, no free paths.** Every argument is checked against a fixed schema and against setup's own choices. An
+  unknown argument, a model or family setup does not offer, or a `gpus` value other than digits and commas is
+  refused before anything runs. Commands are passed as argument lists; nothing goes through a shell.
+- **Paths.** The only path argument is `data_dir`. It must be one of:
+  - the current Strata data folder;
+  - a folder inside the Strata folder;
+  - a new absolute folder whose name starts with `Strata` (e.g. `D:\Strata-data`), whose parent exists, and which is
+    not in a system folder.
+
+  `..`, relative paths and network (UNC) paths are refused. Logs are read only from `.strata-mcp/` and from the
+  engine log a config names, if that log is inside the Strata folder or the data folder.
+- **Only its own processes.** The server records the process id *and start time* of every process it starts, in
+  `.strata-mcp/*.json`. It only ends a process when both match, so a process id the OS has given to another program
+  is never touched. A Strata started some other way (its window, a run script, a service) is only asked over its
+  API to unload the model. To end that server, close its window.
+- **Big downloads need a yes.** `strata_install` does nothing until it is called with `confirm: true`. The plan
+  shows the download size and the free space first, and the install is refused if the disk is too small.
+- **Local only.** The server checks `127.0.0.1` and never changes the network settings: it does not install with
+  `--host 0.0.0.0` or an API key. To share Strata on your network, run setup yourself (see
+  [Using it](DETAILS.md#using-it)). Experimental options are left off too: the speed projection, rope scaling past
+  262K and `--build`. Run setup by hand for those.
+
+## Troubleshooting
+
+- **The assistant does not see the tools.** Run the command by hand:
+  `python C:\Users\you\Strata\tools\strata_mcp.py`. It should print `[strata-mcp] ready (Strata folder ...)` to
+  stderr and wait for input (Ctrl+C ends it). If it reports `not a Strata folder`, the path is wrong or `--root` is
+  needed. Python older than 3.10 is refused.
+- **The install failed.** Ask the assistant for the setup log (`strata_logs` with `source: setup`), or open
+  `.strata-mcp/setup.log`. Setup's `[X]` line says what to fix. Then install again: every finished step is skipped.
+- **The model does not start.** Get the server log with `strata_logs`, and the engine log with `source: engine`.
+  [Troubleshooting](DETAILS.md#troubleshooting) has the usual causes.
+- **Tests.** `python -m unittest tools/test_strata_mcp.py` runs the tests with a fake setup and server. They do not
+  download anything or use the GPU.

docs/MODELS.md

@@ -0,0 +1,139 @@
+# Which model? Sizes, versions and what fits
+
+Strata runs one model, [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), in several **sizes**
+(the same model, compressed more or less) and several **versions** (the original, a coding version, a fine-tune).
+The installer recommends one for your PC; this page explains the choice. Back to the [README](../README.md).
+
+> **On this page:** [Pick by RAM](#pick-by-ram) · [Speed](#how-fast-is-each-size) · [The sizes](#the-sizes) ·
+> [Will it fit?](#will-it-fit) · [The versions](#the-versions) · [Adding another model](#adding-or-switching-models)
+
+## Pick by RAM
+
+| Your RAM | Take | Why |
+| --- | --- | --- |
+| **32 GB** | **Coder** | it fits 32 GB, and it is made for code (with a 24 GB card, Q2_0 and IQ2_XS run too: [low-RAM mode](#a-big-graphics-card-and-little-ram)) |
+| **48 GB** | **IQ2_XS** (or Q2_0, the fastest) | the larger sizes do not fit |
+| **64 GB** | **IQ2_XS** (recommended), or IQ3_XXS / IQ3_S | every size fits (IQ3_S with little else open) |
+| **96 GB or more** | **IQ3_S**, or [Unsloth's 4-bit](#unsloth-ud-q4_k_xl-experimental) (experimental) | room for the largest sizes |
+
+Not sure? Take **IQ2_XS** - or the **Coder** if you mainly write code, or have 32-48 GB of RAM.
+
+## How fast is each size
+
+Measured on an RTX 5070 (12 GB), a Ryzen 5 7600 and 64 GB of RAM:
+
+| Size | Writes answers (short chat) | Writes answers (128K context) | Reads your prompt |
+| --- | ---: | ---: | ---: |
+| **Q2_0** | 93 tokens/s | 74 tokens/s | 2,170 tokens/s |
+| **IQ2_XS** | 79 tokens/s | 63 tokens/s | 2,090 tokens/s |
+| **IQ3_XXS** | 62 tokens/s | 49 tokens/s | 1,750 tokens/s |
+| **IQ3_S** | 53 tokens/s | 46 tokens/s | 1,620 tokens/s |
+| **Coder** (IQ1_M) | 55 tokens/s | 43 tokens/s | 2,180 tokens/s |
+
+Measured on an AMD RX 9070 XT (16 GB), a Ryzen 9 3900X and 47 GB of RAM (Linux, setup's own install; Q2_0 is
+above setup's RAM estimate for 47 GB and was installed with `--model Q2_0 --yes`):
+
+| Size | Writes answers (short chat) | Writes answers (128K context) | Reads your prompt |
+| --- | ---: | ---: | ---: |
+| **Q2_0** | 60 tokens/s | 48 tokens/s | 1,160 tokens/s |
+| **IQ2_XS** | 52 tokens/s | 36 tokens/s | 1,110 tokens/s |
+| **Coder** (IQ1_M) | 44 tokens/s | 33 tokens/s | 1,420 tokens/s |
+
+- **Writes answers** = how fast the reply appears (tokens per second; a token is about ¾ of a word).
+- **Reads your prompt** = how fast it takes in what you send (long documents, code, chat history), measured on a
+  32K-token prompt; a 4K prompt reads at 910-1,580 tokens/s. A 32K prompt takes about 15 seconds with Q2_0.
+
+A card with more VRAM is faster, because more of the model fits on the GPU: an RTX 3090 (24 GB) should do roughly
+100-140 tokens per second. All measurements, long-context numbers and estimates for other cards are in the
+[details](DETAILS.md#speed-measured). The AMD measurements per card (RX 9070 XT, Radeon AI PRO R9700, RX 7800 XT,
+RX 9060 XT, RX 6900 XT) are in [AMD_HIP.md](AMD_HIP.md#rdna4-gfx1201).
+
+Every PC is different: `START-HERE.bat --calibrate` measures a few engine settings on yours and keeps the fastest
+(about 5-10 minutes; on the PC above it made the Coder 7% faster; NVIDIA cards for now). Measured Strata on your own
+PC? See [Community benchmark results](COMMUNITY_BENCHMARKS.md) for a report template and how to share your results
+in a pull request.
+
+## The sizes
+
+| Model | RAM+VRAM Requirements | Speed | Quality |
+| --- | ---: | --- | --- |
+| **Q2_0** | 37.6 GB | fastest | good |
+| **IQ2_XS** | 39.2 GB | fast | better (**recommended**) |
+| **IQ3_XXS** | 47.0 GB | slower | great |
+| **IQ3_S** | 54.8 GB | slowest | best: matches the full model on the published tests (original model only) |
+
+The download is 66-76 GB for the three smaller sizes ([details](DETAILS.md#which-model)); the first start also
+fetches the MTP draft layer (~6 GB, +1 GB with images).
+
+## Will it fit?
+
+Shard 1 is the part of the model that gets loaded when it starts: its experts go into your **RAM**, the rest onto
+your graphics card (the second shard, a 29 GB lookup table, stays on the SSD). So it fits when your **RAM is at least
+shard 1 + about 10 GB** for Windows and your other programs. With 64 GB of RAM every size fits (IQ3_S with little
+else open); with 48 GB, Q2_0 and IQ2_XS. A bigger graphics card makes it faster, but it doesn't lower the RAM needed
+- except in the low-RAM mode below.
+
+### A big graphics card and little RAM
+
+On a PC whose RAM cannot hold the experts beside the system, setup maps them from the model's files instead and keeps
+in RAM only what the graphics card does not hold (the [low-RAM mode](DETAILS.md#speed-measured), chosen by setup). For
+example, a 32 GB PC with a 24 GB GPU runs Q2_0, IQ2_XS and the Coder this way, and a 32 GB PC with a 12-16 GB GPU the
+Coder. With a small card most experts then come from the SSD and it is much slower (setup says so).
+`START-HERE.bat --setup --low-ram on|off` overrides the choice.
+
+## The versions
+
+### Qwen3.8-Flash-Next
+
+The original, in all four sizes. With images, and with the
+[experimental speed projection](DETAILS.md#experimental-speed-projection-experimental-off-by-default) as an option.
+
+### Coder
+
+**[Coder](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF)** - ISTA-DASLab's coding version:
+half of the experts removed, keeping the ones that code, tool use and images need (91% of the full model's SWE-bench
+Verified score, 99% of LiveCodeBench, by its authors). One size (IQ1_M: its experts stored like IQ3_S): shard 1 is
+**29.6 GB**, so it fits a PC with **32 GB of RAM**, runs 262K context on 64 GB, and reads long prompts the fastest of
+all. It is weaker outside code, and that includes Chinese and other CJK text (#438: Chinese answers came out wrong
+or looping where English was fine). For general chat or CJK text, take a size that keeps every expert: Q2_0,
+IQ2_XS or IQ3_S.
+More: [details](DETAILS.md#or-the-coder-half-the-experts-for-code).
+
+```
+START-HERE.bat --setup --family coder
+```
+
+### Swift 1.5
+
+**[Swift 1.5](https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF)** - a fine-tune by UkisAI that
+thinks much shorter before answering, so you get the answer sooner, with about the same quality. Same speed per
+token, and about the same RAM as the same size of the original (no IQ3_S). Its own license applies (see its page).
+More: [details](DETAILS.md#or-swift-15-a-fine-tune-that-thinks-shorter).
+
+```
+START-HERE.bat --setup --family swift --model IQ2_XS
+```
+
+### Unsloth UD-Q4_K_XL (experimental)
+
+**Unsloth's 4-bit UD-Q4_K_XL** (experimental) is the fourth version in setup's menu (`--family unsloth`): the closest
+to the full model, but a 111 GB download whose 77 GB of experts do not fit in RAM. Strata keeps your RAM minus 24 GB
+of them in RAM and reads the rest from the SSD while it answers: 7-8.5 tokens/s on a 64 GB PC with a 12 GB GPU, several
+times slower than the sizes above, and long prompts are slow. It needs 48 GB of RAM or more, an NVMe SSD and one
+NVIDIA GPU (no images yet). Details and measurements: [UD-Q4_K_XL](UNSLOTH_Q4.md).
+
+```
+START-HERE.bat --setup --family unsloth --model UD-Q4_K_XL
+```
+
+### OrcaRouter Uncensored IQ3_XXS
+
+For **OrcaRouter's Flash-Next Uncensored IQ3_XXS**, see the [manual compatibility setup](ORCA.md). It needs an
+explicit packing conversion and is not an installer menu option.
+
+## Adding or switching models
+
+You can add another model any time with `SETUP.bat` (the same as `START-HERE.bat --setup`; on Linux
+`./setup.sh --setup`). Files another model shares are not downloaded again (the Coder uses the original's shard 2 and
+vision encoder). With more than one model installed, `START-HERE.bat` asks which one to start; `run-<model>.bat`
+(Linux: `run-<model>.sh`) starts one directly.

docs/MULTI_GPU.md

@@ -43,9 +43,10 @@ now on; the answer is kept.
 **Not supported** (setup says so and names the cards that can be used instead):
 - a card older than the RTX 20 series (compute capability below 7.5: GTX 10 and older);
 - a card with less than 8 GB of VRAM, together with others (each card holds a copy of the dense weights and its
-  own prompt buffers);
-- AMD and Intel GPUs, and a mix of NVIDIA with them. (Two AMD RDNA4 cards run the split from a hand-written
-  config; see [AMD_HIP.md](AMD_HIP.md#rdna4-gfx1201).)
+  own prompt buffers) - unless you name it with `--gpus`: then setup says the risk and asks (`--yes` with the named
+  cards goes ahead);
+- Intel GPUs, and a mix of NVIDIA and AMD cards. (AMD cards share a model among themselves: `./setup.sh --backend
+  hip --gpus 1,0`, see [AMD_HIP.md](AMD_HIP.md).)
 
 Or edit an existing config (`strata-*.json`), then restart:
 
@@ -54,6 +55,21 @@ Or edit an existing config (`strata-*.json`), then restart:
 "layer_split": "auto"
 ```
 
+**Skip the split when the first card holds everything** (opt-in, 0.1.31): `"split_skip_if_fits": true` in the config
+(engine flag `--split-skip-if-fits`, with `--layer-split auto`) runs on the first card alone when it holds every
+profiled expert plus the context's KV, the draft layer and the reserve, and says so in the log; otherwise the split
+stays. On an R9700 32 GB + RX 9070 XT the R9700 holds all of the Coder's experts: with the flag 4K prompts read at
+1,776 tok/s instead of 1,244 (split) and decode runs at ~60 tok/s instead of ~51 (16K prompts ~5% slower than split).
+
+**Short prompts on a split (0.1.32, #340).** 0.1.30 gave each card's prompt path a loan from its own expert cache,
+refilled after every request; on cards that hold nearly all their experts that cost short prompts up to a third of
+their speed. 0.1.32 refills all cards at once, uses a smaller streaming ring on a split, and lets a card with free
+VRAM keep its own prompt buffers - the same output as 0.1.31, measured on an R9700 + RX 9070 XT: 2K prompts 993 ->
+1,265 tok/s, 16K 1,852 -> 1,950, decode unchanged. `STRATA_SPLIT_OWN=1` (opt-in) gives every card its own buffers:
+2K 1,450 and 16K 2,227 tok/s there, but a full card then keeps a different set of experts resident, so the output
+differs from the default's (stable and coherent); `STRATA_SPLIT_OWN=auto` does that only where the buffers are at
+most 12% of each card's VRAM.
+
 The engine flags behind it: `--layer-split K1[,K2..]|auto` and `--split-device D1[,D2..]` (the later stages'
 devices; default the next visible ones). `--layer-split K --split-device 0` runs both stages on one card sharing
 everything - the bit-exact check of the hand-off, not a speed mode.
@@ -95,8 +111,10 @@ into the card that owns the layer.
 - The prompt path has its own buffers on every card (1.5 GB each at the default 2048-token chunk; `--prefill 1024`
   halves that) instead of borrowing cache slots as one card does. An explicit `--expert-cache` on the first card is
   capped to leave room for them.
-- On Windows only 8 GiB of the expert arena is pinned (more, mapped into two GPU contexts, leaves WDDM refusing
-  allocations); the PCIe share covers those layers.
+- Under WDDM (Windows, and WSL2) only 8 GiB of the expert arena is pinned (more, mapped into two GPU contexts,
+  leaves WDDM refusing allocations); the rest streams through the pinned staging ring. A Linux driver has no such
+  limit, so there the whole arena is pinned (since 0.1.31; the cap cost a 4090 + 3060 split two thirds of its
+  prompt speed, #253). `STRATA_ARENA_PIN_GIB=N` pins at most N GiB, `0` the whole arena, on any OS.
 - Every card needs compute capability 7.5 (RTX 20 or newer). The pre-sm_80 QSA scorer path is fp32 FMAs, so a
   Turing card runs the same kernels instead of the tensor-core prompt attention.
 

docs/ORCA_Q4_K_S.md

@@ -0,0 +1,36 @@
+# OrcaRouter Uncensored Q4_K_S (manual setup)
+
+This path uses the three original `orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF` Q4_K_S shards without rewriting their expert, embedding or PLE table bytes. It is separate from the installer model menu. Keep all three shard filenames together.
+
+The ordinary GGUF quantizes some small projections which Strata reads as BF16. `--compat-bf16` dequantizes and rounds those projections to BF16 in the separate pack; it does not restore their original precision. The Q5_0 PLE table remains in the source GGUF and currently uses the mapped reader (`--ple-io mmap`). The Q4_K gate/up experts and Q5_0/Q5_1 down experts remain in GGUF form, including layers whose three tensors cross shard boundaries (each role is read from its own shard). The GPU expert kernels take Q4_K with Q5_1 (or Q8_0) downs; Q5_0 downs are not among them yet, and the engine refuses to start on a pack with such a layer (it names the layer).
+
+## Build and prepare
+
+Build Strata with CUDA, native experts, `STRATA_ORCA_Q4KS_MMQ=ON` (and `STRATA_BUILD_TESTS=ON` for the two parity programs below). This opt-in compiles the Q5_0 GGML MMQ instance for CUDA or HIP (the Q4_K and Q5_1 instances come with the engine's K-quant MMQ build on CUDA); the usual Q8_0 draft-layer instance remains in both builds. HIP also needs `STRATA_PREFILL_MMQ=ON`. Select your card's compute capability, for example `89` on an L40S. On Windows with Ninja and CUDA 12.6, use the Visual Studio developer environment and `-DCMAKE_CUDA_RUNTIME_LIBRARY=Shared` to keep the CUDA runtime linkage consistent. The default pinned ggml checkout is supported by the project build.
+
+```sh
+cmake -S . -B build -G Ninja -DSTRATA_ENABLE_CUDA=ON -DSTRATA_NATIVE_EXPERTS=ON -DSTRATA_ORCA_Q4KS_MMQ=ON -DCMAKE_CUDA_ARCHITECTURES=89 -DCMAKE_CUDA_RUNTIME_LIBRARY=Shared -DCMAKE_BUILD_TYPE=Release
+cmake --build build --target strata native_expert_parity ple_q5_parity
+python tools/iq_pack.py --gguf /models/Qwen3.8-Flash-Next-Uncensored-Q4_K_S-00001-of-00003.gguf --out packs/orca-q4_k_s --compat-bf16
+```
+
+`iq_pack.py` needs NumPy and the `gguf-py` module from the pinned llama.cpp dependency (`STRATA_GGUF_PY` can point to that directory). It writes a roughly 1.43 GiB dense pack and a native-expert manifest. The original 112 GB of model shards are read in place; `experts.bin` is optional and is not written by this command.
+
+Check the changed kernels against the real files before starting the service:
+
+```sh
+build/native_expert_parity /models/Qwen3.8-Flash-Next-Uncensored-Q4_K_S-00001-of-00003.gguf /models/Qwen3.8-Flash-Next-Uncensored-Q4_K_S-00002-of-00003.gguf /models/Qwen3.8-Flash-Next-Uncensored-Q4_K_S-00003-of-00003.gguf 0 5 6 20 47
+build/ple_q5_parity /models/Qwen3.8-Flash-Next-Uncensored-Q4_K_S-00001-of-00003.gguf
+```
+
+## Engine flags
+
+Pass the same first shard to `--native` and `--ple-gguf`; Strata discovers the other two shards by name. For a first text-only run, use 32K context and a modest cache. Once that passes, increase the context and cache with memory measurements on the target machine.
+
+```sh
+--pack packs/orca-q4_k_s --native /models/Qwen3.8-Flash-Next-Uncensored-Q4_K_S-00001-of-00003.gguf --ple-gguf /models/Qwen3.8-Flash-Next-Uncensored-Q4_K_S-00001-of-00003.gguf --ple-io mmap --expert-profile data/expert-profile.bin --expert-cache auto --prefill 512 --spec 4 --mtp /path/to/mtp/rt --max-context 32768 --kv int8
+```
+
+For two GPUs in persistent server mode, add `--layer-split auto --split-device 1` (with CUDA0 and CUDA1 visible). The usual Strata MTP runtime is built from the original model's draft layer; draft acceptance on the Uncensored fine-tune must be measured. Keep this model's own tokenizer from the pack. Do not combine `--expert-cache-per-layer` with a variable-size native cache: that policy's slot ordering is independent of the pack's per-layer blob sizes.
+
+This port does not claim that Q4_K_S beats llama.cpp on a given computer. Compare the same model bytes, prompt, context, output length and GPU load; report prefill and decode separately.