# the change with the GEMM route on, the same switches and requests
strata serve: prompt chunk auto: 8192 tokens, a 384-slot ring
strata prefill timing: 560 tokens, GPU timeline 1904 ms, wall 1911 ms, host staging 11 ms: embed+steps 4 (0.2%) hc read 175 (9.2%) gdn 257 (13.5%) qsa proj 116 (6.1%) qsa indexer 0 (0.0%) qsa select 1 (0.0%) qsa attn 31 (1.6%) router+shared 70 (3.7%) host grouping 147 (7.7%) gather 9 (0.4%) wait copy 42 (2.2%) dequant 139 (7.3%) gemm gate/up 524 (27.5%) gemm down 160 (8.4%) combine 77 (4.0%) ple 6 (0.3%) kv stage 0 (0.0%) gdn conv+gates 9 (0.5%) gdn recurrence 34 (1.8%) gdn out proj 103 (5.4%)
strata prefill timing: host: chunk setup (PLE rows, the expert stream plan) 4 ms, waiting for each chunk 12 ms, after each chunk (the draft layer, progress) 7 ms, PLE 73 ms
strata decode timing: 110 windows, avg T 2.14, 1.82 tokens/window, 54.70 ms/window = verify 47.64 (GPU-reach wait 32.24 + per-layer host 9.72 [plan 0.22 actq 0.24 jobs 0.07 CPU 8.81] + stage 0.35) + commit/emit 0.39 + draft 4.08; per layer-window: CPU experts 1.77 (2.32 entries), VRAM hits 17.56, PCIe 0.00
strata decode GPU stages (ms/window): GDN layers: q8+qkv/q-idx gemv 3.86 conv 0.21 ab 0.36 z 2.21 rec 0.91 out-proj 2.75 router+plan+bell 1.52 shared+quant 2.23 waitA 1.37 VRAM hits 6.39 waitB 0.32 PCIe grp 0.30 waitCPU 2.98 copy+combine 0.56 (gap) 0.32 head 2.45   hc0 norm 0.65   hc0 down 1.44   hc0 up 1.29   hc1 norm 0.34   hc1 down 1.40   hc1 up 1.38 | QSA layers: q8+kv-idx 0.12 k/v+norm-rope 0.37 kv+idx append 0.17 q+q-idx 1.59 scores+topk 0.11 kv-resolve 0.17 attention 0.58 gate 0.05 out-proj 0.97 router+plan+bell 0.51 shared+quant 0.73 waitA 0.12 VRAM hits 2.09 waitB 0.10 PCIe grp 0.10 waitCPU 1.40 copy+combine 0.19 (gap) 0.10   hc0 norm 0.10   hc0 down 0.48   hc0 up 0.43   hc1 norm 0.11   hc1 down 0.47   hc1 up 0.46 | total 46.77 ms/window over 111 windows
strata serve: prompt 567 tokens = 0 reused + 567 read in 2157 ms (262.9 tok/s), 200 generated in 6017 ms (33.2 tok/s), drafts accepted 91 of 125, 1 checkpoints
strata prefill timing: 559 tokens, GPU timeline 1971 ms, wall 1977 ms, host staging 28 ms: embed+steps 0 (0.0%) hc read 175 (8.9%) gdn 256 (13.0%) qsa proj 113 (5.7%) qsa indexer 0 (0.0%) qsa select 1 (0.0%) qsa attn 31 (1.6%) router+shared 69 (3.5%) host grouping 134 (6.8%) gather 9 (0.4%) wait copy 53 (2.7%) dequant 138 (7.0%) gemm gate/up 497 (25.2%) gemm down 161 (8.2%) combine 184 (9.3%) ple 6 (0.3%) kv stage 0 (0.0%) gdn conv+gates 9 (0.4%) gdn recurrence 34 (1.7%) gdn out proj 101 (5.1%)
strata prefill timing: host: chunk setup (PLE rows, the expert stream plan) 0 ms, waiting for each chunk 11 ms, after each chunk (the draft layer, progress) 6 ms, PLE 73 ms
strata decode timing: 112 windows, avg T 2.21, 1.79 tokens/window, 50.26 ms/window = verify 45.34 (GPU-reach wait 37.54 + per-layer host 3.39 [plan 0.20 actq 0.19 jobs 0.03 CPU 2.15] + stage 0.80) + commit/emit 0.36 + draft 4.08; per layer-window: CPU experts 0.44 (0.51 entries), VRAM hits 21.02, PCIe 0.00
strata decode GPU stages (ms/window): GDN layers: q8+qkv/q-idx gemv 3.87 conv 0.21 ab 0.37 z 2.22 rec 0.91 out-proj 2.78 router+plan+bell 1.51 shared+quant 2.24 waitA 1.75 VRAM hits 7.48 waitB 0.32 PCIe grp 0.30 waitCPU 1.10 copy+combine 0.49 (gap) 0.32 head 2.45   hc0 norm 0.66   hc0 down 1.45   hc0 up 1.30   hc1 norm 0.34   hc1 down 1.41   hc1 up 1.40 | QSA layers: q8+kv-idx 0.13 k/v+norm-rope 0.37 kv+idx append 0.17 q+q-idx 1.60 scores+topk 0.11 kv-resolve 0.17 attention 0.58 gate 0.05 out-proj 0.98 router+plan+bell 0.50 shared+quant 0.74 waitA 0.18 VRAM hits 2.46 waitB 0.11 PCIe grp 0.10 waitCPU 0.11 copy+combine 0.16 (gap) 0.10   hc0 norm 0.10   hc0 down 0.48   hc0 up 0.44   hc1 norm 0.11   hc1 down 0.47   hc1 up 0.47 | total 45.58 ms/window over 113 windows
strata serve: prompt 566 tokens = 0 reused + 566 read in 2258 ms (250.7 tok/s), 200 generated in 5629 ms (35.5 tok/s), drafts accepted 90 of 135, 1 checkpoints
strata prefill timing: 560 tokens, GPU timeline 1799 ms, wall 1805 ms, host staging 43 ms: embed+steps 0 (0.0%) hc read 175 (9.7%) gdn 257 (14.3%) qsa proj 112 (6.2%) qsa indexer 0 (0.0%) qsa select 1 (0.0%) qsa attn 31 (1.7%) router+shared 68 (3.8%) host grouping 142 (7.9%) gather 9 (0.5%) wait copy 46 (2.6%) dequant 133 (7.4%) gemm gate/up 477 (26.5%) gemm down 157 (8.7%) combine 39 (2.2%) ple 6 (0.4%) kv stage 0 (0.0%) gdn conv+gates 9 (0.5%) gdn recurrence 34 (1.9%) gdn out proj 103 (5.7%)
strata prefill timing: host: chunk setup (PLE rows, the expert stream plan) 0 ms, waiting for each chunk 12 ms, after each chunk (the draft layer, progress) 5 ms, PLE 73 ms
strata decode timing: 109 windows, avg T 2.24, 1.83 tokens/window, 49.29 ms/window = verify 44.71 (GPU-reach wait 38.14 + per-layer host 2.30 [plan 0.20 actq 0.12 jobs 0.02 CPU 1.39] + stage 0.55) + commit/emit 0.28 + draft 4.03; per layer-window: CPU experts 0.26 (0.30 entries), VRAM hits 21.71, PCIe 0.00
strata decode GPU stages (ms/window): GDN layers: q8+qkv/q-idx gemv 3.89 conv 0.21 ab 0.37 z 2.23 rec 0.92 out-proj 2.80 router+plan+bell 1.50 shared+quant 2.25 waitA 1.20 VRAM hits 7.69 waitB 0.32 PCIe grp 0.30 waitCPU 0.73 copy+combine 0.47 (gap) 0.32 head 2.45   hc0 norm 0.66   hc0 down 1.45   hc0 up 1.31   hc1 norm 0.34   hc1 down 1.42   hc1 up 1.41 | QSA layers: q8+kv-idx 0.13 k/v+norm-rope 0.38 kv+idx append 0.17 q+q-idx 1.60 scores+topk 0.11 kv-resolve 0.17 attention 0.58 gate 0.05 out-proj 0.99 router+plan+bell 0.50 shared+quant 0.74 waitA 0.13 VRAM hits 2.52 waitB 0.11 PCIe grp 0.10 waitCPU 0.10 copy+combine 0.15 (gap) 0.10   hc0 norm 0.10   hc0 down 0.49   hc0 up 0.44   hc1 norm 0.11   hc1 down 0.48   hc1 up 0.47 | total 44.96 ms/window over 110 windows
strata serve: prompt 567 tokens = 0 reused + 567 read in 2101 ms (269.9 tok/s), 200 generated in 5373 ms (37.2 tok/s), drafts accepted 92 of 135, 1 checkpoints
strata prefill timing: 7042 tokens, GPU timeline 14615 ms, wall 14679 ms, host staging 95 ms: embed+steps 8 (0.1%) hc read 2029 (13.9%) gdn 2447 (16.7%) qsa proj 1139 (7.8%) qsa indexer 0 (0.0%) qsa select 112 (0.8%) qsa attn 1767 (12.1%) router+shared 530 (3.6%) host grouping 41 (0.3%) gather 147 (1.0%) wait copy 21 (0.1%) dequant 307 (2.1%) gemm gate/up 2515 (17.2%) gemm down 1162 (8.0%) combine 670 (4.6%) ple 67 (0.5%) kv stage 0 (0.0%) gdn conv+gates 106 (0.7%) gdn recurrence 415 (2.8%) gdn out proj 1131 (7.7%)
strata prefill timing: host: chunk setup (PLE rows, the expert stream plan) 7 ms, waiting for each chunk 71 ms, after each chunk (the draft layer, progress) 63 ms, PLE 475 ms
strata decode timing: 37 windows, avg T 2.03, 1.62 tokens/window, 48.39 ms/window = verify 44.15 (GPU-reach wait 38.54 + per-layer host 1.57 [plan 0.15 actq 0.06 jobs 0.02 CPU 0.97] + stage 0.36) + commit/emit 0.22 + draft 3.74; per layer-window: CPU experts 0.18 (0.19 entries), VRAM hits 19.79, PCIe 0.00
strata decode GPU stages (ms/window): GDN layers: q8+qkv/q-idx gemv 3.88 conv 0.21 ab 0.36 z 2.23 rec 0.90 out-proj 2.75 router+plan+bell 1.49 shared+quant 2.22 waitA 1.02 VRAM hits 7.42 waitB 0.32 PCIe grp 0.30 waitCPU 0.53 copy+combine 0.46 (gap) 0.32 head 2.48   hc0 norm 0.65   hc0 down 1.45   hc0 up 1.30   hc1 norm 0.34   hc1 down 1.41   hc1 up 1.39 | QSA layers: q8+kv-idx 0.13 k/v+norm-rope 0.37 kv+idx append 0.18 q+q-idx 1.61 scores+topk 0.64 kv-resolve 0.43 attention 1.02 gate 0.05 out-proj 0.96 router+plan+bell 0.50 shared+quant 0.73 waitA 0.12 VRAM hits 2.44 waitB 0.11 PCIe grp 0.10 waitCPU 0.07 copy+combine 0.15 (gap) 0.10   hc0 norm 0.10   hc0 down 0.49   hc0 up 0.44   hc1 norm 0.11   hc1 down 0.47   hc1 up 0.46 | total 45.19 ms/window over 38 windows
strata serve: prompt 7049 tokens = 0 reused + 7049 read in 15293 ms (460.9 tok/s), 60 generated in 1790 ms (33.5 tok/s), drafts accepted 23 of 38, 1 checkpoints
strata prefill timing: 7042 tokens, GPU timeline 14264 ms, wall 14328 ms, host staging 143 ms: embed+steps 6 (0.0%) hc read 1973 (13.8%) gdn 2450 (17.2%) qsa proj 1153 (8.1%) qsa indexer 0 (0.0%) qsa select 112 (0.8%) qsa attn 1770 (12.4%) router+shared 532 (3.7%) host grouping 40 (0.3%) gather 147 (1.0%) wait copy 24 (0.2%) dequant 306 (2.1%) gemm gate/up 2496 (17.5%) gemm down 1165 (8.2%) combine 362 (2.5%) ple 68 (0.5%) kv stage 0 (0.0%) gdn conv+gates 105 (0.7%) gdn recurrence 417 (2.9%) gdn out proj 1138 (8.0%)
strata prefill timing: host: chunk setup (PLE rows, the expert stream plan) 6 ms, waiting for each chunk 71 ms, after each chunk (the draft layer, progress) 63 ms, PLE 475 ms
strata decode timing: 36 windows, avg T 2.00, 1.67 tokens/window, 48.02 ms/window = verify 44.10 (GPU-reach wait 38.67 + per-layer host 1.41 [plan 0.14 actq 0.06 jobs 0.02 CPU 0.95] + stage 0.23) + commit/emit 0.16 + draft 3.73; per layer-window: CPU experts 0.17 (0.17 entries), VRAM hits 19.58, PCIe 0.00
strata decode GPU stages (ms/window): GDN layers: q8+qkv/q-idx gemv 3.88 conv 0.21 ab 0.36 z 2.23 rec 0.90 out-proj 2.74 router+plan+bell 1.49 shared+quant 2.22 waitA 1.02 VRAM hits 7.51 waitB 0.32 PCIe grp 0.30 waitCPU 0.42 copy+combine 0.45 (gap) 0.32 head 2.49   hc0 norm 0.65   hc0 down 1.45   hc0 up 1.30   hc1 norm 0.34   hc1 down 1.41   hc1 up 1.38 | QSA layers: q8+kv-idx 0.13 k/v+norm-rope 0.36 kv+idx append 0.18 q+q-idx 1.61 scores+topk 0.64 kv-resolve 0.44 attention 1.02 gate 0.05 out-proj 0.96 router+plan+bell 0.50 shared+quant 0.73 waitA 0.12 VRAM hits 2.48 waitB 0.11 PCIe grp 0.10 waitCPU 0.08 copy+combine 0.15 (gap) 0.10   hc0 norm 0.10   hc0 down 0.48   hc0 up 0.44   hc1 norm 0.11   hc1 down 0.47   hc1 up 0.46 | total 45.19 ms/window over 37 windows
strata serve: prompt 7049 tokens = 0 reused + 7049 read in 14979 ms (470.6 tok/s), 60 generated in 1729 ms (34.7 tok/s), drafts accepted 24 of 36, 1 checkpoints
strata prefill timing: 7039 tokens, GPU timeline 14315 ms, wall 14379 ms, host staging 193 ms: embed+steps 6 (0.0%) hc read 1940 (13.5%) gdn 2449 (17.1%) qsa proj 1154 (8.1%) qsa indexer 0 (0.0%) qsa select 113 (0.8%) qsa attn 1776 (12.4%) router+shared 548 (3.8%) host grouping 41 (0.3%) gather 147 (1.0%) wait copy 21 (0.1%) dequant 311 (2.2%) gemm gate/up 2546 (17.8%) gemm down 1167 (8.2%) combine 361 (2.5%) ple 66 (0.5%) kv stage 0 (0.0%) gdn conv+gates 105 (0.7%) gdn recurrence 419 (2.9%) gdn out proj 1146 (8.0%)
strata prefill timing: host: chunk setup (PLE rows, the expert stream plan) 6 ms, waiting for each chunk 71 ms, after each chunk (the draft layer, progress) 63 ms, PLE 475 ms
strata decode timing: 37 windows, avg T 1.95, 1.62 tokens/window, 47.57 ms/window = verify 43.70 (GPU-reach wait 38.33 + per-layer host 1.22 [plan 0.14 actq 0.05 jobs 0.02 CPU 0.97] + stage 0.01) + commit/emit 0.18 + draft 3.68; per layer-window: CPU experts 0.16 (0.16 entries), VRAM hits 19.07, PCIe 0.00
strata decode GPU stages (ms/window): GDN layers: q8+qkv/q-idx gemv 3.86 conv 0.21 ab 0.36 z 2.22 rec 0.89 out-proj 2.73 router+plan+bell 1.48 shared+quant 2.20 waitA 1.18 VRAM hits 7.37 waitB 0.32 PCIe grp 0.30 waitCPU 0.33 copy+combine 0.45 (gap) 0.32 head 2.48   hc0 norm 0.64   hc0 down 1.44   hc0 up 1.28   hc1 norm 0.34   hc1 down 1.40   hc1 up 1.37 | QSA layers: q8+kv-idx 0.12 k/v+norm-rope 0.36 kv+idx append 0.18 q+q-idx 1.60 scores+topk 0.63 kv-resolve 0.43 attention 1.00 gate 0.05 out-proj 0.95 router+plan+bell 0.49 shared+quant 0.73 waitA 0.13 VRAM hits 2.43 waitB 0.11 PCIe grp 0.10 waitCPU 0.10 copy+combine 0.15 (gap) 0.10   hc0 norm 0.10   hc0 down 0.48   hc0 up 0.43   hc1 norm 0.11   hc1 down 0.47   hc1 up 0.46 | total 44.87 ms/window over 38 windows
strata serve: prompt 7046 tokens = 0 reused + 7046 read in 15010 ms (469.4 tok/s), 60 generated in 1760 ms (34.1 tok/s), drafts accepted 23 of 35, 1 checkpoints
