Heretek-AI/Strata
1
0 Fork 0
mirror of https://github.com/Heretek-AI/Strata.git synced 2026-11-08 06:48:28 -10:00
Code Issues Packages Projects Releases Wiki Activity

UD-IQ1_S on a Mac: the measured line from the M5 Max run (alone: 15-18 tok/s, 3K prompt 230-310, 14K 520)

Browse Source
This commit is contained in:
Niko1221 committed 2026-10-30 22:44:19 +01:00
1 parent 417765b8d8
commit 6a7d750578
1 file changed
+2 -2
Download Patch File Download Diff File Expand all files Collapse all files
strata_control/dsv41_fit.py
+2 -2
View File
@@ -531,8 +531,8 @@ MAC_MEASURED_PACK = {
"IQ2_XXS": ("M5 Max, 128 GB, 2.00 bpw: 15-16 tokens/s for a short answer (14-15 after a long prompt), a 3,000-token prompt at about 240 "
"tokens/s, 32,000 at about 610; the engine kept about 69 GB of the experts resident and read about 48 MB a token from the SSD"),
"IQ2_XS": MAC_MEASURED,
"UD-IQ1_S": ("M5 Max, 128 GB, Unsloth UD-IQ1_S: 13-18 tokens/s for a short answer (12-18 after a 3,000-token prompt), a 3,000-token prompt at "
"about 210-320 tokens/s; one session, ten answers and a tool call checked by eye, nothing else measured"),
"UD-IQ1_S": ("M5 Max, 128 GB, Unsloth UD-IQ1_S: 15-18 tokens/s for a short answer (16-18 after a 3,000-token prompt), a 3,000-token prompt at "
"about 230-310 tokens/s, 14,000 at about 520; the engine kept about 68 GB of the experts resident and the process used about 79 GB"),
}