mirror of
https://github.com/Niko1221/Strata.git
synced 2026-11-07 06:15:58 -10:00
Niko1221
f5410a6feb
- the plain summary and the README get a Capped column (requests that reached the 512-token cap) and leave out the short case's prompt rate (an 80-token prompt measures the fixed cost of a request); results.json keeps it - BENCHMARK_PROTOCOL.md, COMMUNITY_BENCHMARKS.md, BENCH.bat and bench.sh: the standard suite took 2 to 4 minutes after loading on an RTX 3060 and on 2x RTX A4000, not 20 to 30
20 lines
950 B
Batchfile
20 lines
950 B
Batchfile
@echo off
|
|
rem Benchmark this Strata install the way it is used and write a report you can share (docs/COMMUNITY_BENCHMARKS.md).
|
|
rem It starts the model with your settings (or uses the one already running), takes about 10 minutes and changes nothing.
|
|
rem Options: --clients 4 (several requests at once), --long (a 32K prompt), --compare <older results folder>.
|
|
rem This is the quick speed check. The standard benchmark (docs/BENCHMARK_PROTOCOL.md), whose results other people can compare with, is `strata bench run`.
|
|
setlocal
|
|
title Strata - benchmark
|
|
cd /d "%~dp0"
|
|
if not exist ".venv\Scripts\python.exe" (
|
|
echo Strata is not set up here yet: run START-HERE.bat first.
|
|
pause
|
|
exit /b 1
|
|
)
|
|
echo Quick speed check. For results other people can compare with (the standard benchmark, a few minutes), run: .venv\Scripts\python.exe strata.py bench run
|
|
echo.
|
|
".venv\Scripts\python.exe" tools\strata_bench.py %*
|
|
set "RC=%ERRORLEVEL%"
|
|
pause
|
|
exit /b %RC%
|