mirror of
https://github.com/Heretek-AI/Strata.git
synced 2026-11-07 10:05:58 -10:00
Niko1221
f5410a6feb
- the plain summary and the README get a Capped column (requests that reached the 512-token cap) and leave out the short case's prompt rate (an 80-token prompt measures the fixed cost of a request); results.json keeps it - BENCHMARK_PROTOCOL.md, COMMUNITY_BENCHMARKS.md, BENCH.bat and bench.sh: the standard suite took 2 to 4 minutes after loading on an RTX 3060 and on 2x RTX A4000, not 20 to 30
14 lines
853 B
Bash
Executable File
14 lines
853 B
Bash
Executable File
#!/bin/sh
|
|
# Benchmark this Strata install the way it is used and write a report you can share (docs/COMMUNITY_BENCHMARKS.md).
|
|
# It starts the model with your settings (or uses the one already running), takes about 10 minutes and changes nothing.
|
|
# Options: --clients 4 (several requests at once), --long (a 32K prompt), --compare <older results folder>.
|
|
# This is the quick speed check. The standard benchmark (docs/BENCHMARK_PROTOCOL.md), whose results other people can compare with, is `strata bench run`.
|
|
cd "$(dirname "$0")" || exit 1
|
|
if [ -x .venv/bin/python ]; then
|
|
echo "Quick speed check. For results other people can compare with (the standard benchmark, a few minutes), run: .venv/bin/python strata.py bench run"
|
|
echo
|
|
exec .venv/bin/python tools/strata_bench.py "$@"
|
|
fi
|
|
echo "Strata is not set up here yet: run ./setup.sh first."
|
|
exit 1
|