huggingface/text-generation-inference
1
0
Fork 0
mirror of https://github.com/huggingface/text-generation-inference.git synced 2026-10-12 02:15:47 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
85790a19a7fa9917733a9a4536e50057f8d513d8
text-generation-inference/benchmark
T
History
Alvaro Moran efb94e0d3d Patch version 3.3.6 (#3329) 
* chore: prepare version 3.3.6

* fix(benchmark): clear up progress_gauge fn signature

Otherwise there is a compiler error.
2025-09-16 19:15:23 -04:00
..
src
Patch version 3.3.6 (#3329)
2025-09-16 19:15:23 -04:00
Cargo.toml
Simplify crossterm imports (#2545)
2024-09-24 14:57:20 +02:00
README.md
Update the link to the Ratatui organization (#2546)
2024-09-24 14:51:48 +02:00

README.md

Text Generation Inference benchmarking tool

benchmark

A lightweight benchmarking tool based inspired by oha and powered by Ratatui.

Install

make install-benchmark

Run

First, start text-generation-inference:

text-generation-launcher --model-id bigscience/bloom-560m

Then run the benchmarking tool:

text-generation-benchmark --tokenizer-name bigscience/bloom-560m
Reference in New Issue View Git Blame Copy Permalink