Korpus Zeichen: 4405976 Dateien: 325
{'length': '1k', 'run': 1, 'prompt_tokens': 1024, 'reused': 0, 'prompt_tok_s': 1869.0, 'generated': 256, 'output_tok_s': 126.2, 'drafts_accepted': 140, 'drafts_total': 197}
{'length': '1k', 'run': 2, 'prompt_tokens': 1023, 'reused': 0, 'prompt_tok_s': 1842.2, 'generated': 256, 'output_tok_s': 132.3, 'drafts_accepted': 141, 'drafts_total': 183}
{'length': '1k', 'run': 3, 'prompt_tokens': 1023, 'reused': 0, 'prompt_tok_s': 1862.2, 'generated': 256, 'output_tok_s': 144.8, 'drafts_accepted': 156, 'drafts_total': 186}
{'length': '4k', 'run': 1, 'prompt_tokens': 4233, 'reused': 0, 'prompt_tok_s': 2964.2, 'generated': 256, 'output_tok_s': 127.0, 'drafts_accepted': 140, 'drafts_total': 198}
{'length': '4k', 'run': 2, 'prompt_tokens': 4233, 'reused': 0, 'prompt_tok_s': 2950.0, 'generated': 256, 'output_tok_s': 128.5, 'drafts_accepted': 142, 'drafts_total': 194}
{'length': '4k', 'run': 3, 'prompt_tokens': 4233, 'reused': 0, 'prompt_tok_s': 2959.5, 'generated': 256, 'output_tok_s': 122.3, 'drafts_accepted': 131, 'drafts_total': 186}
{'length': '32k', 'run': 1, 'prompt_tokens': 30792, 'reused': 0, 'prompt_tok_s': 5060.5, 'generated': 256, 'output_tok_s': 127.2, 'drafts_accepted': 147, 'drafts_total': 203}
{'length': '32k', 'run': 2, 'prompt_tokens': 30793, 'reused': 0, 'prompt_tok_s': 5064.7, 'generated': 256, 'output_tok_s': 130.0, 'drafts_accepted': 157, 'drafts_total': 220}
{'length': '32k', 'run': 3, 'prompt_tokens': 30792, 'reused': 0, 'prompt_tok_s': 5065.6, 'generated': 256, 'output_tok_s': 134.0, 'drafts_accepted': 155, 'drafts_total': 206}
{'length': '128k', 'run': 1, 'prompt_tokens': 120605, 'reused': 0, 'prompt_tok_s': 5779.6, 'generated': 256, 'output_tok_s': 104.8, 'drafts_accepted': 121, 'drafts_total': 214}
{'length': '128k', 'run': 2, 'prompt_tokens': 120605, 'reused': 0, 'prompt_tok_s': 5761.9, 'generated': 256, 'output_tok_s': 108.4, 'drafts_accepted': 120, 'drafts_total': 181}
{'length': '128k', 'run': 3, 'prompt_tokens': 120605, 'reused': 0, 'prompt_tok_s': 5751.8, 'generated': 256, 'output_tok_s': 103.4, 'drafts_accepted': 118, 'drafts_total': 207}
| Length | Prompt tokens | Prompt tok/s (median, range) | Output tok/s (median, range) | Draft accept |
| --- | ---: | ---: | ---: | ---: |
| 1k | 1024 | 1862 (1842-1869) | 132.3 (126.2-144.8) | 77% |
| 4k | 4233 | 2960 (2950-2964) | 127.0 (122.3-128.5) | 71% |
| 32k | 30792 | 5065 (5060-5066) | 130.0 (127.2-134.0) | 73% |
| 128k | 120605 | 5762 (5752-5780) | 104.8 (103.4-108.4) | 60% |
