chore: Add tiktoken assets (missing from 69a3c19)

Browse Source

Assets were included in PyPI wheel 2.0.4b9 but not committed to Git.

...

This commit is contained in:

The BROKE Cluster Team

2026-02-04 20:56:23 +01:00

parent 69a3c19c16

commit 40479b0247

2 changed files with 100513 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

mlxk2/assets/whisper/gpt2.tiktoken

+50256

File diff suppressed because it is too large Load Diff

mlxk2/assets/whisper/multilingual.tiktoken

+50257

File diff suppressed because it is too large Load Diff