run-llama/semtools
1
0
Fork 0
mirror of https://github.com/run-llama/semtools.git synced 2026-07-21 11:15:24 -04:00
Code Issues 4 Packages Projects Releases 10 Wiki Activity

[chore] Increase default batch size #2

New Issue
Closed
opened 2026-02-16 04:15:05 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 04:15:05 -05:00
Owner
Copy Link
Copy Source

Originally created by @logan-markewich on GitHub (Sep 2, 2025).

The default batch size is pretty small right now (1024). We can probably get some minor speedup improvements by bumping this up.

Initial benchmarks show little-to-no impact on memory usage by increasing this. Hot-spot seems to be 16k->66k ish. Might go on the lower side to be safe

Originally created by @logan-markewich on GitHub (Sep 2, 2025). The default batch size is pretty small right now (1024). We can probably get some minor speedup improvements by bumping this up. Initial benchmarks show little-to-no impact on memory usage by increasing this. Hot-spot seems to be 16k->66k ish. Might go on the lower side to be safe
yindo added the enhancement label 2026-02-16 04:15:05 -05:00
yindo closed this issue 2026-02-16 04:15:05 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
enhancement
pull-request
Mirrored from GitHub Pull Request
No Label enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: run-llama/semtools#2
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?