run-llama/semtools
1
0
Fork 0
mirror of https://github.com/run-llama/semtools.git synced 2026-07-21 03:05:31 -04:00
Code Issues 4 Packages Projects Releases 10 Wiki Activity

[PR #18] [MERGED] Increase batch size in search, refactor search tests #27

New Issue
Closed
opened 2026-02-16 04:15:16 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 04:15:16 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/run-llama/semtools/pull/18
Author: @logan-markewich
Created: 9/3/2025
Status: ✅ Merged
Merged: 9/3/2025
Merged by: @logan-markewich

Base: main ← Head: logan/increase_batch_size

📝 Commits (3)

  • dbc9c35 Increase the search encode batch size from 1024 to 16384
  • a75ce5e improve testing
  • ac38b1e linting

📊 Changes

1 file changed (+70 additions, -35 deletions)

View changed files

📝 src/bin/search.rs (+70 -35)

📄 Description

This PR does two things

  1. Increases the default batch size in search -- 1024 is quite small, and we can increase this with minimal impact to resource usage according to local testing. This should given some ok speedup in certain conditions
  2. Refactors the search tests to use the actual model. This ensures less floating point error/differences due to use constant vectors, and also ensures more realistically that the model and similarity search works as expected

Fixes #17
Fixes #15

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/run-llama/semtools/pull/18 **Author:** [@logan-markewich](https://github.com/logan-markewich) **Created:** 9/3/2025 **Status:** ✅ Merged **Merged:** 9/3/2025 **Merged by:** [@logan-markewich](https://github.com/logan-markewich) **Base:** `main` ← **Head:** `logan/increase_batch_size` --- ### 📝 Commits (3) - [`dbc9c35`](https://github.com/run-llama/semtools/commit/dbc9c35199c322757c2766db1fafcc7d1b239d7a) Increase the search encode batch size from 1024 to 16384 - [`a75ce5e`](https://github.com/run-llama/semtools/commit/a75ce5e66a9465b0b39337781a25c3bbbdc0e6ad) improve testing - [`ac38b1e`](https://github.com/run-llama/semtools/commit/ac38b1e6bc86506cfa6baa60b4eb59dd06daacc6) linting ### 📊 Changes **1 file changed** (+70 additions, -35 deletions) <details> <summary>View changed files</summary> 📝 `src/bin/search.rs` (+70 -35) </details> ### 📄 Description This PR does two things 1. Increases the default batch size in `search` -- 1024 is quite small, and we can increase this with minimal impact to resource usage according to local testing. This should given some ok speedup in certain conditions 2. Refactors the `search` tests to use the actual model. This ensures less floating point error/differences due to use constant vectors, and also ensures more realistically that the model and similarity search works as expected Fixes #17 Fixes #15 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-16 04:15:16 -05:00
yindo closed this issue 2026-02-16 04:15:16 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
enhancement
pull-request
Mirrored from GitHub Pull Request
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: run-llama/semtools#27
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?