Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3649] [FEAT]: Retrieval performance remains poor --- is a reranking strategy planned? #2356

New Issue
Closed
opened 2026-02-22 18:29:19 -05:00 by yindo · 6 comments
yindo commented 2026-02-22 18:29:19 -05:00
Owner
Copy Link
Copy Source

Originally created by @AI4SClab on GitHub (Apr 15, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3649

What would you like to see?

Hi team,

Thanks for your great work on AnythingLLM. We've been testing the platform with different embedding models, but unfortunately the retrieval performance remains relatively poor and inaccurate, regardless of the model used. We also tried adjusting similarity thresholds and other parameters, but the improvements were marginal.

From our experience, a more effective approach in such cases is to adopt a two-stage retrieval framework, where a reranker (e.g., based on a cross-encoder) is used to refine the results after the initial embedding-based retrieval.

May I ask if there's any plan to support reranking strategies (e.g., using BERT-based cross-encoders or similar models) in future releases? We'd love to hear your thoughts or roadmap on this.

Thanks again!

Originally created by @AI4SClab on GitHub (Apr 15, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3649 ### What would you like to see? Hi team, Thanks for your great work on AnythingLLM. We've been testing the platform with different embedding models, but unfortunately the retrieval performance remains relatively poor and inaccurate, regardless of the model used. We also tried adjusting similarity thresholds and other parameters, but the improvements were marginal. From our experience, a more effective approach in such cases is to adopt a two-stage retrieval framework, where a reranker (e.g., based on a cross-encoder) is used to refine the results after the initial embedding-based retrieval. May I ask if there's any plan to support reranking strategies (e.g., using BERT-based cross-encoders or similar models) in future releases? We'd love to hear your thoughts or roadmap on this. Thanks again!
yindo added the enhancementfeature request labels 2026-02-22 18:29:19 -05:00
yindo closed this issue 2026-02-22 18:29:19 -05:00
yindo commented 2026-02-22 18:29:20 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 15, 2025):

It is already possible:
-> Open Workspace Settings
-> Vector Database
-> Search Preference -> "Accuracy Optimized"
https://docs.anythingllm.com/llm-not-using-my-docs#vector-database-settings--search-preference

@timothycarambat commented on GitHub (Apr 15, 2025): It is already possible: -> Open Workspace Settings -> Vector Database -> Search Preference -> "Accuracy Optimized" https://docs.anythingllm.com/llm-not-using-my-docs#vector-database-settings--search-preference
yindo commented 2026-02-22 18:29:20 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 15, 2025):

As for expanding reranking support, I am very open to it - the base model we use that runs on CPU seems to work really well, but the reason it is not enabled by default is because of resource demand. Do you have a preferred model??

We should allow overriding of the default model since reranking, like embedding, can be language specific

@timothycarambat commented on GitHub (Apr 15, 2025): As for _expanding_ reranking support, I am very open to it - the base model we use that runs on CPU seems to work really well, but the reason it is not enabled by default is because of resource demand. Do you have a preferred model?? We should allow overriding of the default model since reranking, like embedding, can be language specific
yindo commented 2026-02-22 18:29:20 -05:00
Author
Owner
Copy Link
Copy Source

@AI4SClab commented on GitHub (Apr 17, 2025):

Thanks for the clarification!

We’re currently working with Chinese documents, and we did try enabling the “Accuracy Optimized” mode as you suggested. However, we still observe consistently poor retrieval performance — once we upload more than 5 .txt files into a workspace, the results become mostly irrelevant or incorrect.

We’re wondering if this could be due to limited Chinese language support in the embedding models we're currently using. At the moment, we’ve tested with bge-m3 (latest) and nomic-embed-text-v1, but the performance remains unstable.

Would you recommend any specific embedding + reranker combo that’s better suited for Chinese-language use cases? Or do you think allowing users to specify their own reranker model (e.g., a Chinese-pretrained cross-encoder) might improve things in our case?

Appreciate your thoughts!

@AI4SClab commented on GitHub (Apr 17, 2025): Thanks for the clarification! We’re currently working with Chinese documents, and we did try enabling the “Accuracy Optimized” mode as you suggested. However, we still observe consistently poor retrieval performance — once we upload more than 5 .txt files into a workspace, the results become mostly irrelevant or incorrect. We’re wondering if this could be due to limited Chinese language support in the embedding models we're currently using. At the moment, we’ve tested with bge-m3 (latest) and nomic-embed-text-v1, but the performance remains unstable. Would you recommend any specific embedding + reranker combo that’s better suited for Chinese-language use cases? Or do you think allowing users to specify their own reranker model (e.g., a Chinese-pretrained cross-encoder) might improve things in our case? Appreciate your thoughts!
yindo commented 2026-02-22 18:29:20 -05:00
Author
Owner
Copy Link
Copy Source

@nader939 commented on GitHub (Jun 4, 2025):

We should allow overriding of the default model since reranking, like embedding, can be language specific

Would like to see this option soon. I am interested in re-rankers that support both English and Arabic

@nader939 commented on GitHub (Jun 4, 2025): > We should allow overriding of the default model since reranking, like embedding, can be language specific Would like to see this option soon. I am interested in re-rankers that support both English and Arabic
yindo commented 2026-02-22 18:29:20 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 4, 2025):

https://github.com/Mintplex-Labs/anything-llm/issues/3942 is that issue.

Reranking was originally pushed to pilot how it performed in our use case across the customer base. Now looking that is is immensely better we can add it to other providers + make it configurable

@timothycarambat commented on GitHub (Jun 4, 2025): https://github.com/Mintplex-Labs/anything-llm/issues/3942 is that issue. Reranking was originally pushed to pilot how it performed in our use case across the customer base. Now looking that is is immensely better we can add it to other providers + make it configurable
yindo commented 2026-02-22 18:29:20 -05:00
Author
Owner
Copy Link
Copy Source

@momusticks commented on GitHub (Jun 17, 2025):

至于_扩展_重新排序支持，我对此持非常开放的态度——我们使用的在 CPU 上运行的基础模型似乎运行良好，但由于资源需求，它默认未启用。您有偏好的模型吗？

我们应该允许覆盖默认模型，因为重新排序（如嵌入）可以是特定于语言的

Hello, can you consider adding support for qwen3 embedding rearrangement model

@momusticks commented on GitHub (Jun 17, 2025): > 至于_扩展_重新排序支持，我对此持非常开放的态度——我们使用的在 CPU 上运行的基础模型似乎运行良好，但由于资源需求，它默认未启用。您有偏好的模型吗？ > > 我们应该允许覆盖默认模型，因为重新排序（如嵌入）可以是特定于语言的 Hello, can you consider adding support for qwen3 embedding rearrangement model
yindo changed title from [FEAT]: Retrieval performance remains poor --- is a reranking strategy planned? to [GH-ISSUE #3649] [FEAT]: Retrieval performance remains poor --- is a reranking strategy planned? 2026-06-05 14:46:05 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2356
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?