mirror of
https://github.com/langchain-ai/langchain-postgres.git
synced 2026-07-16 01:33:18 -04:00
[BUG] Improvements in HybridSearchConfig — Modifications in Score Fusion Logic and Configuration
#81
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @alberto-agudo on GitHub (Jul 8, 2025).
Originally assigned to: @dishaprakash on GitHub.
Hello @vishwarajanand. mentioning you here as you've been the contributor of the Hybrid Search solution. First, thank you for working on it.
I have recently refactored my code to include the
HybridSearchConfig, and in doing so, I think I've encountered several critical issues that need addressing to ensure correctness in hybrid search behavior.1. Major Issue: Incorrect Score Fusion Logic
The current default implementation linearly combines the semantic search distance (dense vector):
https://github.com/langchain-ai/langchain-postgres/blob/18b1bcdb75ed152da717e3d624e1ed822d17d60f/langchain_postgres/v2/async_vectorstore.py#L618-L622
with the full-text search score from
ts_rank_cd(sparse):https://github.com/langchain-ai/langchain-postgres/blob/18b1bcdb75ed152da717e3d624e1ed822d17d60f/langchain_postgres/v2/async_vectorstore.py#L663
This is fundamentally flawed because:
Current logic (
weighted_sum_ranking):https://github.com/langchain-ai/langchain-postgres/blob/18b1bcdb75ed152da717e3d624e1ed822d17d60f/langchain_postgres/v2/hybrid_search_config.py#L36-L64
This leads to an inconsistent score, where worse semantic matches can be ranked higher than better ones.
Note that reciprocal rank fusion overcomes these problems, but relies on each of the retrieved documents to be ordered in a ranking where the first element is the highest ranked and the last element is the lowest. This is the case for the inputs given to the reciprocal rank fusion function. However, both are sorted in descending order, which invalidates the assumption that lower distances are better (since the first element in the list will be the one with the highest distance).
https://github.com/langchain-ai/langchain-postgres/blob/18b1bcdb75ed152da717e3d624e1ed822d17d60f/langchain_postgres/v2/hybrid_search_config.py#L93-L95
Hence, both alternatives for retrieving the top k documents from hybrid search are flawed.
Recommendations:
You could also use a modified reciprocal rank fusion function that doesn't sort the inputs again.
2. Configuration and Querying Issues
Inconsistent use of
secondary_search_top_khybrid_search_config.secondary_top_kinstead.kfor both queries.HybridSearchConfig placement in the
__query_collectionmethodkis defined https://github.com/langchain-ai/langchain-postgres/blob/18b1bcdb75ed152da717e3d624e1ed822d17d60f/langchain_postgres/v2/async_vectorstore.py#L583-L592 to ensure proper parameter propagation.Best,
Alberto.
@alberto-agudo commented on GitHub (Aug 26, 2025):
Note, I've edited the original comment to indicate that, while reciprocal rank fusion usually solves the mismatch between different metrics, its assumption that distances from primary and secondary search should be sorted before calculation is also flawed. The rest of the reciprocal rank fusion function is ok
@dishaprakash commented on GitHub (Sep 17, 2025):
Hi @alberto-agudo Thank you for opening this detailed issue!
We've logged this as a bug and we will review it. We'll update this thread with our findings or any follow-up questions. We appreciate you taking the time to help improve the project!
@dishaprakash commented on GitHub (Oct 12, 2025):
@alberto-agudo This issue has been resolved in #255, #256 & #257. The fix will be available in the next release. Thanks again for the contribution! Closing this issue.