langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 04:25:29 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #31309] fix(api): fix IRIS hybrid search returning zero results #33178

New Issue
Closed
opened 2026-02-21 20:52:48 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:52:48 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/31309

State: closed
Merged: Yes

Fixes #31215

Summary

Fix IRIS hybrid search returning 0 results when vector search and full-text search work individually. The root cause was that search_by_full_text() wasn't returning BM25 relevance scores in document metadata, causing WeightRerankRunner to filter out all full-text results during hybrid search.

What changed

  • Use IRIS auto-generated Rank function to calculate BM25 relevance scores
  • Fix Rank function name generation by removing underscores per IRIS naming convention
  • Pass %ID (system ID) instead of id column to Rank function
  • Add metadata["score"] to returned documents for hybrid search compatibility
  • Support document_ids_filter parameter in full-text search
  • Add error handling with fallback to fixed score (0.5) when Rank function fails
  • Improve debug logging for troubleshooting

Modified file: api/core/rag/datasource/vdb/iris/iris_vector.py

Why it matters

IRIS users cannot use hybrid search (a core feature combining vector + full-text search) without this fix. The bug affects all IRIS deployments with text indexing enabled.

Technical details

IRIS InterSystems auto-generates BM25 scoring functions for %iFind.Index.Basic indexes:

  • Function naming: {schema}.{table}_{index}Rank with underscores removed
  • Function signature: Rank(%ID, query) requires system ID, not application id column
  • Algorithm: Uses Okapi BM25 for relevance scoring

Tested with:

  • IRIS database with text indexing enabled
  • Verified BM25 scores are correctly calculated
  • Confirmed hybrid search combines vector and full-text results
  • Verified fallback behavior when Rank function unavailable

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran make lint and make type-check (backend) and cd web && npx lint-staged (frontend) to appease the lint gods
**Original Pull Request:** https://github.com/langgenius/dify/pull/31309 **State:** closed **Merged:** Yes --- Fixes #31215 ## Summary Fix IRIS hybrid search returning 0 results when vector search and full-text search work individually. The root cause was that `search_by_full_text()` wasn't returning BM25 relevance scores in document metadata, causing `WeightRerankRunner` to filter out all full-text results during hybrid search. ## What changed - Use IRIS auto-generated Rank function to calculate BM25 relevance scores - Fix Rank function name generation by removing underscores per IRIS naming convention - Pass `%ID` (system ID) instead of `id` column to Rank function - Add `metadata["score"]` to returned documents for hybrid search compatibility - Support `document_ids_filter` parameter in full-text search - Add error handling with fallback to fixed score (0.5) when Rank function fails - Improve debug logging for troubleshooting **Modified file:** `api/core/rag/datasource/vdb/iris/iris_vector.py` ## Why it matters IRIS users cannot use hybrid search (a core feature combining vector + full-text search) without this fix. The bug affects all IRIS deployments with text indexing enabled. ## Technical details IRIS InterSystems auto-generates BM25 scoring functions for `%iFind.Index.Basic` indexes: - Function naming: `{schema}.{table}_{index}Rank` with underscores removed - Function signature: `Rank(%ID, query)` requires system ID, not application id column - Algorithm: Uses Okapi BM25 for relevance scoring **Tested with:** - IRIS database with text indexing enabled - Verified BM25 scores are correctly calculated - Confirmed hybrid search combines vector and full-text results - Verified fallback behavior when Rank function unavailable ## Checklist - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs) - [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!) - [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change. - [ ] I've updated the documentation accordingly. - [x] I ran `make lint` and `make type-check` (backend) and `cd web && npx lint-staged` (frontend) to appease the lint gods
yindo added the pull-request label 2026-02-21 20:52:48 -05:00
yindo closed this issue 2026-02-21 20:52:48 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#33178
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?