Recall Issues in multiple Journal Impact Factor Queries #5325

Closed
opened 2026-02-21 18:10:25 -05:00 by yindo · 2 comments
Owner

Originally created by @iray1991 on GitHub (Aug 31, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

Dify 0.7.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Querying the journal impact factor knowledge base in bulk: When querying a single journal name, the recall results are correct (as shown in Figure 1). However, when sending multiple journals at once, the recall results are incomplete. It seems that not every journal is queried individually in the knowledge base, but rather multiple journal names are combined into one query, resulting in some journal names not being recalled (as shown in Figure 2).

企业微信截图_17251006752742 企业微信截图_17251006776260

✔️ Expected Behavior

No response

Actual Behavior

No response

Originally created by @iray1991 on GitHub (Aug 31, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:) - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version Dify 0.7.1 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce Querying the journal impact factor knowledge base in bulk: When querying a single journal name, the recall results are correct (as shown in Figure 1). However, when sending multiple journals at once, the recall results are incomplete. It seems that not every journal is queried individually in the knowledge base, but rather multiple journal names are combined into one query, resulting in some journal names not being recalled (as shown in Figure 2). <img width="1200" alt="企业微信截图_17251006752742" src="https://github.com/user-attachments/assets/f3dd6c38-b7a8-4ec2-ac53-42d755a611e6"> <img width="1192" alt="企业微信截图_17251006776260" src="https://github.com/user-attachments/assets/1f7393cb-778b-46e6-a0b4-0daef31b23d9"> ### ✔️ Expected Behavior _No response_ ### ❌ Actual Behavior _No response_
yindo added the 🐞 bug label 2026-02-21 18:10:25 -05:00
yindo closed this issue 2026-02-21 18:10:25 -05:00
Author
Owner

@dosubot[bot] commented on GitHub (Aug 31, 2024):

To address the issue of incomplete recall results when querying multiple journal names at once in the Dify 0.7.1 self-hosted (Docker) environment, you should configure the retrieval strategy appropriately. Specifically, you might want to use the "Multi-Path Retrieval" mode, which involves configuring a Rerank model for result re-ranking. This mode is designed to handle more complex queries and can improve the completeness of the recall results.

Here are the steps to set up the "Multi-Path Retrieval" mode:

  1. Navigate to Context -- Retrieval Settings -- Rerank Setting.
  2. Enable the Multi-Path Retrieval mode.
  3. Configure the Rerank model by providing an API Key for the Rerank model (such as Cohere, Jina, etc.) on the "Model Provider" page.
  4. Adjust parameters like TopK and Score Threshold to fine-tune the retrieval process.

For more detailed instructions, refer to the help documentation [1][2].

Additionally, it's important to note that the "N-of-1 Retrieval Mode" may not be suitable for applications with a large number of datasets or where dataset descriptions are not significantly different. This mode relies on the model's inference ability to select the most appropriate single dataset based on user intent, which can lead to incomplete recall results in such scenarios. Therefore, for applications with multiple datasets, the "Multi-Path Retrieval Mode" is recommended as it does not depend on the model's inference ability or dataset descriptions, thus providing higher quality recall results [3].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Aug 31, 2024): <!-- Answer --> To address the issue of incomplete recall results when querying multiple journal names at once in the Dify 0.7.1 self-hosted (Docker) environment, you should configure the retrieval strategy appropriately. Specifically, you might want to use the "Multi-Path Retrieval" mode, which involves configuring a Rerank model for result re-ranking. This mode is designed to handle more complex queries and can improve the completeness of the recall results. Here are the steps to set up the "Multi-Path Retrieval" mode: 1. Navigate to **Context -- Retrieval Settings -- Rerank Setting**. 2. Enable the Multi-Path Retrieval mode. 3. Configure the Rerank model by providing an API Key for the Rerank model (such as Cohere, Jina, etc.) on the "Model Provider" page. 4. Adjust parameters like **TopK** and **Score Threshold** to fine-tune the retrieval process. For more detailed instructions, refer to the [help documentation](https://docs.dify.ai/guides/knowledge-base/create-knowledge-and-upload-documents#id-5-indexing-methods) <sup>[[1]](https://github.com/langgenius/dify-docs/blob/main/en/guides/workflow/node/knowledge-retrieval.md)</sup><sup>[[2]](https://github.com/langgenius/dify-docs/blob/main/en/guides/knowledge-base/integrate-knowledge-within-application.md)</sup>. Additionally, it's important to note that the "N-of-1 Retrieval Mode" may not be suitable for applications with a large number of datasets or where dataset descriptions are not significantly different. This mode relies on the model's inference ability to select the most appropriate single dataset based on user intent, which can lead to incomplete recall results in such scenarios. Therefore, for applications with multiple datasets, the "Multi-Path Retrieval Mode" is recommended as it does not depend on the model's inference ability or dataset descriptions, thus providing higher quality recall results <sup>[[3]](https://github.com/langgenius/dify-docs/blob/main/zh_CN/learn-more/extended-reading/retrieval-augment/retrieval.md)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
Author
Owner

@crazywoola commented on GitHub (Aug 31, 2024):

This is an expected behavior.

@crazywoola commented on GitHub (Aug 31, 2024): This is an expected behavior.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#5325