langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-24 13:06:09 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Knowledge retrieval example not working on self hosted #12127

New Issue
Closed
opened 2026-02-21 19:05:54 -05:00 by yindo · 6 comments
yindo commented 2026-02-21 19:05:54 -05:00
Owner
Copy Link
Copy Source

Originally created by @saedsaify on GitHub (Mar 25, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.1.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Hi,
I have created 2 identical implementation of a knowledge retrieval chat workflow (the basic one in the market place). One using an AWS machine and another on the free cloud version of dify (cloud.dify.ai).
The cloud version works perfectly on using different models like openAI and deepseek-r1. However, when using the same setup on my self hosted dify, it doesnt work at all and the chat always return no result. Checking the knowledge base I noticed that retrieval count on the self hosted version is always zero. I am using the same embedding, ranking and AI model on both implementations. I cant seem to get any error messages from the self hosted one, it just doesnt seem to retrieve the document in the knowledge base.

✔️ Expected Behavior

I expect same results on both cloud and self hosted, meaning I expect self hosted to reply with relevant info from the document using the the knowledge retrieval step.

❌ Actual Behavior

The system always returns unknown answer as instructed by the prompt when it doenst find relevant content.

Originally created by @saedsaify on GitHub (Mar 25, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.1.2 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce Hi, I have created 2 identical implementation of a knowledge retrieval chat workflow (the basic one in the market place). One using an AWS machine and another on the free cloud version of dify (cloud.dify.ai). The cloud version works perfectly on using different models like openAI and deepseek-r1. However, when using the same setup on my self hosted dify, it doesnt work at all and the chat always return no result. Checking the knowledge base I noticed that retrieval count on the self hosted version is always zero. I am using the same embedding, ranking and AI model on both implementations. I cant seem to get any error messages from the self hosted one, it just doesnt seem to retrieve the document in the knowledge base. ### ✔️ Expected Behavior I expect same results on both cloud and self hosted, meaning I expect self hosted to reply with relevant info from the document using the the knowledge retrieval step. ### ❌ Actual Behavior The system always returns unknown answer as instructed by the prompt when it doenst find relevant content.
yindo added the 🐞 bug label 2026-02-21 19:05:54 -05:00
yindo closed this issue 2026-02-21 19:05:54 -05:00
yindo commented 2026-02-21 19:05:55 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Mar 25, 2025):

Hey @saedsaify! 👋 I'm here to help you with bugs, questions, and more while you wait for a human maintainer. Let's tackle this issue together!

There is a known issue in Dify version 0.15.3 that might be relevant to your problem. This issue causes the rerank model switch in the 'Recall Settings' to automatically turn on, even after being manually disabled, leading to empty results from the knowledge retrieval module. This issue has been fixed in a recent update, so make sure you are using the latest version of Dify to resolve this problem [1][2].

Additionally, ensure that both the knowledge base level setting and the workflow/node level setting are aligned, as discrepancies can cause the Knowledge Retrieval node to return a different number of documents compared to retrieval testing [1].

If you are experiencing a validation error when uploading documents to the knowledge base in version 1.1.2, it is a known issue. You can either wait for the next version release where the bug will be fixed or revert to Dify version 1.1.0 where this issue does not occur [3].

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Mar 25, 2025): <!-- Greeting --> Hey @saedsaify! 👋 I'm here to help you with bugs, questions, and more while you wait for a human maintainer. Let's tackle this issue together! <!-- Answer --> There is a known issue in Dify version 0.15.3 that might be relevant to your problem. This issue causes the rerank model switch in the 'Recall Settings' to automatically turn on, even after being manually disabled, leading to empty results from the knowledge retrieval module. This issue has been fixed in a recent update, so make sure you are using the latest version of Dify to resolve this problem <sup>[[1]](https://github.com/langgenius/dify/issues/13622)</sup><sup>[[2]](https://github.com/langgenius/dify/issues/13770)</sup>. Additionally, ensure that both the knowledge base level setting and the workflow/node level setting are aligned, as discrepancies can cause the Knowledge Retrieval node to return a different number of documents compared to retrieval testing <sup>[[1]](https://github.com/langgenius/dify/issues/13622)</sup>. If you are experiencing a validation error when uploading documents to the knowledge base in version 1.1.2, it is a known issue. You can either wait for the next version release where the bug will be fixed or revert to Dify version 1.1.0 where this issue does not occur <sup>[[3]](https://github.com/langgenius/dify/issues/16591)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/dd89aeb6-d48d-4f09-8575-45b834803ef4?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/dd89aeb6-d48d-4f09-8575-45b834803ef4?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/dd89aeb6-d48d-4f09-8575-45b834803ef4?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/dd89aeb6-d48d-4f09-8575-45b834803ef4?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/dd89aeb6-d48d-4f09-8575-45b834803ef4?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/dd89aeb6-d48d-4f09-8575-45b834803ef4?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/dd89aeb6-d48d-4f09-8575-45b834803ef4?feedback_type=other)</sup>
yindo commented 2026-02-21 19:05:55 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Mar 25, 2025):

  • Current latest self-hosted version is v1.1.3
  • SaaS contains some hot-fixes and those are still not released to self-hosted version yet.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
@crazywoola commented on GitHub (Mar 25, 2025): - Current latest self-hosted version is v1.1.3 - SaaS contains some hot-fixes and those are still not released to self-hosted version yet. - This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
yindo commented 2026-02-21 19:05:55 -05:00
Author
Owner
Copy Link
Copy Source

@saedsaify commented on GitHub (Mar 26, 2025):

@dosu
I tried the latest release 1.1.3 but still the same issue. The document retrieval count is still zero and it doesnt seem to load it. I have attached a screenshot from the flow and the knowledge settings. Which is exactly the same as the one on the cloud account.

Image Image Image Image
@saedsaify commented on GitHub (Mar 26, 2025): @dosu I tried the latest release 1.1.3 but still the same issue. The document retrieval count is still zero and it doesnt seem to load it. I have attached a screenshot from the flow and the knowledge settings. Which is exactly the same as the one on the cloud account. <img width="776" alt="Image" src="https://github.com/user-attachments/assets/9e295021-8492-4a18-891e-5e22cca54498" /> <img width="786" alt="Image" src="https://github.com/user-attachments/assets/b65bd8f4-db4f-4928-a11e-4bccbe493309" /> <img width="864" alt="Image" src="https://github.com/user-attachments/assets/3a9da191-7441-4fec-9a2b-36de4ed4bc63" /> <img width="1199" alt="Image" src="https://github.com/user-attachments/assets/de73a8a8-0dca-4f3f-9fe2-4c870683156a" />
yindo commented 2026-02-21 19:05:55 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Mar 26, 2025):

The issue you're experiencing with the retrieval count remaining at zero in your self-hosted Dify setup could be related to several factors. Here are some steps and considerations to help troubleshoot and resolve the problem:

  1. Check ETL Configuration: Ensure that your self-hosted setup is correctly configured to use the appropriate ETL solution. If you're using Unstructured ETL, verify that the environment variables ETL_TYPE and UNSTRUCTURED_API_URL are set correctly [1].

  2. Verify Retrieval Settings: Double-check that the retrieval settings in your self-hosted setup match those in the cloud version. Pay particular attention to the Hybrid Search settings and ensure that the Rerank Model is correctly configured.

  3. Document Metadata: Ensure that the document "pma.docx" has the necessary metadata and is correctly indexed. The retrieval count will only increment if the document is part of a dataset with a doc_form of IndexType.PARENT_CHILD_INDEX and a corresponding ChildChunk is found [2].

  4. Logs and Debugging: Although specific logs or debugging tools weren't found in the documentation, check your server logs for any errors or warnings that might indicate issues with the knowledge retrieval process.

  5. Version Differences: Keep in mind that the SaaS version may contain hot-fixes not yet available in the self-hosted version. This could account for differences in behavior between the two environments [3].

If these steps do not resolve the issue, consider reviewing the configuration and setup of your self-hosted environment to ensure it aligns with the cloud version as closely as possible.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Mar 26, 2025): <!-- Answer --> The issue you're experiencing with the retrieval count remaining at zero in your self-hosted Dify setup could be related to several factors. Here are some steps and considerations to help troubleshoot and resolve the problem: 1. **Check ETL Configuration**: Ensure that your self-hosted setup is correctly configured to use the appropriate ETL solution. If you're using Unstructured ETL, verify that the environment variables `ETL_TYPE` and `UNSTRUCTURED_API_URL` are set correctly <sup>[[1]](https://github.com/langgenius/dify-docs/blob/main/jp/getting-started/install-self-hosted/environments.md)</sup>. 2. **Verify Retrieval Settings**: Double-check that the retrieval settings in your self-hosted setup match those in the cloud version. Pay particular attention to the Hybrid Search settings and ensure that the Rerank Model is correctly configured. 3. **Document Metadata**: Ensure that the document "pma.docx" has the necessary metadata and is correctly indexed. The retrieval count will only increment if the document is part of a dataset with a `doc_form` of `IndexType.PARENT_CHILD_INDEX` and a corresponding `ChildChunk` is found <sup>[[2]](https://github.com/langgenius/dify/blob/main/api/core/rag/retrieval/dataset_retrieval.py)</sup>. 4. **Logs and Debugging**: Although specific logs or debugging tools weren't found in the documentation, check your server logs for any errors or warnings that might indicate issues with the knowledge retrieval process. 5. **Version Differences**: Keep in mind that the SaaS version may contain hot-fixes not yet available in the self-hosted version. This could account for differences in behavior between the two environments <sup>[[3]](https://github.com/langgenius/dify/issues/16259)</sup>. If these steps do not resolve the issue, consider reviewing the configuration and setup of your self-hosted environment to ensure it aligns with the cloud version as closely as possible. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/e8166917-07e7-435a-af19-90fff4716a6a?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/e8166917-07e7-435a-af19-90fff4716a6a?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/e8166917-07e7-435a-af19-90fff4716a6a?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/e8166917-07e7-435a-af19-90fff4716a6a?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/e8166917-07e7-435a-af19-90fff4716a6a?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/e8166917-07e7-435a-af19-90fff4716a6a?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/e8166917-07e7-435a-af19-90fff4716a6a?feedback_type=other)</sup>
yindo commented 2026-02-21 19:05:55 -05:00
Author
Owner
Copy Link
Copy Source

@saedsaify commented on GitHub (Mar 26, 2025):

Thanks @dosu but I am not sure where to find those env variables and I am also not using an ETL, just a simple chat with a documents that has 3 steps and its installed from the market place. Everything else is the same as the cloud.

@saedsaify commented on GitHub (Mar 26, 2025): Thanks @dosu but I am not sure where to find those env variables and I am also not using an ETL, just a simple chat with a documents that has 3 steps and its installed from the market place. Everything else is the same as the cloud.
yindo commented 2026-02-21 19:05:55 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Mar 26, 2025):

In a self-hosted Dify installation, the environment variables are typically set in a .env file or directly in the docker-compose.yaml file if you're using Docker. These variables configure various aspects of the Dify environment, including API URLs, database connections, and more.

For your setup, since you're not using an ETL, you can focus on ensuring that the basic environment variables are correctly configured. Here are some key variables you might want to check:

  • CONSOLE_API_URL: Backend URL for the console API.
  • CONSOLE_WEB_URL: Front-end URL of the console web interface.
  • SERVICE_API_URL: Service API URL for front-end display.
  • APP_API_URL: WebApp API backend URL.
  • APP_WEB_URL: WebApp URL for file preview or download.
  • FILES_URL: Prefix for file preview or download URLs.
  • DB_USERNAME, DB_PASSWORD, DB_HOST, DB_PORT, DB_DATABASE: Database connection details.

These variables should be set to match your deployment environment. If you're using Docker, you can find these settings in the docker-compose.yaml file or a .env file in the root of your Dify installation directory [1].

If everything else is the same as the cloud version, ensure that your self-hosted environment is configured to match the cloud settings as closely as possible, especially regarding the retrieval settings and model configurations.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Mar 26, 2025): <!-- Answer --> In a self-hosted Dify installation, the environment variables are typically set in a `.env` file or directly in the `docker-compose.yaml` file if you're using Docker. These variables configure various aspects of the Dify environment, including API URLs, database connections, and more. For your setup, since you're not using an ETL, you can focus on ensuring that the basic environment variables are correctly configured. Here are some key variables you might want to check: - `CONSOLE_API_URL`: Backend URL for the console API. - `CONSOLE_WEB_URL`: Front-end URL of the console web interface. - `SERVICE_API_URL`: Service API URL for front-end display. - `APP_API_URL`: WebApp API backend URL. - `APP_WEB_URL`: WebApp URL for file preview or download. - `FILES_URL`: Prefix for file preview or download URLs. - `DB_USERNAME`, `DB_PASSWORD`, `DB_HOST`, `DB_PORT`, `DB_DATABASE`: Database connection details. These variables should be set to match your deployment environment. If you're using Docker, you can find these settings in the `docker-compose.yaml` file or a `.env` file in the root of your Dify installation directory <sup>[[1]](https://github.com/langgenius/dify-docs/blob/main/en/getting-started/install-self-hosted/environments.md)</sup>. If everything else is the same as the cloud version, ensure that your self-hosted environment is configured to match the cloud settings as closely as possible, especially regarding the retrieval settings and model configurations. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/a5a17796-9bd9-4ab8-ad5d-8ead402e099d?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/a5a17796-9bd9-4ab8-ad5d-8ead402e099d?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/a5a17796-9bd9-4ab8-ad5d-8ead402e099d?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/a5a17796-9bd9-4ab8-ad5d-8ead402e099d?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/a5a17796-9bd9-4ab8-ad5d-8ead402e099d?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/a5a17796-9bd9-4ab8-ad5d-8ead402e099d?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/a5a17796-9bd9-4ab8-ad5d-8ead402e099d?feedback_type=other)</sup>
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#12127
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?