Two bugs regarding custom Extendtion to support external knowledge bases #869

Closed
opened 2026-02-16 10:20:48 -05:00 by yindo · 3 comments
Owner

Originally created by @Michael-C-H on GitHub (Dec 11, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues Dify issues & Dify Official Plugins, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.10.1

Plugin version

0.0.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

First of all, I would like to express my sincere gratitude to the official for launching the llamaCloud plugin, which has provided me with ideas and references for using other knowledge bases as external knowledge bases for dify.

During the development of my own Extendtion, I discovered the following two issues, both related to score_threshold:

  1. Try through retrieval_settings.get("score_threshold"), no matter how the knowledge base is set up, the obtained parameters are always the source code of the 0
  2. llamaCloud plugin, which may have a writing error. See the attachment
Image

✔️ Error log

No response

Originally created by @Michael-C-H on GitHub (Dec 11, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [Dify issues](https://github.com/langgenius/dify/issues) & [Dify Official Plugins](https://github.com/langgenius/dify-official-plugins/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.10.1 ### Plugin version 0.0.2 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce First of all, I would like to express my sincere gratitude to the official for launching the llamaCloud plugin, which has provided me with ideas and references for using other knowledge bases as external knowledge bases for dify. During the development of my own Extendtion, I discovered the following two issues, both related to score_threshold: 1. Try through retrieval_settings.get("score_threshold"), no matter how the knowledge base is set up, the obtained parameters are always the source code of the 0 2. llamaCloud plugin, which may have a writing error. See the attachment <img width="1206" height="928" alt="Image" src="https://github.com/user-attachments/assets/3c6ed8f1-b333-4983-b6f7-6009c0ba4168" /> ### ✔️ Error log _No response_
yindo added the bug label 2026-02-16 10:20:48 -05:00
yindo closed this issue 2026-02-16 10:20:48 -05:00
Author
Owner

@dosubot[bot] commented on GitHub (Dec 11, 2025):

Hi @Michael-C-H! I'm Dosu and I’m helping the dify-official-plugins team.

Here’s what’s happening with the two bugs:

  1. In the llamaCloud plugin, there’s a typo in the filtering logic: instead of using the string key "score_threshold", the code uses the variable score_threshold as a key when calling retrieval_settings.get(score_threshold, .0). This means it always returns 0.0 and never uses the actual configured threshold. To fix this, change the filtering line to use either the string key or the previously extracted variable, like if node.score < score_threshold: or retrieval_settings.get("score_threshold", 0.0) (source).

  2. For custom extensions, the reason retrieval_settings.get("score_threshold") always returns 0 is that Dify only forwards the configured threshold if score_threshold_enabled is set to true in the dataset’s retrieval model. Otherwise, it forces the value to 0.0 (source). Make sure you enable score_threshold_enabled in your dataset configuration or knowledge base UI to pass the actual value.

Let me know if you need help with the exact code changes or configuration steps!

To reply, just mention @dosu.


How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Dec 11, 2025): <!-- Greeting --> Hi @Michael-C-H! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the dify-official-plugins team. <!-- Answer --> Here’s what’s happening with the two bugs: 1. In the llamaCloud plugin, there’s a typo in the filtering logic: instead of using the string key `"score_threshold"`, the code uses the variable `score_threshold` as a key when calling `retrieval_settings.get(score_threshold, .0)`. This means it always returns `0.0` and never uses the actual configured threshold. To fix this, change the filtering line to use either the string key or the previously extracted variable, like `if node.score < score_threshold:` or `retrieval_settings.get("score_threshold", 0.0)` ([source](https://github.com/langgenius/dify-official-plugins/blob/5b58359887c6e620532fc7224971d589096e6d52/extensions/llamacloud/endpoints/llamacloud.py#L33-L64)). 2. For custom extensions, the reason `retrieval_settings.get("score_threshold")` always returns `0` is that Dify only forwards the configured threshold if `score_threshold_enabled` is set to `true` in the dataset’s retrieval model. Otherwise, it forces the value to `0.0` ([source](https://github.com/langgenius/dify/blob/61ee1b9094ff077e94ec4c01f7d8f24e08930e82/api/services/external_knowledge_service.py#L304-L324)). Make sure you enable `score_threshold_enabled` in your dataset configuration or knowledge base UI to pass the actual value. Let me know if you need help with the exact code changes or configuration steps! <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/1b545fb6-12d6-46f7-a475-db8f9f02a613?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/1b545fb6-12d6-46f7-a475-db8f9f02a613?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/1b545fb6-12d6-46f7-a475-db8f9f02a613?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/1b545fb6-12d6-46f7-a475-db8f9f02a613?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/1b545fb6-12d6-46f7-a475-db8f9f02a613?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/1b545fb6-12d6-46f7-a475-db8f9f02a613?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/1b545fb6-12d6-46f7-a475-db8f9f02a613?feedback_type=other)</sup> [![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=Here%E2%80%99s%20what%E2%80%99s%20happening%20with%20the%20two%20bugs%3A%0A%0A1.%20In%20the%20llamaCloud%20plugin%2C%20there%E2%80%99s%20a%20typo%20in%20the%20filtering%20logic%3A%20instead%20of%20using%20the%20string%20key%20%60%22score_threshold%22%60%2C%20the%20code%20uses%20the%20variable%20%60score_threshold%60%20as%20a%20key%20when%20calling%20%60retrieval_settings.get%28score_threshold%2C%20.0%29%60.%20This%20means%20it%20always%20returns%20%600.0%60%20and%20never%20uses%20the%20actual%20configured%20threshold.%20To%20fix%20this%2C%20change%20the%20filtering%20line%20to%20use%20either%20the%20string%20key%20or%20the%20previously%20extracted%20variable%2C%20like%20%60if%20node.score%20%3C%20score_threshold%3A%60%20or%20%60retrieval_settings.get%28%22score_threshold%22%2C%200.0%29%60%20%28%5Bsource%5D%28https%3A//github.com/langgenius/dify-official-plugins/blob/5b58359887c6e620532fc7224971d589096e6d52/extensions/llamacloud/endpoints/llamacloud.py%23L33-L64%29%29.%0A%0A2.%20For%20custom%20extensions%2C%20the%20reason%20%60retrieval_settings.get%28%22score_threshold%22%29%60%20always%20returns%20%600%60%20is%20that%20Dify%20only%20forwards%20the%20configured%20threshold%20if%20%60score_threshold_enabled%60%20is%20set%20to%20%60true%60%20in%20the%20dataset%E2%80%99s%20retrieval%20model.%20Otherwise%2C%20it%20forces%20the%20value%20to%20%600.0%60%20%28%5Bsource%5D%28https%3A//github.com/langgenius/dify/blob/61ee1b9094ff077e94ec4c01f7d8f24e08930e82/api/services/external_knowledge_service.py%23L304-L324%29%29.%20Make%20sure%20you%20enable%20%60score_threshold_enabled%60%20in%20your%20dataset%20configuration%20or%20knowledge%20base%20UI%20to%20pass%20the%20actual%20value.%0A%0ALet%20me%20know%20if%20you%20need%20help%20with%20the%20exact%20code%20changes%20or%20configuration%20steps%21)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify-official-plugins/issues/2242)
Author
Owner

@Michael-C-H commented on GitHub (Dec 11, 2025):

The knowledge base is set up as follows:Image

The source code of extendtion is as follows:
Image

The results of the recall test are as follows:
Image

@dosu

@Michael-C-H commented on GitHub (Dec 11, 2025): The knowledge base is set up as follows:<img width="1340" height="688" alt="Image" src="https://github.com/user-attachments/assets/24bb5ec8-f7cb-4053-8a97-892a9380e363" /> The source code of extendtion is as follows: <img width="942" height="963" alt="Image" src="https://github.com/user-attachments/assets/9ab844a6-f08f-4323-b082-fecb27f266e8" /> The results of the recall test are as follows: <img width="1913" height="702" alt="Image" src="https://github.com/user-attachments/assets/4dba46bf-c9bf-4f12-b73b-6a2cf33579b7" /> @dosu
Author
Owner

@dosubot[bot] commented on GitHub (Dec 11, 2025):

Your knowledge base UI and extension code look correct, but the recall test results show that results below the score_threshold (0.5) are not being filtered out. Here’s what to check:

  1. There was a known UI bug where the score threshold input box only accepted 0 or 1, but the slider allows float values like 0.5. Make sure you set the threshold using the slider, not the input box or arrows, to ensure a float value is saved and sent downstream. You can confirm this by re-saving the setting with the slider and testing again (details).

  2. In Dify’s backend, score_threshold is only forwarded to your extension if score_threshold_enabled is true. If you see results below the threshold, add a debug log in your extension to print the value and type of score_threshold received at runtime. This will confirm if the extension is actually getting 0.5 or just 0.0 (backend logic).

  3. Double-check your filtering logic: ensure you’re comparing floats, not strings, and that the filter is applied before returning results. For example:

if float(node['score']) < float(score_threshold):
    continue

If the value is correct and the logic is correct, but results below the threshold are still returned, the issue may be with the vector search API or a post-processing step ignoring the filter. In that case, add logging before and after filtering to see what’s happening.

Let me know what your debug output shows or if you need help with the logging code!

To reply, just mention @dosu.


How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Dec 11, 2025): <!-- Answer --> Your knowledge base UI and extension code look correct, but the recall test results show that results below the score_threshold (0.5) are not being filtered out. Here’s what to check: 1. There was a known UI bug where the score threshold input box only accepted 0 or 1, but the slider allows float values like 0.5. Make sure you set the threshold using the slider, not the input box or arrows, to ensure a float value is saved and sent downstream. You can confirm this by re-saving the setting with the slider and testing again [(details)](https://github.com/langgenius/dify/issues/15959#issuecomment-2727866540). 2. In Dify’s backend, score_threshold is only forwarded to your extension if score_threshold_enabled is true. If you see results below the threshold, add a debug log in your extension to print the value and type of score_threshold received at runtime. This will confirm if the extension is actually getting 0.5 or just 0.0 [(backend logic)](https://github.com/langgenius/dify/blob/8daf9ce98d13aa37c1542010c57961b4fad95c22/api/services/external_knowledge_service.py#L279-L329). 3. Double-check your filtering logic: ensure you’re comparing floats, not strings, and that the filter is applied before returning results. For example: ```python if float(node['score']) < float(score_threshold): continue ``` If the value is correct and the logic is correct, but results below the threshold are still returned, the issue may be with the vector search API or a post-processing step ignoring the filter. In that case, add logging before and after filtering to see what’s happening. Let me know what your debug output shows or if you need help with the logging code! <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/fe25c4b2-96a2-49b8-801d-f15ae0a3fe57?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/fe25c4b2-96a2-49b8-801d-f15ae0a3fe57?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/fe25c4b2-96a2-49b8-801d-f15ae0a3fe57?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/fe25c4b2-96a2-49b8-801d-f15ae0a3fe57?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/fe25c4b2-96a2-49b8-801d-f15ae0a3fe57?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/fe25c4b2-96a2-49b8-801d-f15ae0a3fe57?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/fe25c4b2-96a2-49b8-801d-f15ae0a3fe57?feedback_type=other)</sup> [![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=Your%20knowledge%20base%20UI%20and%20extension%20code%20look%20correct%2C%20but%20the%20recall%20test%20results%20show%20that%20results%20below%20the%20score_threshold%20%280.5%29%20are%20not%20being%20filtered%20out.%20Here%E2%80%99s%20what%20to%20check%3A%0A%0A1.%20There%20was%20a%20known%20UI%20bug%20where%20the%20score%20threshold%20input%20box%20only%20accepted%200%20or%201%2C%20but%20the%20slider%20allows%20float%20values%20like%200.5.%20Make%20sure%20you%20set%20the%20threshold%20using%20the%20slider%2C%20not%20the%20input%20box%20or%20arrows%2C%20to%20ensure%20a%20float%20value%20is%20saved%20and%20sent%20downstream.%20You%20can%20confirm%20this%20by%20re-saving%20the%20setting%20with%20the%20slider%20and%20testing%20again%20%5B%28details%29%5D%28https%3A//github.com/langgenius/dify/issues/15959%23issuecomment-2727866540%29.%0A%0A2.%20In%20Dify%E2%80%99s%20backend%2C%20score_threshold%20is%20only%20forwarded%20to%20your%20extension%20if%20score_threshold_enabled%20is%20true.%20If%20you%20see%20results%20below%20the%20threshold%2C%20add%20a%20debug%20log%20in%20your%20extension%20to%20print%20the%20value%20and%20type%20of%20score_threshold%20received%20at%20runtime.%20This%20will%20confirm%20if%20the%20extension%20is%20actually%20getting%200.5%20or%20just%200.0%20%5B%28backend%20logic%29%5D%28https%3A//github.com/langgenius/dify/blob/8daf9ce98d13aa37c1542010c57961b4fad95c22/api/services/external_knowledge_service.py%23L279-L329%29.%0A%0A3.%20Double-check%20your%20filtering%20logic%3A%20ensure%20you%E2%80%99re%20comparing%20floats%2C%20not%20strings%2C%20and%20that%20the%20filter%20is%20applied%20before%20returning%20results.%20For%20example%3A%0A%60%60%60python%0Aif%20float%28node%5B%27score%27%5D%29%20%3C%20float%28score_threshold%29%3A%0A%20%20%20%20continue%0A%60%60%60%0A%0AIf%20the%20value%20is%20correct%20and%20the%20logic%20is%20correct%2C%20but%20results%20below%20the%20threshold%20are%20still%20returned%2C%20the%20issue%20may%20be%20with%20the%20vector%20search%20API%20or%20a%20post-processing%20step%20ignoring%20the%20filter.%20In%20that%20case%2C%20add%20logging%20before%20and%20after%20filtering%20to%20see%20what%E2%80%99s%20happening.%0A%0ALet%20me%20know%20what%20your%20debug%20output%20shows%20or%20if%20you%20need%20help%20with%20the%20logging%20code%21)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify-official-plugins/issues/2242)
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#869