langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Looks like context is not retrieved when external data tool is enabled #1074

New Issue
Closed
opened 2026-02-21 17:29:50 -05:00 by yindo · 9 comments
yindo commented 2026-02-21 17:29:50 -05:00
Owner
Copy Link
Copy Source

Originally created by @ipv4sq on GitHub (Feb 20, 2024).

Originally assigned to: @Yeuoly on GitHub.

Self Checks

Dify version

0.5.6

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

  1. Create a new app
  2. create a new customized tool
  3. set this app to smart mode and add the customized tool
  4. add any random context from knowledge base

✔️ Expected Behavior

the context should be retrieved both from knowledge base and external tool(when necessary)

❌ Actual Behavior

knowledge base retrieval never happened, you can verify this from knowledge base retrieval log and actual response, the app acts like a retard

Originally created by @ipv4sq on GitHub (Feb 20, 2024). Originally assigned to: @Yeuoly on GitHub. ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to file this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). ### Dify version 0.5.6 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce 1. Create a new app 2. create a new customized tool 3. set this app to smart mode and add the customized tool 4. add any random context from knowledge base ### ✔️ Expected Behavior the context should be retrieved both from knowledge base and external tool(when necessary) ### ❌ Actual Behavior knowledge base retrieval never happened, you can verify this from knowledge base retrieval log and actual response, the app acts like a retard
yindo added the 🐞 bug label 2026-02-21 17:29:50 -05:00
yindo closed this issue 2026-02-21 17:29:50 -05:00
yindo commented 2026-02-21 17:29:50 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Feb 20, 2024):

🤖

Hey @ipv6sq! Great to see you around again. How's everything going?

Based on the context provided, it seems like the issue you're experiencing might be related to the dataset_query_variable in the ChatConfig. This variable is being set to an empty string when there is no context variable found in the modelConfig.configs.prompt_variables.

Here's the relevant code snippet:

const contextVar = modelConfig.configs.prompt_variables.find(item => item.is_context_var)?.key
const config: ChatConfig = {
  ...
  dataset_query_variable: contextVar || '',
  ...
}

In this code, contextVar is assigned the key of the first item in modelConfig.configs.prompt_variables that is a context variable (item.is_context_var is true). If no such item is found, contextVar is undefined, and the dataset_query_variable in the ChatConfig is set to an empty string ('').

To resolve this issue, you need to ensure that a context variable is defined in modelConfig.configs.prompt_variables and that item.is_context_var is true for that variable. If a context variable is not required, you should modify the code to handle the case where contextVar is undefined appropriately.

You can find this code in the web/app/components/app/configuration/debug/hooks.tsx file.

As for the dataset_query_variable in the ChatConfig, it's a new column added to the 'app_model_configs' table in the database. The exact purpose of this variable is not explicitly stated in the provided context. However, based on its name, it can be inferred that it might be used to store some sort of query related to a dataset in the application's model configurations. The exact usage would depend on the application's logic and how this variable is used in the codebase. You can find more about this in the api/migrations/versions/ab23c11305d4_add_dataset_query_variable_at_app_model_.py file.

I hope this helps! If you need further clarification, please let me know.

Sources

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

@dosubot[bot] commented on GitHub (Feb 20, 2024): _🤖_ <!-- Greeting --> Hey @ipv6sq! Great to see you around again. How's everything going? <!-- Answer --> Based on the context provided, it seems like the issue you're experiencing might be related to the `dataset_query_variable` in the `ChatConfig`. This variable is being set to an empty string when there is no context variable found in the `modelConfig.configs.prompt_variables`. Here's the relevant code snippet: ```tsx const contextVar = modelConfig.configs.prompt_variables.find(item => item.is_context_var)?.key const config: ChatConfig = { ... dataset_query_variable: contextVar || '', ... } ``` In this code, `contextVar` is assigned the key of the first item in `modelConfig.configs.prompt_variables` that is a context variable (`item.is_context_var` is true). If no such item is found, `contextVar` is `undefined`, and the `dataset_query_variable` in the `ChatConfig` is set to an empty string (`''`). To resolve this issue, you need to ensure that a context variable is defined in `modelConfig.configs.prompt_variables` and that `item.is_context_var` is true for that variable. If a context variable is not required, you should modify the code to handle the case where `contextVar` is `undefined` appropriately. You can find this code in the [web/app/components/app/configuration/debug/hooks.tsx](https://github.com/langgenius/dify/blob/main/web/app/components/app/configuration/debug/hooks.tsx) file. As for the `dataset_query_variable` in the `ChatConfig`, it's a new column added to the 'app_model_configs' table in the database. The exact purpose of this variable is not explicitly stated in the provided context. However, based on its name, it can be inferred that it might be used to store some sort of query related to a dataset in the application's model configurations. The exact usage would depend on the application's logic and how this variable is used in the codebase. You can find more about this in the [api/migrations/versions/ab23c11305d4_add_dataset_query_variable_at_app_model_.py](https://github.com/langgenius/dify/blob/main/api/migrations/versions/ab23c11305d4_add_dataset_query_variable_at_app_model_.py) file. I hope this helps! If you need further clarification, please let me know. ## Sources - [External data tool not work](https://github.com/langgenius/dify/issues/1934) - [web/app/components/app/configuration/debug/hooks.tsx](https://github.com/langgenius/dify/blob/main/web/app/components/app/configuration/debug/hooks.tsx) - [api/migrations/versions/ab23c11305d4_add_dataset_query_variable_at_app_model_.py](https://github.com/langgenius/dify/blob/main/api/migrations/versions/ab23c11305d4_add_dataset_query_variable_at_app_model_.py) <!-- Dosu Comment Footer --> <details open> <summary> #### About Dosu </summary> This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot. </details>
yindo commented 2026-02-21 17:29:50 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Feb 20, 2024):

The LLM itself will determine when to use the contexts based on it's description. In this case, you can not just link some random text, you should refine it's description for LLM to retrieve.
And choose a smarter LLM. eg OpenAI. Some of them just not smart enough to determine what to do.

@crazywoola commented on GitHub (Feb 20, 2024): The LLM itself will determine when to use the contexts based on it's description. In this case, you can not just link some random text, you should refine it's description for LLM to retrieve. And choose a smarter LLM. eg OpenAI. Some of them just not smart enough to determine what to do.
yindo commented 2026-02-21 17:29:51 -05:00
Author
Owner
Copy Link
Copy Source

@ipv4sq commented on GitHub (Feb 20, 2024):

By random I mean choose any knowledge set and ask questions accordingly,
not non sense
i increased the context window but it didn't work
You can easily reproduce that I believe, by comparing to a simple agent

crazywoola @.***>于2024年2月20日 周二下午7:47写道：

The LLM itself will determine when to use the contexts based on it's
description. In this case, you can not just link some random text, you
should refine it's description for LLM to retrieve.
And choose a smarter LLM. eg OpenAI. Some of them just not smart enough to
determine what to do.

—
Reply to this email directly, view it on GitHub
https://github.com/langgenius/dify/issues/2496#issuecomment-1954046692,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BC7SOIEZP55TXI2ENJ2BJNLYUSEMZAVCNFSM6AAAAABDQ4EJUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJUGA2DMNRZGI
.
You are receiving this because you were mentioned.Message ID:
@.***>

@ipv4sq commented on GitHub (Feb 20, 2024): By random I mean choose any knowledge set and ask questions accordingly, not non sense i increased the context window but it didn't work You can easily reproduce that I believe, by comparing to a simple agent crazywoola ***@***.***>于2024年2月20日 周二下午7:47写道： > The LLM itself will determine when to use the contexts based on it's > description. In this case, you can not just link some random text, you > should refine it's description for LLM to retrieve. > And choose a smarter LLM. eg OpenAI. Some of them just not smart enough to > determine what to do. > > — > Reply to this email directly, view it on GitHub > <https://github.com/langgenius/dify/issues/2496#issuecomment-1954046692>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/BC7SOIEZP55TXI2ENJ2BJNLYUSEMZAVCNFSM6AAAAABDQ4EJUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJUGA2DMNRZGI> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >
yindo commented 2026-02-21 17:29:51 -05:00
Author
Owner
Copy Link
Copy Source

@ipv4sq commented on GitHub (Feb 20, 2024):

of same prompt and knowledge set

By random I mean choose any knowledge set and ask questions accordingly,
not non sense
i increased the context window but it didn't work
You can easily reproduce that I believe, by comparing to a simple agent

crazywoola @.***>于2024年2月20日 周二下午7:47写道：

The LLM itself will determine when to use the contexts based on it's
description. In this case, you can not just link some random text, you
should refine it's description for LLM to retrieve.
And choose a smarter LLM. eg OpenAI. Some of them just not smart enough
to determine what to do.

—
Reply to this email directly, view it on GitHub
https://github.com/langgenius/dify/issues/2496#issuecomment-1954046692,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BC7SOIEZP55TXI2ENJ2BJNLYUSEMZAVCNFSM6AAAAABDQ4EJUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJUGA2DMNRZGI
.
You are receiving this because you were mentioned.Message ID:
@.***>

@ipv4sq commented on GitHub (Feb 20, 2024): of same prompt and knowledge set > By random I mean choose any knowledge set and ask questions accordingly, > not non sense > i increased the context window but it didn't work > You can easily reproduce that I believe, by comparing to a simple agent > > crazywoola ***@***.***>于2024年2月20日 周二下午7:47写道： > >> The LLM itself will determine when to use the contexts based on it's >> description. In this case, you can not just link some random text, you >> should refine it's description for LLM to retrieve. >> And choose a smarter LLM. eg OpenAI. Some of them just not smart enough >> to determine what to do. >> >> — >> Reply to this email directly, view it on GitHub >> <https://github.com/langgenius/dify/issues/2496#issuecomment-1954046692>, >> or unsubscribe >> <https://github.com/notifications/unsubscribe-auth/BC7SOIEZP55TXI2ENJ2BJNLYUSEMZAVCNFSM6AAAAABDQ4EJUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJUGA2DMNRZGI> >> . >> You are receiving this because you were mentioned.Message ID: >> ***@***.***> >> >
yindo commented 2026-02-21 17:29:51 -05:00
Author
Owner
Copy Link
Copy Source

@Yeuoly commented on GitHub (Feb 20, 2024):

Does your customized tool and datasets have a detailed description of themself? the quality of response of Agent deeply depends on the description of tools, so that Agent could choose the right tool, or you can just set strict instructions to tell Agent they has to use the tool to answer.

@Yeuoly commented on GitHub (Feb 20, 2024): Does your customized tool and datasets have a detailed description of themself? the quality of response of Agent deeply depends on the description of tools, so that Agent could choose the right tool, or you can just set strict instructions to tell Agent they has to use the tool to answer.
yindo commented 2026-02-21 17:29:51 -05:00
Author
Owner
Copy Link
Copy Source

@ipv4sq commented on GitHub (Feb 21, 2024):

Yes, I do have a customized tool description.
Here is my example, omit details but should be good to demonstrate
prompt:
you are a helpful customer service bot
customized tool:
Return some recommended products of this shop
knowledge set
The refund policy is that customer can return their goods within 30 days.

Ask

  1. recommend some products (this works fine, invoked customized tool)
  2. what's your refund policy(this does not retrieve the knowledge set, can be verified from knowledge set log and this was good on simple agent mode)
@ipv4sq commented on GitHub (Feb 21, 2024): Yes, I do have a customized tool description. Here is my example, omit details but should be good to demonstrate prompt: you are a helpful customer service bot customized tool: Return some recommended products of this shop knowledge set The refund policy is that customer can return their goods within 30 days. Ask 1. recommend some products (this works fine, invoked customized tool) 2. what's your refund policy(this does not retrieve the knowledge set, can be verified from knowledge set log and this was good on simple agent mode)
yindo commented 2026-02-21 17:29:51 -05:00
Author
Owner
Copy Link
Copy Source

@Yeuoly commented on GitHub (Feb 21, 2024):

What's the description of your datasets? you can find it here
image
Smart Agent uses this to detect use the dataset or not.

@Yeuoly commented on GitHub (Feb 21, 2024): What's the description of your datasets? you can find it here <img width="901" alt="image" src="https://github.com/langgenius/dify/assets/45712896/42abbdee-009f-4e5e-a914-c75d64a415e5"> Smart Agent uses this to detect use the dataset or not.
yindo commented 2026-02-21 17:29:51 -05:00
Author
Owner
Copy Link
Copy Source

@ipv4sq commented on GitHub (Feb 21, 2024):

bingo, that's the reason why it's not retrieving, thanks man!
btw, can you point me to the code base around this so I can dive into it further to better understand it. It looks like dataset is also consider as a special "customized tool" in this case, while in simple agent mode it's like a must hit tool

@ipv4sq commented on GitHub (Feb 21, 2024): bingo, that's the reason why it's not retrieving, thanks man! btw, can you point me to the code base around this so I can dive into it further to better understand it. It looks like dataset is also consider as a special "customized tool" in this case, while in simple agent mode it's like a must hit tool
yindo commented 2026-02-21 17:29:51 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Feb 21, 2024):

The LLM itself will determine when to use the contexts based on it's description. In this case, you can not just link some random text, you should refine it's description for LLM to retrieve. And choose a smarter LLM. eg OpenAI. Some of them just not smart enough to determine what to do.

Well, guess you didn't notice what I am saying. I am trying to say the same thing.

image

For codes related to this: https://github.com/langgenius/dify/blob/063191889d86a4d593337603321e6f785990809e/api/core/agent/agent/structed_multi_dataset_router_agent.py#L52

You can treat dataset as a tool as well.

@crazywoola commented on GitHub (Feb 21, 2024): > The LLM itself will determine when to use the contexts based on it's description. In this case, you can not just link some random text, you should refine it's description for LLM to retrieve. And choose a smarter LLM. eg OpenAI. Some of them just not smart enough to determine what to do. Well, guess you didn't notice what I am saying. I am trying to say the same thing. ![image](https://github.com/langgenius/dify/assets/100913391/8b7d58e9-e3ac-404b-9849-875c601be737) For codes related to this: https://github.com/langgenius/dify/blob/063191889d86a4d593337603321e6f785990809e/api/core/agent/agent/structed_multi_dataset_router_agent.py#L52 You can treat dataset as a tool as well.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#1074
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?