langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 03:55:25 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Metadata filter work in Chatbot but NOT work in workflow #11950

New Issue
Closed
opened 2026-02-21 19:04:55 -05:00 by yindo · 7 comments
yindo commented 2026-02-21 19:04:55 -05:00
Owner
Copy Link
Copy Source

Originally created by @gemchen on GitHub (Mar 21, 2025).

Originally assigned to: @JohnJyong on GitHub.

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.1.1

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

Created “Metadata Filter Conditions” in a Chatbot to ensure it recall documents with 'document_name' start with '<filter_word>'.
while create same filter in workflow, it just ignore the filter and recall all documents with search input.

✔️ Expected Behavior

metadata filter works well as expect.

❌ Actual Behavior

I tried below criteria in the "workflow", the result is in the table.

contains ： ❌
not contains: ❌
start with: ❌
end with: ❌
is: ❌
empty: ✅
not empty/before/after/≥/≤: not test

Originally created by @gemchen on GitHub (Mar 21, 2025). Originally assigned to: @JohnJyong on GitHub. ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.1.1 ### Cloud or Self Hosted Self Hosted (Source) ### Steps to reproduce Created “Metadata Filter Conditions” in a Chatbot to ensure it recall documents with 'document_name' start with '<filter_word>'. while create same filter in workflow, it just ignore the filter and recall all documents with search input. ### ✔️ Expected Behavior metadata filter works well as expect. ### ❌ Actual Behavior I tried below criteria in the "workflow", the result is in the table. contains ： ❌ not contains: ❌ start with: ❌ end with: ❌ is: ❌ empty: ✅ not empty/before/after/≥/≤: not test
yindo added the 🌊 feat:workflow label 2026-02-21 19:04:55 -05:00
yindo closed this issue 2026-02-21 19:04:55 -05:00
yindo commented 2026-02-21 19:04:55 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Mar 21, 2025):

Hello @gemchen! I'm here to help you with any bugs, questions, or contributions you have in mind. Let's tackle this issue together!

The issue you're experiencing with metadata filters not working in the workflow component, while they work in the Chatbot, might be related to how the filters are implemented or applied in the workflow component.

In the workflow component, metadata filters are applied in the file web/app/components/workflow/panel/version-history-panel/filter/index.tsx. The filtering logic is handled by the Filter component, which uses the useFilterOptions hook to retrieve filter options and applies them through the FilterItem component. The handleOnClick function is used to apply a selected filter option, and the FilterSwitch component is used to toggle the display of named versions [1].

Additionally, the code structure for handling metadata filters involves functions that define how to extract and filter file metadata based on specified keys and conditions. These functions use Python's match statement to determine the appropriate operation based on the provided key or condition [2].

Given that the filters work in the Chatbot but not in the workflow, it might be worth checking if there are any discrepancies in how the filters are configured or applied in the workflow component. You may also want to verify if there are any differences in the input data or conditions used in both components.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Mar 21, 2025): <!-- Greeting --> Hello @gemchen! I'm here to help you with any bugs, questions, or contributions you have in mind. Let's tackle this issue together! <!-- Answer --> The issue you're experiencing with metadata filters not working in the workflow component, while they work in the Chatbot, might be related to how the filters are implemented or applied in the workflow component. In the workflow component, metadata filters are applied in the file `web/app/components/workflow/panel/version-history-panel/filter/index.tsx`. The filtering logic is handled by the `Filter` component, which uses the `useFilterOptions` hook to retrieve filter options and applies them through the `FilterItem` component. The `handleOnClick` function is used to apply a selected filter option, and the `FilterSwitch` component is used to toggle the display of named versions <sup>[[1]](https://github.com/langgenius/dify/blob/main/web/app/components/workflow/panel/version-history-panel/filter/index.tsx)</sup>. Additionally, the code structure for handling metadata filters involves functions that define how to extract and filter file metadata based on specified keys and conditions. These functions use Python's `match` statement to determine the appropriate operation based on the provided key or condition <sup>[[2]](https://github.com/langgenius/dify/blob/main/api/core/workflow/nodes/list_operator/node.py)</sup>. Given that the filters work in the Chatbot but not in the workflow, it might be worth checking if there are any discrepancies in how the filters are configured or applied in the workflow component. You may also want to verify if there are any differences in the input data or conditions used in both components. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/b4ca96db-f8d8-4bac-85bd-c2402144fc24?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/b4ca96db-f8d8-4bac-85bd-c2402144fc24?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/b4ca96db-f8d8-4bac-85bd-c2402144fc24?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/b4ca96db-f8d8-4bac-85bd-c2402144fc24?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/b4ca96db-f8d8-4bac-85bd-c2402144fc24?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/b4ca96db-f8d8-4bac-85bd-c2402144fc24?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/b4ca96db-f8d8-4bac-85bd-c2402144fc24?feedback_type=other)</sup>
yindo commented 2026-02-21 19:04:55 -05:00
Author
Owner
Copy Link
Copy Source

@JohnJyong commented on GitHub (Mar 21, 2025):

@gemchen could pls provide more detail of this issue, such as some screen shot ? thanks

@JohnJyong commented on GitHub (Mar 21, 2025): @gemchen could pls provide more detail of this issue, such as some screen shot ? thanks
yindo commented 2026-02-21 19:04:55 -05:00
Author
Owner
Copy Link
Copy Source

@gemchen commented on GitHub (Mar 21, 2025):

Bad news metadata flow in both my chatbot and workflow are not work now.

Workflow case bad case

  1. I created a very simple workflow and set a filter with "[document_name] [start with] [Singapore]"
Image 2. I make a query, and you can find the output with document name NOT start with "Singapore" Image

Chatbot case

I created a chatbot with same kb and metadata filter and make a query. Expect reference documents are named start with "Singapore" but which is not.

Image
@gemchen commented on GitHub (Mar 21, 2025): Bad news metadata flow in both my chatbot and workflow are not work now. **Workflow case** bad case 1. I created a very simple workflow and set a filter with "[document_name] [start with] [Singapore]" <img width="947" alt="Image" src="https://github.com/user-attachments/assets/f6db2606-75d0-4187-a6b6-fe7776f414de" /> 2. I make a query, and you can find the output with document name NOT start with "Singapore" <img width="433" alt="Image" src="https://github.com/user-attachments/assets/d79935a4-cc35-4f43-97a1-992e8353c199" /> **Chatbot case** I created a chatbot with same kb and metadata filter and make a query. Expect reference documents are named start with "Singapore" but which is not. <img width="1125" alt="Image" src="https://github.com/user-attachments/assets/d347c5ec-5316-4ba3-b040-37fe4c4e7995" />
yindo commented 2026-02-21 19:04:55 -05:00
Author
Owner
Copy Link
Copy Source

@stoplyy commented on GitHub (Mar 21, 2025):

version 1.1.1

Image

Image

@stoplyy commented on GitHub (Mar 21, 2025): version 1.1.1 ![Image](https://github.com/user-attachments/assets/9ba6ab58-b409-4d40-afc8-782afd8cb7c6) ![Image](https://github.com/user-attachments/assets/b2e753e0-9d74-4fdb-b77e-fd8674b33949)
yindo commented 2026-02-21 19:04:55 -05:00
Author
Owner
Copy Link
Copy Source

@gemchen commented on GitHub (Mar 21, 2025):

I noticed in the _retriever function in dataset_restrival.py link
the parameter "metadata_condition" not used in the else branch of dify provider (maybe 'vender'), is this the cause?

Image
@gemchen commented on GitHub (Mar 21, 2025): I noticed in the _retriever function in dataset_restrival.py [link](https://github.com/langgenius/dify/blob/0e2e2db3fa1745c451c5bdcd20e79d35ae2fb46a/api/core/rag/retrieval/dataset_retrieval.py#L570) the parameter "metadata_condition" not used in the else branch of dify provider (maybe 'vender'), is this the cause? <img width="938" alt="Image" src="https://github.com/user-attachments/assets/cc1b981a-2031-4731-b560-4e22a06605e3" />
yindo commented 2026-02-21 19:04:55 -05:00
Author
Owner
Copy Link
Copy Source

@rabbit1753 commented on GitHub (Mar 24, 2025):

Same Problem.
I customized a metadata field: series. I want to filter out only the documents where the series field contains 'GS', but the retrieval output includes documents where the field is not 'GS'.

Also self hosted (source)

Image
Image

@rabbit1753 commented on GitHub (Mar 24, 2025): Same Problem. I customized a metadata field: series. I want to filter out only the documents where the series field contains 'GS', but the retrieval output includes documents where the field is not 'GS'. Also self hosted (source) ![Image](https://github.com/user-attachments/assets/efe508d3-9b23-41a7-8f08-ee647502ff06) ![Image](https://github.com/user-attachments/assets/5a4a70f2-293e-4839-b399-e813f63e30e8)
yindo commented 2026-02-21 19:04:55 -05:00
Author
Owner
Copy Link
Copy Source

@JohnJyong commented on GitHub (Mar 24, 2025):

thanks @rabbit1753 @gemchen @stoplyy for your reminder, we have fixed the metadata filter is not affect in keyword and full-text search , the pr has linked in the issue .

@JohnJyong commented on GitHub (Mar 24, 2025): thanks @rabbit1753 @gemchen @stoplyy for your reminder, we have fixed the metadata filter is not affect in keyword and full-text search , the pr has linked in the issue .
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🌊 feat:workflow
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#11950
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?