langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 03:55:25 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Support Custom Plugins to Inject retriever_resources into ChatFlow for Citations and Attributions #21132

New Issue
Open
opened 2026-02-21 20:11:03 -05:00 by yindo · 7 comments
yindo commented 2026-02-21 20:11:03 -05:00
Owner
Copy Link
Copy Source

Originally created by @MrXionGe on GitHub (Dec 17, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

As is well known, after creating a ChatFlow, if we enable Citations and Attributions and add a Knowledge Retrieval node, the system retrieves relevant segments and documents from the built-in knowledge base. The retrieved results—specifically the retriever_resources field—are then included in the response message, enabling the frontend client to display which segments and documents were referenced.

However, our documents are not limited to Dify’s built-in knowledge base; they may reside in external, self-hosted knowledge systems (e.g., RAGFlow or in-house enterprise knowledge bases). To accommodate this, I intend to develop a custom knowledge retrieval plugin. My goal is for this plugin to be able to report the retrieved segments and documents back to the ChatFlow, populating the retriever_resources field just like the native Knowledge Retrieval node does.

I’m unsure whether this is technically feasible—specifically, whether the ChatFlow engine supports third-party plugins injecting retriever_resources for citation and attribution purposes.

Could the Dify team clarify whether this extension is possible, and if so, what interfaces or conventions such a plugin would need to follow?

2. Additional context or comments

No response

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @MrXionGe on GitHub (Dec 17, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. As is well known, after creating a **ChatFlow**, if we enable **Citations and Attributions** and add a **Knowledge Retrieval** node, the system retrieves relevant segments and documents from the built-in knowledge base. The retrieved results—specifically the `retriever_resources` field—are then included in the response message, enabling the frontend client to display which segments and documents were referenced. However, our documents are not limited to Dify’s built-in knowledge base; they may reside in external, self-hosted knowledge systems (e.g., RAGFlow or in-house enterprise knowledge bases). To accommodate this, I intend to develop a custom knowledge retrieval plugin. My goal is for this plugin to be able to report the retrieved segments and documents back to the **ChatFlow**, populating the `retriever_resources` field just like the native **Knowledge Retrieval** node does. I’m unsure whether this is technically feasible—specifically, whether the **ChatFlow** engine supports third-party plugins injecting `retriever_resources` for citation and attribution purposes. Could the Dify team clarify whether this extension is possible, and if so, what interfaces or conventions such a plugin would need to follow? ### 2. Additional context or comments _No response_ ### 3. Can you help us with this feature? - [ ] I am interested in contributing to this feature.
yindo added the 💪 enhancement label 2026-02-21 20:11:03 -05:00
yindo commented 2026-02-21 20:11:04 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Dec 17, 2025):

Yes, Dify's ChatFlow engine supports custom plugins injecting retriever_resources for citations and attributions, similar to the native Knowledge Retrieval node. The mechanism is event-driven: any code (including a plugin) that can access the AppQueueManager.publish method and emit a QueueRetrieverResourcesEvent with the correct metadata shape can inject citation resources into the workflow. The required metadata should match what the native node uses—fields like _source: "knowledge", dataset_id, document_id, score, content, and so on. These resources are then merged into the response metadata and displayed as citations in the frontend client. You can see how the native node builds and emits these resources here, and how the event is handled and published here.

For plugin development, you can use the pattern in DatasetIndexToolCallbackHandler.return_retriever_resource_info to publish your own retriever resources as a list of RetrievalSourceMetadata objects via QueueRetrieverResourcesEvent (reference). There is no explicit, documented plugin interface solely for this, but the event system is designed to be extensible.

If you want to hook deeper into node execution, the new GraphEngine "layers" extensibility mechanism lets plugins hook into node execution events like on_node_run_start and on_node_run_end, which could be used to inject custom behavior or resources (reference).

No major limitations surfaced in the code or issues, but ensure your resources follow the metadata conventions and avoid duplicates, as the merging logic in MessageCycleManager expects unique dataset/document pairs (reference).

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Chat with Dosu Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Dec 17, 2025): <!-- Answer --> Yes, Dify's ChatFlow engine supports custom plugins injecting `retriever_resources` for citations and attributions, similar to the native Knowledge Retrieval node. The mechanism is event-driven: any code (including a plugin) that can access the `AppQueueManager.publish` method and emit a `QueueRetrieverResourcesEvent` with the correct metadata shape can inject citation resources into the workflow. The required metadata should match what the native node uses—fields like `_source: "knowledge"`, `dataset_id`, `document_id`, `score`, `content`, and so on. These resources are then merged into the response metadata and displayed as citations in the frontend client. You can see how the native node builds and emits these resources [here](https://github.com/langgenius/dify/blob/acbeb04edc8b507ffeb418c527861d6e39a6ce6a/api/core/workflow/nodes/knowledge_retrieval/knowledge_retrieval_node.py#L85-L431), and how the event is handled and published [here](https://github.com/langgenius/dify/blob/acbeb04edc8b507ffeb418c527861d6e39a6ce6a/api/core/app/apps/workflow_app_runner.py#L347-L573). For plugin development, you can use the pattern in `DatasetIndexToolCallbackHandler.return_retriever_resource_info` to publish your own retriever resources as a list of `RetrievalSourceMetadata` objects via `QueueRetrieverResourcesEvent` ([reference](https://github.com/langgenius/dify/blob/acbeb04edc8b507ffeb418c527861d6e39a6ce6a/api/core/callback_handler/index_tool_callback_handler.py#L19-L96)). There is no explicit, documented plugin interface solely for this, but the event system is designed to be extensible. If you want to hook deeper into node execution, the new GraphEngine "layers" extensibility mechanism lets plugins hook into node execution events like `on_node_run_start` and `on_node_run_end`, which could be used to inject custom behavior or resources ([reference](https://github.com/langgenius/dify/pull/28583)). No major limitations surfaced in the code or issues, but ensure your resources follow the metadata conventions and avoid duplicates, as the merging logic in `MessageCycleManager` expects unique dataset/document pairs ([reference](https://github.com/langgenius/dify/blob/acbeb04edc8b507ffeb418c527861d6e39a6ce6a/api/core/app/task_pipeline/message_cycle_manager.py#L147-L177)). <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/61fefa45-0b69-4732-8a6c-53c29a14a22c?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/61fefa45-0b69-4732-8a6c-53c29a14a22c?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/61fefa45-0b69-4732-8a6c-53c29a14a22c?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/61fefa45-0b69-4732-8a6c-53c29a14a22c?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/61fefa45-0b69-4732-8a6c-53c29a14a22c?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/61fefa45-0b69-4732-8a6c-53c29a14a22c?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/61fefa45-0b69-4732-8a6c-53c29a14a22c?feedback_type=other)</sup> [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=Yes%2C%20Dify%27s%20ChatFlow%20engine%20supports%20custom%20plugins%20injecting%20%60retriever_resources%60%20for%20citations%20and%20attributions%2C%20similar%20to%20the%20native%20Knowledge%20Retrieval%20node.%20The%20mechanism%20is%20event-driven%3A%20any%20code%20%28including%20a%20plugin%29%20that%20can%20access%20the%20%60AppQueueManager.publish%60%20method%20and%20emit%20a%20%60QueueRetrieverResourcesEvent%60%20with%20the%20correct%20metadata%20shape%20can%20inject%20citation%20resources%20into%20the%20workflow.%20The%20required%20metadata%20should%20match%20what%20the%20native%20node%20uses%E2%80%94fields%20like%20%60_source%3A%20%22knowledge%22%60%2C%20%60dataset_id%60%2C%20%60document_id%60%2C%20%60score%60%2C%20%60content%60%2C%20and%20so%20on.%20These%20resources%20are%20then%20merged%20into%20the%20response%20metadata%20and%20displayed%20as%20citations%20in%20the%20frontend%20client.%20You%20can%20see%20how%20the%20native%20node%20builds%20and%20emits%20these%20resources%20%5Bhere%5D%28https%3A//github.com/langgenius/dify/blob/acbeb04edc8b507ffeb418c527861d6e39a6ce6a/api/core/workflow/nodes/knowledge_retrieval/knowledge_retrieval_node.py%23L85-L431%29%2C%20and%20how%20the%20event%20is%20handled%20and%20published%20%5Bhere%5D%28https%3A//github.com/langgenius/dify/blob/acbeb04edc8b507ffeb418c527861d6e39a6ce6a/api/core/app/apps/workflow_app_runner.py%23L347-L573%29.%0A%0AFor%20plugin%20development%2C%20you%20can%20use%20the%20pattern%20in%20%60DatasetIndexToolCallbackHandler.return_retriever_resource_info%60%20to%20publish%20your%20own%20retriever%20resources%20as%20a%20list%20of%20%60RetrievalSourceMetadata%60%20objects%20via%20%60QueueRetrieverResourcesEvent%60%20%28%5Breference%5D%28https%3A//github.com/langgenius/dify/blob/acbeb04edc8b507ffeb418c527861d6e39a6ce6a/api/core/callback_handler/index_tool_callback_handler.py%23L19-L96%29%29.%20There%20is%20no%20explicit%2C%20documented%20plugin%20interface%20solely%20for%20this%2C%20but%20the%20event%20system%20is%20designed%20to%20be%20extensible.%0A%0AIf%20you%20want%20to%20hook%20deeper%20into%20node%20execution%2C%20the%20new%20GraphEngine%20%22layers%22%20extensibility%20mechanism%20lets%20plugins%20hook%20into%20node%20execution%20events%20like%20%60on_node_run_start%60%20and%20%60on_node_run_end%60%2C%20which%20could%20be%20used%20to%20inject%20custom%20behavior%20or%20resources%20%28%5Breference%5D%28https%3A//github.com/langgenius/dify/pull/28583%29%29.%0A%0ANo%20major%20limitations%20surfaced%20in%20the%20code%20or%20issues%2C%20but%20ensure%20your%20resources%20follow%20the%20metadata%20conventions%20and%20avoid%20duplicates%2C%20as%20the%20merging%20logic%20in%20%60MessageCycleManager%60%20expects%20unique%20dataset/document%20pairs%20%28%5Breference%5D%28https%3A//github.com/langgenius/dify/blob/acbeb04edc8b507ffeb418c527861d6e39a6ce6a/api/core/app/task_pipeline/message_cycle_manager.py%23L147-L177%29%29.)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/29832)
yindo commented 2026-02-21 20:11:04 -05:00
Author
Owner
Copy Link
Copy Source

@MrXionGe commented on GitHub (Dec 17, 2025):

Thank you for your guidance. @dosu
Please do not close this issue for now—thank you again. @crazywoola

I will follow up based on the approach you suggested. If I encounter any issues or make progress, I will provide timely updates here. I also hope this issue can serve as a helpful reference for others with similar needs.

@MrXionGe commented on GitHub (Dec 17, 2025): Thank you for your guidance. @dosu Please do not close this issue for now—thank you again. @crazywoola I will follow up based on the approach you suggested. If I encounter any issues or make progress, I will provide timely updates here. I also hope this issue can serve as a helpful reference for others with similar needs.
yindo commented 2026-02-21 20:11:04 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Dec 17, 2025):

You're welcome! The issue will stay open—feel free to update here with any progress or questions. Looking forward to hearing how it goes, and this thread will definitely help others with similar needs.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Chat with Dosu Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Dec 17, 2025): <!-- Answer --> You're welcome! The issue will stay open—feel free to update here with any progress or questions. Looking forward to hearing how it goes, and this thread will definitely help others with similar needs. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/70c0cb87-05c2-4303-858c-ffe9f70ff3c0?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/70c0cb87-05c2-4303-858c-ffe9f70ff3c0?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/70c0cb87-05c2-4303-858c-ffe9f70ff3c0?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/70c0cb87-05c2-4303-858c-ffe9f70ff3c0?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/70c0cb87-05c2-4303-858c-ffe9f70ff3c0?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/70c0cb87-05c2-4303-858c-ffe9f70ff3c0?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/70c0cb87-05c2-4303-858c-ffe9f70ff3c0?feedback_type=other)</sup> [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=You%27re%20welcome%21%20The%20issue%20will%20stay%20open%E2%80%94feel%20free%20to%20update%20here%20with%20any%20progress%20or%20questions.%20Looking%20forward%20to%20hearing%20how%20it%20goes%2C%20and%20this%20thread%20will%20definitely%20help%20others%20with%20similar%20needs.)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/29832)
yindo commented 2026-02-21 20:11:04 -05:00
Author
Owner
Copy Link
Copy Source

@MrXionGe commented on GitHub (Dec 22, 2025):

@crazywoola

Hello, I’d like to ask: To implement the “Citations and Attributions” feature for the knowledge retrieval plugin, does Dify itself need to add any additional functionality?

Previously, I attempted this based on version 1.9.1, and after retrieval, the logs showed the warning:
WARNING [GraphDispatcher] [event_handlers.py:102] - Unhandled event type: NodeRunRetrieverResourceEvent.

Later, I upgraded to version 1.11.1, and this warning disappeared; however, the “Citations and Attributions” still do not appear.

@MrXionGe commented on GitHub (Dec 22, 2025): @crazywoola Hello, I’d like to ask: To implement the “Citations and Attributions” feature for the knowledge retrieval plugin, does Dify itself need to add any additional functionality? Previously, I attempted this based on version 1.9.1, and after retrieval, the logs showed the warning: `WARNING [GraphDispatcher] [event_handlers.py:102] - Unhandled event type: NodeRunRetrieverResourceEvent`. Later, I upgraded to version 1.11.1, and this warning disappeared; however, the “Citations and Attributions” still do not appear.
yindo commented 2026-02-21 20:11:04 -05:00
Author
Owner
Copy Link
Copy Source

@MrXionGe commented on GitHub (Dec 22, 2025):

@crazywoola

Clearly, some functional enhancements are needed in Dify.

Based on version 1.11.1, I modified the file:

https://github.com/langgenius/dify/blob/1.11.1/api/core/workflow/nodes/tool/tool_node.py

  • Changed line 23 to:

    from core.workflow.node_events import NodeEventBase, NodeRunResult, RunRetrieverResourceEvent, StreamChunkEvent, StreamCompletedEvent

  • Added a check for the RETRIEVER_RESOURCES message type at line 401:

    elif message.type == ToolInvokeMessage.MessageType.RETRIEVER_RESOURCES:
    assert isinstance(message.message, ToolInvokeMessage.RetrieverResourceMessage)
    yield RunRetrieverResourceEvent(
        retriever_resources=message.message.retriever_resources,
        context=message.message.context,
    )

After testing, it can successfully receive, process, and display Citations and Attributions.

I’m not entirely familiar with Dify’s pull request (PR) workflow and contribution guidelines, so I may not be able to submit a perfect PR myself. However, I’m happy to share the modified file I’ve prepared, and I hope someone can help submit a PR—let’s work together to improve Dify!

tool_node.py

@MrXionGe commented on GitHub (Dec 22, 2025): @crazywoola Clearly, some functional enhancements are needed in Dify. Based on version 1.11.1, I modified the file: ``` https://github.com/langgenius/dify/blob/1.11.1/api/core/workflow/nodes/tool/tool_node.py ``` - Changed line 23 to: ```python from core.workflow.node_events import NodeEventBase, NodeRunResult, RunRetrieverResourceEvent, StreamChunkEvent, StreamCompletedEvent ``` - Added a check for the `RETRIEVER_RESOURCES` message type at line 401: ```python elif message.type == ToolInvokeMessage.MessageType.RETRIEVER_RESOURCES: assert isinstance(message.message, ToolInvokeMessage.RetrieverResourceMessage) yield RunRetrieverResourceEvent( retriever_resources=message.message.retriever_resources, context=message.message.context, ) ``` After testing, it can successfully receive, process, and display **Citations and Attributions**. I’m not entirely familiar with Dify’s pull request (PR) workflow and contribution guidelines, so I may not be able to submit a perfect PR myself. However, I’m happy to share the modified file I’ve prepared, and I hope someone can help submit a PR—let’s work together to improve Dify! [tool_node.py](https://github.com/user-attachments/files/24286844/tool_node.py)
yindo commented 2026-02-21 20:11:04 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 22, 2026):

Hi, @MrXionGe. I'm Dosu, and I'm helping the Dify team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You requested support for custom plugins in ChatFlow to inject retriever_resources for citations similar to the built-in Knowledge Retrieval node.
  • The event-driven system supports this via QueueRetrieverResourcesEvent, with shared code references for extensibility.
  • You reported a warning in version 1.9.1 that disappeared by 1.11.1, but citations still did not show.
  • You shared a code modification to tool_node.py that enabled proper citation display and offered to share your patch for a potential PR.
  • The issue remains open without a merged resolution or further updates.

Next Steps:

  • Please let me know if this issue is still relevant with the latest version of Dify by commenting here.
  • If I do not hear back within 15 days, I will automatically close this issue.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Jan 22, 2026): Hi, @MrXionGe. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog and am marking this issue as stale. **Issue Summary:** - You requested support for custom plugins in ChatFlow to inject `retriever_resources` for citations similar to the built-in Knowledge Retrieval node. - The event-driven system supports this via `QueueRetrieverResourcesEvent`, with shared code references for extensibility. - You reported a warning in version 1.9.1 that disappeared by 1.11.1, but citations still did not show. - You shared a code modification to `tool_node.py` that enabled proper citation display and offered to share your patch for a potential PR. - The issue remains open without a merged resolution or further updates. **Next Steps:** - Please let me know if this issue is still relevant with the latest version of Dify by commenting here. - If I do not hear back within 15 days, I will automatically close this issue. Thank you for your understanding and contribution!
yindo commented 2026-02-21 20:11:04 -05:00
Author
Owner
Copy Link
Copy Source

@MrXionGe commented on GitHub (Jan 22, 2026):

@dosu
Hello, as of version 1.11.4, this issue still persists. I also noticed that tool_node.py hasn't been updated for many versions. I plan to submit a PR soon to try to fix this issue.

@MrXionGe commented on GitHub (Jan 22, 2026): @dosu Hello, as of version 1.11.4, this issue still persists. I also noticed that `tool_node.py` hasn't been updated for many versions. I plan to submit a PR soon to try to fix this issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 💪 enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#21132
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?