langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Knowledge Retrieval with external knowledge api response metadata #7091

New Issue
Closed
opened 2026-02-21 18:18:52 -05:00 by yindo · 3 comments
yindo commented 2026-02-21 18:18:52 -05:00
Owner
Copy Link
Copy Source

Originally created by @yuhaowin on GitHub (Dec 6, 2024).

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

I'm a workflow base app api user, and connect knowledge base with external knowledge api, I really except /chat-messages api could response external knowledge api provided metadata such as path in retriever_resources field.

{
    "records": [
        {
            "metadata": {
                "path": "s3://dify/knowledge.txt",
                "description": "dify knowledge document"
            },
            "score": 0.98,
            "title": "knowledge.txt",
            "content": "This is the document for external knowledge."
        }
    ]
}

records.metadata.path -> metadata.retriever_resources.path

{
    "event": "message",
    "task_id": "xxxx",
    "id": "xxxx",
    "message_id": "xxxx",
    "conversation_id": "xxxx",
    "mode": "advanced-chat",
    "answer": "xxxx",
    "metadata": {
        "retriever_resources": [
            {
                "position": 1,
                "dataset_id": "xxxx",
                "dataset_name": "xxxx",
                "document_id": null,
                "document_name": "xxxx",
                "data_source_type": null,
                "segment_id": null,
                "retriever_from": "workflow",
                "score": 0.9943581,
                "hit_count": null,
                "word_count": null,
                "segment_position": null,
                "index_node_hash": null,
                "content": "",
                "page": null
            }
        ],
        "usage": {
        }
    },
    "created_at": 1733464931
}

2. Additional context or comments

No response

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @yuhaowin on GitHub (Dec 6, 2024). ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. I'm a workflow base app api user, and connect knowledge base with external knowledge api, I really except **/chat-messages** api could response external knowledge api provided metadata such as **path** in **retriever_resources** field. ``` { "records": [ { "metadata": { "path": "s3://dify/knowledge.txt", "description": "dify knowledge document" }, "score": 0.98, "title": "knowledge.txt", "content": "This is the document for external knowledge." } ] } ``` **records.metadata.path -> metadata.retriever_resources.path** ``` { "event": "message", "task_id": "xxxx", "id": "xxxx", "message_id": "xxxx", "conversation_id": "xxxx", "mode": "advanced-chat", "answer": "xxxx", "metadata": { "retriever_resources": [ { "position": 1, "dataset_id": "xxxx", "dataset_name": "xxxx", "document_id": null, "document_name": "xxxx", "data_source_type": null, "segment_id": null, "retriever_from": "workflow", "score": 0.9943581, "hit_count": null, "word_count": null, "segment_position": null, "index_node_hash": null, "content": "", "page": null } ], "usage": { } }, "created_at": 1733464931 } ``` ### 2. Additional context or comments _No response_ ### 3. Can you help us with this feature? - [ ] I am interested in contributing to this feature.
yindo added the 💪 enhancement🌊 feat:workflow labels 2026-02-21 18:18:52 -05:00
yindo closed this issue 2026-02-21 18:18:52 -05:00
yindo commented 2026-02-21 18:18:52 -05:00
Author
Owner
Copy Link
Copy Source

@yuhaowin commented on GitHub (Dec 6, 2024):

response external_document['metadate']

https://github.com/langgenius/dify/blob/a8d32f99644859949e434070a5c80d5269b3814c/api/core/workflow/nodes/knowledge_retrieval/knowledge_retrieval_node.py#L180-L194

@yuhaowin commented on GitHub (Dec 6, 2024): response **external_document['metadate']** https://github.com/langgenius/dify/blob/a8d32f99644859949e434070a5c80d5269b3814c/api/core/workflow/nodes/knowledge_retrieval/knowledge_retrieval_node.py#L180-L194
yindo commented 2026-02-21 18:18:52 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 6, 2025):

Hi, @yuhaowin. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

  • Enhancement request for /chat-messages API to include "path" metadata in retriever_resources.
  • This would improve integration with external knowledge APIs.
  • You provided a code link to illustrate the metadata handling section.

Next Steps:

  • Is this issue still relevant to the latest version of the Dify repository? If so, please comment to keep the discussion open.
  • Otherwise, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Jan 6, 2025): Hi, @yuhaowin. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog. I'm marking this issue as stale. **Issue Summary:** - Enhancement request for **/chat-messages** API to include "path" metadata in **retriever_resources**. - This would improve integration with external knowledge APIs. - You provided a code link to illustrate the metadata handling section. **Next Steps:** - Is this issue still relevant to the latest version of the Dify repository? If so, please comment to keep the discussion open. - Otherwise, this issue will be automatically closed in 15 days. Thank you for your understanding and contribution!
yindo commented 2026-02-21 18:18:53 -05:00
Author
Owner
Copy Link
Copy Source

@AlaaSenjab commented on GitHub (Feb 14, 2025):

response external_document['metadate']

dify/api/core/workflow/nodes/knowledge_retrieval/knowledge_retrieval_node.py

Lines 180 to 194 in a8d32f9

for item in external_documents:
source = {
"metadata": {
"_source": "knowledge",
"dataset_id": item.metadata.get("dataset_id"),
"dataset_name": item.metadata.get("dataset_name"),
"document_name": item.metadata.get("title"),
"data_source_type": "external",
"retriever_from": "workflow",
"score": item.metadata.get("score"),
},
"title": item.metadata.get("title"),
"content": item.page_content,
}
retrieval_resource_list.append(source)

as a temp fix:

            source = {
                "metadata": {
                    "_source": "knowledge",
                    # "dataset_id": item.metadata.get("dataset_id"),
                    # "dataset_name": item.metadata.get("dataset_name"),
                    "document_name": item.metadata.get("title"),
                    "data_source_type": "external",
                    "retriever_from": "workflow",
                    # "score": item.metadata.get("score"),
                    # unpack item metadata dict to metadata
                    **item.metadata,
                },
                "title": item.metadata.get("title"),
                "content": item.page_content,
            }
@AlaaSenjab commented on GitHub (Feb 14, 2025): > response **external_document['metadate']** > > [dify/api/core/workflow/nodes/knowledge_retrieval/knowledge_retrieval_node.py](https://github.com/langgenius/dify/blob/a8d32f99644859949e434070a5c80d5269b3814c/api/core/workflow/nodes/knowledge_retrieval/knowledge_retrieval_node.py#L180-L194) > > Lines 180 to 194 in [a8d32f9](/langgenius/dify/commit/a8d32f99644859949e434070a5c80d5269b3814c) > > for item in external_documents: > source = { > "metadata": { > "_source": "knowledge", > "dataset_id": item.metadata.get("dataset_id"), > "dataset_name": item.metadata.get("dataset_name"), > "document_name": item.metadata.get("title"), > "data_source_type": "external", > "retriever_from": "workflow", > "score": item.metadata.get("score"), > }, > "title": item.metadata.get("title"), > "content": item.page_content, > } > retrieval_resource_list.append(source) as a temp fix: ``` source = { "metadata": { "_source": "knowledge", # "dataset_id": item.metadata.get("dataset_id"), # "dataset_name": item.metadata.get("dataset_name"), "document_name": item.metadata.get("title"), "data_source_type": "external", "retriever_from": "workflow", # "score": item.metadata.get("score"), # unpack item metadata dict to metadata **item.metadata, }, "title": item.metadata.get("title"), "content": item.page_content, } ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🌊 feat:workflow 💪 enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#7091
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?