langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 04:25:29 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

DataInsight Knowledge base Integration #14996

New Issue
Closed
opened 2026-02-21 19:19:42 -05:00 by yindo · 2 comments
yindo commented 2026-02-21 19:19:42 -05:00
Owner
Copy Link
Copy Source

Originally created by @wpf710 on GitHub (Jun 30, 2025).

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

DataInsight Knowedge base

Function Description:
we'v developed a thirdparty middleware, which can povide the following features
1 it's a standalone software and provide the funtional RestAPI
2 it can manage the meta-data of the S3 object in bucket, and our high performance distributed file system YRCloudFile (https://www.yanrongyun.com/en-US), and will support the NFS,SMB and etc. anyway it's not bundled with our private YRCloudFile system.

3 it can provide the very fast query performance for the metadata, such as you can find the the files or s3 object that access time is older than three months and larger than 1GB size, of course ,you can combile other meta-fields that the data source supported.

4 and according to the user defined query criteria, it can push back the file list or file content to some call-back endpoint ,such as the knowledge base. either the existing documment or the incremental documents that match the pre-defined query criteria will be in the knowledge base , no manully upload again

5 for the S3 bucket (and others that supported), DataInsight can get the source meta-data changes in time and fanout the changes to ,such as knowledge base .

we have already implement the integration with FastGPT (https://github.com/labring/FastGPT) by adding an DataInsight Knowledge base type, and other facilities, when user create a knowledge base ,it can choose the type of DataInsight ,and provide some parameters of DataInsight service endpoint ,token and etc in the UI page,

so if we also want to implement it in Dify, it will be accepted?

2. Additional context or comments

No response

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @wpf710 on GitHub (Jun 30, 2025). ### Self Checks - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. DataInsight Knowedge base Function Description: we'v developed a thirdparty middleware, which can povide the following features 1 it's a standalone software and provide the funtional RestAPI 2 it can manage the meta-data of the S3 object in bucket, and our high performance distributed file system YRCloudFile (https://www.yanrongyun.com/en-US), and will support the NFS,SMB and etc. anyway it's not bundled with our private YRCloudFile system. 3 it can provide the very fast query performance for the metadata, such as you can find the the files or s3 object that access time is older than three months and larger than 1GB size, of course ,you can combile other meta-fields that the data source supported. 4 and according to the user defined query criteria, it can push back the file list or file content to some call-back endpoint ,such as the knowledge base. either the existing documment or the incremental documents that match the pre-defined query criteria will be in the knowledge base , no manully upload again 5 for the S3 bucket (and others that supported), DataInsight can get the source meta-data changes in time and fanout the changes to ,such as knowledge base . we have already implement the integration with FastGPT (https://github.com/labring/FastGPT) by adding an DataInsight Knowledge base type, and other facilities, when user create a knowledge base ,it can choose the type of DataInsight ,and provide some parameters of DataInsight service endpoint ,token and etc in the UI page, so if we also want to implement it in Dify, it will be accepted? ### 2. Additional context or comments _No response_ ### 3. Can you help us with this feature? - [x] I am interested in contributing to this feature.
yindo added the 💪 enhancement👻 feat:rag labels 2026-02-21 19:19:42 -05:00
yindo closed this issue 2026-02-21 19:19:43 -05:00
yindo commented 2026-02-21 19:19:43 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Aug 7, 2025):

Sure, but I suggest implementing this during Oct. We are doing some refactoring at this moment.

@crazywoola commented on GitHub (Aug 7, 2025): Sure, but I suggest implementing this during Oct. We are doing some refactoring at this moment.
yindo commented 2026-02-21 19:19:43 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Sep 8, 2025):

Hi, @wpf710. I'm Dosu, and I'm helping the Dify team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You proposed integrating your third-party middleware, DataInsight, as a new knowledge base type in Dify.
  • DataInsight offers high-performance metadata management and real-time sync features.
  • You have successfully integrated DataInsight with FastGPT and are seeking approval for similar integration in Dify.
  • A maintainer expressed positive interest but suggested scheduling the implementation for October due to ongoing refactoring.
  • The issue remains unresolved and no recent updates have been provided.

Next Steps:

  • Please let me know if this integration is still relevant to the latest version of Dify by commenting on this issue.
  • If I do not hear back within 15 days, I will automatically close this issue.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Sep 8, 2025): Hi, @wpf710. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog and am marking this issue as stale. **Issue Summary:** - You proposed integrating your third-party middleware, DataInsight, as a new knowledge base type in Dify. - DataInsight offers high-performance metadata management and real-time sync features. - You have successfully integrated DataInsight with FastGPT and are seeking approval for similar integration in Dify. - A maintainer expressed positive interest but suggested scheduling the implementation for October due to ongoing refactoring. - The issue remains unresolved and no recent updates have been provided. **Next Steps:** - Please let me know if this integration is still relevant to the latest version of Dify by commenting on this issue. - If I do not hear back within 15 days, I will automatically close this issue. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 👻 feat:rag 💪 enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#14996
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?