langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Feat: Supports read and write separation in postgresql #6337

New Issue
Closed
opened 2026-02-21 18:15:17 -05:00 by yindo · 2 comments
yindo commented 2026-02-21 18:15:17 -05:00
Owner
Copy Link
Copy Source

Originally created by @luckylhb90 on GitHub (Oct 25, 2024).

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Because within our team, we've been heavily promoting the use of DIFY internally.
One of the challenges is that single-machine postgresql can't handle the volume of requests.
The postgresql is faulty for many times.

So we need to support postgresql's read-write separation feature.
But I don't think that's enough to solve the problem completely. But at the moment, it seems that this may help to improve the current problem.

2. Additional context or comments

Further thoughts are:
Database sharding and splitting, service decomposition , use mq, etc...
These are all aspects that could be considered.

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @luckylhb90 on GitHub (Oct 25, 2024). ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. Because within our team, we've been heavily promoting the use of DIFY internally. One of the challenges is that single-machine postgresql can't handle the volume of requests. The postgresql is faulty for many times. So we need to support postgresql's read-write separation feature. But I don't think that's enough to solve the problem completely. But at the moment, it seems that this may help to improve the current problem. ### 2. Additional context or comments Further thoughts are: Database sharding and splitting, service decomposition , use mq, etc... These are all aspects that could be considered. ### 3. Can you help us with this feature? - [ ] I am interested in contributing to this feature.
yindo added the 💪 enhancement label 2026-02-21 18:15:17 -05:00
yindo closed this issue 2026-02-21 18:15:17 -05:00
yindo commented 2026-02-21 18:15:17 -05:00
Author
Owner
Copy Link
Copy Source

@luckylhb90 commented on GitHub (Nov 3, 2024):

Based on our usage, I also want to make some additions:
1: pgsql
There are a few tables we need to focus on, their data volume is growing rapidly, we need to consider how to optimize:
document_segments
dataset_keyword_tables
conversations
messages
workflow_runs
workflow_node_executions

2: redis
redis grew too fast because there was no specific understanding of the logic of the relevant code, and it was uncertain whether it was being used correctly.

image

3: llm
Although the load balancing policy is now supported, the retry policy is missing because we have found that there are times when the model is not stable and it might be better to try again.

@luckylhb90 commented on GitHub (Nov 3, 2024): Based on our usage, I also want to make some additions: 1: pgsql There are a few tables we need to focus on, their data volume is growing rapidly, we need to consider how to optimize: document_segments dataset_keyword_tables conversations messages workflow_runs workflow_node_executions 2: redis redis grew too fast because there was no specific understanding of the logic of the relevant code, and it was uncertain whether it was being used correctly. <img width="308" alt="image" src="https://github.com/user-attachments/assets/d90a067a-4048-415c-bed6-2019d6a804d7"> 3: llm Although the load balancing policy is now supported, the retry policy is missing because we have found that there are times when the model is not stable and it might be better to try again.
yindo commented 2026-02-21 18:15:17 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Dec 4, 2024):

Hi, @luckylhb90. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary

  • Request for read and write separation in PostgreSQL to enhance performance.
  • Concerns about rapidly growing data volumes in specific PostgreSQL tables.
  • Unclear code logic regarding Redis usage.
  • Need for a retry policy in load balancing for LLMs due to instability.
  • You are not currently interested in contributing to these features.

Next Steps

  • Please let me know if this issue is still relevant to the latest version of Dify by commenting here.
  • If there is no further activity, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Dec 4, 2024): Hi, @luckylhb90. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog. I'm marking this issue as stale. **Issue Summary** - Request for read and write separation in PostgreSQL to enhance performance. - Concerns about rapidly growing data volumes in specific PostgreSQL tables. - Unclear code logic regarding Redis usage. - Need for a retry policy in load balancing for LLMs due to instability. - You are not currently interested in contributing to these features. **Next Steps** - Please let me know if this issue is still relevant to the latest version of Dify by commenting here. - If there is no further activity, this issue will be automatically closed in 15 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 💪 enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#6337
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?