langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Rerank model invoke exception would caused db connection exhausted #15133

New Issue
Closed
opened 2026-02-21 19:20:21 -05:00 by yindo · 2 comments
yindo commented 2026-02-21 19:20:21 -05:00
Owner
Copy Link
Copy Source

Originally created by @Colstuwjx on GitHub (Jul 4, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.1.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

  1. use a low concurrency rerank model, e.g. aws bedrock cohere.embed-multilingual-v3
  2. try to perform >1k requests to retrieve the document with enabling rerank model
  3. we'll observe the high exceptions from rerank model and also high db connections which would cause postgres sql database OOM

✔️ Expected Behavior

even though the rerank model invoke returned error and raised exception, we should also release the db connection rather than leaving it open.

❌ Actual Behavior

the postgres sql database connection numbers rapidly increase and also free memory quickly decreased to 0% .

See exception logs:

Image

The code location is here , and it's instantiated by a Thread, since flask_app.app_context() wouldn't handle with the db session close while the Thread exceptional exited, I'm afraid this would cause db connections keep increasing ?

Originally created by @Colstuwjx on GitHub (Jul 4, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.1.3 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce 1. use a low concurrency rerank model, e.g. aws bedrock cohere.embed-multilingual-v3 2. try to perform >1k requests to retrieve the document with enabling rerank model 3. we'll observe the high exceptions from rerank model and also high db connections which would cause postgres sql database OOM ### ✔️ Expected Behavior even though the rerank model invoke returned error and raised exception, we should also release the db connection rather than leaving it open. ### ❌ Actual Behavior the postgres sql database connection numbers rapidly increase and also free memory quickly decreased to 0% . See exception logs: <img width="348" height="848" alt="Image" src="https://github.com/user-attachments/assets/cec0c47e-ee8e-4895-8a97-260a36f2e19e" /> The code location is [here](https://github.com/langgenius/dify/blob/1.1.3/api/core/rag/datasource/retrieval_service.py#L112) , and it's instantiated by [a Thread](https://github.com/langgenius/dify/blob/1.1.3/api/core/rag/retrieval/dataset_retrieval.py#L463), since `flask_app.app_context()` wouldn't handle with the db session close while the Thread exceptional exited, I'm afraid this would cause db connections keep increasing ?
yindo added the 🐞 bug label 2026-02-21 19:20:21 -05:00
yindo closed this issue 2026-02-21 19:20:21 -05:00
yindo commented 2026-02-21 19:20:21 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jul 4, 2025):

You're correct—when exceptions occur in threaded rerank model invocations, database sessions are not always closed, which can quickly exhaust your PostgreSQL connections and memory. The code in both retrieval_service.py and dataset_retrieval.py uses threads and Flask app contexts, but does not guarantee session cleanup if a thread exits due to an error. This leads to leaked DB connections under high concurrency and error conditions.

A recent PR addressed a similar threading/session leak by explicitly calling db.session.close() before starting a new thread to release the connection and prevent it from being held open if an exception occurs in the thread. This pattern is not yet applied in the rerank/retrieval code paths you referenced, so the issue remains in 1.1.3 and likely in newer versions as well see PR #21726.

To mitigate this, you should wrap your threaded DB operations in a try/finally block or use a context manager to ensure db.session.close() is always called, even if an exception is raised. For example, at the end of each threaded function (like in your search or retriever methods), add:

finally:
    db.session.close()

This will help ensure connections are released back to the pool. Until this is fixed in the main codebase, you can also monitor your DB connection usage and set conservative connection pool limits to reduce the risk of OOM.

Let me know if you want a more detailed code snippet or have questions about where to apply this in your workflow.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Join Discord Share on X

@dosubot[bot] commented on GitHub (Jul 4, 2025): <!-- Answer --> You're correct—when exceptions occur in threaded rerank model invocations, database sessions are not always closed, which can quickly exhaust your PostgreSQL connections and memory. The code in both retrieval_service.py and dataset_retrieval.py uses threads and Flask app contexts, but does not guarantee session cleanup if a thread exits due to an error. This leads to leaked DB connections under high concurrency and error conditions. A recent PR addressed a similar threading/session leak by explicitly calling `db.session.close()` before starting a new thread to release the connection and prevent it from being held open if an exception occurs in the thread. This pattern is not yet applied in the rerank/retrieval code paths you referenced, so the issue remains in 1.1.3 and likely in newer versions as well [see PR #21726](https://github.com/langgenius/dify/pull/21726). To mitigate this, you should wrap your threaded DB operations in a `try/finally` block or use a context manager to ensure `db.session.close()` is always called, even if an exception is raised. For example, at the end of each threaded function (like in your search or retriever methods), add: ```python finally: db.session.close() ``` This will help ensure connections are released back to the pool. Until this is fixed in the main codebase, you can also monitor your DB connection usage and set conservative connection pool limits to reduce the risk of OOM. Let me know if you want a more detailed code snippet or have questions about where to apply this in your workflow. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/cb7b98be-1c26-44fa-b9d3-2fcff0ff2def?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/cb7b98be-1c26-44fa-b9d3-2fcff0ff2def?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/cb7b98be-1c26-44fa-b9d3-2fcff0ff2def?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/cb7b98be-1c26-44fa-b9d3-2fcff0ff2def?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/cb7b98be-1c26-44fa-b9d3-2fcff0ff2def?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/cb7b98be-1c26-44fa-b9d3-2fcff0ff2def?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/cb7b98be-1c26-44fa-b9d3-2fcff0ff2def?feedback_type=other)</sup>&nbsp;&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/21939)
yindo commented 2026-02-21 19:20:21 -05:00
Author
Owner
Copy Link
Copy Source

@Ruyu0224 commented on GitHub (Jul 11, 2025):

anyone solving the problem?Please help me solve it

@Ruyu0224 commented on GitHub (Jul 11, 2025): anyone solving the problem?Please help me solve it
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#15133
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?