langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 04:25:29 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

The database encounters an error while processing the upload of large files to the knowledge base.psycopg2.OperationalError: SSL connection has been closed unexpectedly #2538

New Issue
Closed
opened 2026-02-21 17:46:59 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 17:46:59 -05:00
Owner
Copy Link
Copy Source

Originally created by @xiaobinji on GitHub (Apr 24, 2024).

Originally assigned to: @JohnJyong on GitHub.

Self Checks

  • This is only for bug report, if you would like to ask a quesion, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • Pleas do not modify this template :) and fill in all the required fields.

Dify version

0.6.3

Cloud or Self Hosted

Cloud, Self Hosted (Docker)

Steps to reproduce

1.Create a new Knowledge.Index Method choose 'High Quality'.
image
2.Upload a little big file to the knowledge.
image
3.The file is about 3.6 MB.It is a txt file, an interface description document exported from Swagger.Since it involves the description of internal business interfaces, I will not upload this file.
screenshots is something like that
image
4.After a long period of execution (about 17 minutes or so), the worker service will throw an error.
`
2024-04-24 04:05:56,947: ERROR/MainProcess] consume document failed
Traceback (most recent call last):
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
self.dialect.do_execute(
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
psycopg2.OperationalError: SSL connection has been closed unexpectedly

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/***/PycharmProjects/dify/api/core/indexing_runner.py", line 73, in run
self._load(
File "/Users//PycharmProjects/dify/api/core/indexing_runner.py", line 695, in _load
self._update_document_index_status(
File "/Users//PycharmProjects/***dify/api/core/indexing_runner.py", line 773, in _update_document_index_status
count = DatasetDocument.query.filter_by(id=document_id, is_paused=True).count()
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 3176, in count
return self._from_self(col).enable_eagerloads(False).scalar()
File "/opt/anaconda3/envs/dify/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2893, in scalardify
ret = self.one()
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2870, in one
return self._iter().one()
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2916, in _iter
result = self.session.execute(
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1665, in execute
) = compile_state_cls.orm_pre_session_exec(
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/context.py", line 312, in orm_pre_session_exec
session.autoflush()
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2264, in autoflush
util.raise(e, with_traceback=sys.exc_info()[2])
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise
raise exception
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2253, in _autoflush
self.flush()
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3449, in flush
self.flush(objects)
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3588, in flush
with util.safe_reraise():
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit
compat.raise(
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise
raise exception
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3549, in _flush
flush_context.execute()
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
rec.execute(self)
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
util.preloaded.orm_persistence.save_obj(
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 237, in save_obj
_emit_update_statements(
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 1001, in _emit_update_statements
c = connection._execute_20(
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1710, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
return connection._execute_clauseelement(
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement
ret = self._execute_context(
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context
self.handle_dbapi_exception(
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2134, in handle_dbapi_exception
util.raise(
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise
raise exception
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
self.dialect.do_execute(
File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(psycopg2.OperationalError) SSL connection has been closed unexpectedly

[SQL: UPDATE datasets SET index_struct=%(index_struct)s WHERE datasets.id = %(datasets_id)s]
[parameters: {'index_struct': '{"type": "qdrant", "vector_store": {"class_prefix": "Vector_index_83928be0_4615_4080_8281_df9e3a1c790b_Node"}}', 'datasets_id': UUID('25588ea0-94d5-4c5d-bceb-bc40fe8bcbce')}]
(Background on this error at: https://sqlalche.me/e/14/e3q8)

`
image
5.I have tested it multiple times and can reproduce the issue consistently.
6.The 'idle_in_transaction_session_timeout' of PostgreSQL is 5 minutes.
7.The error log of the database is as follows:
image

✔️ Expected Behavior

All functions work well

❌ Actual Behavior

Encounters an error

Originally created by @xiaobinji on GitHub (Apr 24, 2024). Originally assigned to: @JohnJyong on GitHub. ### Self Checks - [X] This is only for bug report, if you would like to ask a quesion, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] Pleas do not modify this template :) and fill in all the required fields. ### Dify version 0.6.3 ### Cloud or Self Hosted Cloud, Self Hosted (Docker) ### Steps to reproduce 1.Create a new Knowledge.Index Method choose 'High Quality'. <img width="1131" alt="image" src="https://github.com/langgenius/dify/assets/2203229/58f03c89-1cb0-43af-a80e-8f8bd2c3bd91"> 2.Upload a little big file to the knowledge. <img width="1368" alt="image" src="https://github.com/langgenius/dify/assets/2203229/7b7dc4fb-ced0-4b7e-9d3a-35876dabf925"> 3.The file is about 3.6 MB.It is a txt file, an interface description document exported from Swagger.Since it involves the description of internal business interfaces, I will not upload this file. screenshots is something like that <img width="646" alt="image" src="https://github.com/langgenius/dify/assets/2203229/a0c9cbbe-bf3d-40f6-9477-55945c3772d9"> 4.After a long period of execution (about 17 minutes or so), the worker service will throw an error. ` 2024-04-24 04:05:56,947: ERROR/MainProcess] consume document failed Traceback (most recent call last): File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context self.dialect.do_execute( File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute cursor.execute(statement, parameters) psycopg2.OperationalError: SSL connection has been closed unexpectedly The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/***/PycharmProjects/***dify/api/core/indexing_runner.py", line 73, in run self._load( File "/Users/***/PycharmProjects/***dify/api/core/indexing_runner.py", line 695, in _load self._update_document_index_status( File "/Users/***/PycharmProjects/***dify/api/core/indexing_runner.py", line 773, in _update_document_index_status count = DatasetDocument.query.filter_by(id=document_id, is_paused=True).count() File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 3176, in count return self._from_self(col).enable_eagerloads(False).scalar() File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2893, in scalar***dify ret = self.one() File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2870, in one return self._iter().one() File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2916, in _iter result = self.session.execute( File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1665, in execute ) = compile_state_cls.orm_pre_session_exec( File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/context.py", line 312, in orm_pre_session_exec session._autoflush() File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2264, in _autoflush util.raise_(e, with_traceback=sys.exc_info()[2]) File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2253, in _autoflush self.flush() File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3449, in flush self._flush(objects) File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3588, in _flush with util.safe_reraise(): File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ compat.raise_( File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3549, in _flush flush_context.execute() File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute rec.execute(self) File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute util.preloaded.orm_persistence.save_obj( File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 237, in save_obj _emit_update_statements( File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 1001, in _emit_update_statements c = connection._execute_20( File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1710, in _execute_20 return meth(self, args_10style, kwargs_10style, execution_options) File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection return connection._execute_clauseelement( File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement ret = self._execute_context( File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context self._handle_dbapi_exception( File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2134, in _handle_dbapi_exception util.raise_( File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context self.dialect.do_execute( File "/opt/anaconda3/envs/***dify/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.OperationalError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (psycopg2.OperationalError) SSL connection has been closed unexpectedly [SQL: UPDATE datasets SET index_struct=%(index_struct)s WHERE datasets.id = %(datasets_id)s] [parameters: {'index_struct': '{"type": "qdrant", "vector_store": {"class_prefix": "Vector_index_83928be0_4615_4080_8281_df9e3a1c790b_Node"}}', 'datasets_id': UUID('25588ea0-94d5-4c5d-bceb-bc40fe8bcbce')}] (Background on this error at: https://sqlalche.me/e/14/e3q8) ` <img width="1406" alt="image" src="https://github.com/langgenius/dify/assets/2203229/41c898ab-1253-407b-a093-b129fb35615f"> 5.I have tested it multiple times and can reproduce the issue consistently. 6.The 'idle_in_transaction_session_timeout' of PostgreSQL is 5 minutes. 7.The error log of the database is as follows: <img width="1305" alt="image" src="https://github.com/langgenius/dify/assets/2203229/c5dbd138-a58a-4a55-954c-4e23c0983643"> ### ✔️ Expected Behavior All functions work well ### ❌ Actual Behavior Encounters an error
yindo added the 🐞 bug label 2026-02-21 17:46:59 -05:00
yindo closed this issue 2026-02-21 17:46:59 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#2538
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?