langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

db connection error when embedding task takes a long time #18162

New Issue
Closed
opened 2026-02-21 19:44:19 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 19:44:19 -05:00
Owner
Copy Link
Copy Source

Originally created by @AkisAya on GitHub (Sep 24, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.6.0

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

use knowledge base and submit many docs in a time and use a slow embedding model, embedding task fails at last.

here are useful logs

exception.stacktrace=Traceback (most recent call last):
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2114, in _exec_insertmany_context dialect.do_execute(
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 943, in do_execute cursor.execute(statement, parameters)
File "/opt/app/api/.venv/lib/python3.12/site-packages/psycogreen/gevent.py", line 32, in gevent_wait_callback state = conn.poll() ^^^^^^^^^^^ psycopg2.OperationalError: FATAL: terminating connection due to administrator command server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. The above exception was the direct cause of the following exception: Traceback (most recent call last):
File "/opt/app/api/core/rag/embedding/cached_embedding.py", line 93, in embed_documents db.session.commit()
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/scoping.py", line 599, in commit return self._proxied.commit() ^^^^^^^^^^^^^^^^^^^^^^
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2032, in commit trans.commit(_to_root=True)
File "<string>", line 2, in commit
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go ret_value = fn(self, *arg, **kw) ^^^^^^^^^^^^^^^^^^^^
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1313, in commit self._prepare_impl()
File "<string>", line 2, in _prepare_impl
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go ret_value = fn(self, *arg, **kw) ^^^^^^^^^^^^^^^^^^^^
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1288, in _prepare_impl self.session.flush()
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 4345, in flush self._flush(objects)
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 4480, in _flush with util.safe_reraise():
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__ raise exc_value.with_traceback(exc_tb)
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 4441, in _flush flush_context.execute()
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/unitofwork.py", line 466, in execute rec.execute(self)
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/unitofwork.py", line 642, in execute util.preloaded.orm_persistence.save_obj(
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/persistence.py", line 93, in save_obj _emit_insert_statements(
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/persistence.py", line 1143, in _emit_insert_statements result = connection.execute( ^^^^^^^^^^^^^^^^^^^
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1415, in execute return meth( ^^^^^
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 523, in _execute_on_connection return connection._execute_clauseelement( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1637, in _execute_clauseelement ret = self._execute_context( ^^^^^^^^^^^^^^^^^^^^^^
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1840, in _execute_context return self._exec_insertmany_context(dialect, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2122, in _exec_insertmany_context self._handle_dbapi_exception(
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2351, in _handle_dbapi_exception raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2114, in _exec_insertmany_context dialect.do_execute(
File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 943, in do_execute cursor.execute(statement, parameters)
File "/opt/app/api/.venv/lib/python3.12/site-packages/psycogreen/gevent.py", line 32, in gevent_wait_callback state = conn.poll() ^^^^^^^^^^^ sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: terminating connection due to administrator command server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. [SQL: INSERT INTO embeddings (model_name, hash, embedding, provider_name) VALUES (%(model_name)s, %(hash)s, %(embedding)s, %(provider_name)s) RETURNING embeddings.id, embeddings.created_at] [parameters: {'model_name': '762-text-embedding-3-large__1', 'hash': '414aa3653a14c1770ee3449e0ee564977ebe6dc2b7bafb80059428ee5f51d26d', 'embedding': <psycopg2.extensions.Binary object at 0x7f76cfc71500>, 'provider_name': 'langgenius/openai_api_compatible/openai_api_compatible'}] (Background on this error at: https://sqlalche.me/e/20/e3q8)

✔️ Expected Behavior

succeed in embedding task

❌ Actual Behavior

fail

Originally created by @AkisAya on GitHub (Sep 24, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.6.0 ### Cloud or Self Hosted Self Hosted (Source) ### Steps to reproduce use knowledge base and submit many docs in a time and use a slow embedding model, embedding task fails at last. here are useful logs ``` exception.stacktrace=Traceback (most recent call last): File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2114, in _exec_insertmany_context dialect.do_execute( File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 943, in do_execute cursor.execute(statement, parameters) File "/opt/app/api/.venv/lib/python3.12/site-packages/psycogreen/gevent.py", line 32, in gevent_wait_callback state = conn.poll() ^^^^^^^^^^^ psycopg2.OperationalError: FATAL: terminating connection due to administrator command server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/opt/app/api/core/rag/embedding/cached_embedding.py", line 93, in embed_documents db.session.commit() File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/scoping.py", line 599, in commit return self._proxied.commit() ^^^^^^^^^^^^^^^^^^^^^^ File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2032, in commit trans.commit(_to_root=True) File "<string>", line 2, in commit File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go ret_value = fn(self, *arg, **kw) ^^^^^^^^^^^^^^^^^^^^ File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1313, in commit self._prepare_impl() File "<string>", line 2, in _prepare_impl File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go ret_value = fn(self, *arg, **kw) ^^^^^^^^^^^^^^^^^^^^ File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1288, in _prepare_impl self.session.flush() File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 4345, in flush self._flush(objects) File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 4480, in _flush with util.safe_reraise(): File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__ raise exc_value.with_traceback(exc_tb) File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 4441, in _flush flush_context.execute() File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/unitofwork.py", line 466, in execute rec.execute(self) File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/unitofwork.py", line 642, in execute util.preloaded.orm_persistence.save_obj( File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/persistence.py", line 93, in save_obj _emit_insert_statements( File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/persistence.py", line 1143, in _emit_insert_statements result = connection.execute( ^^^^^^^^^^^^^^^^^^^ File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1415, in execute return meth( ^^^^^ File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 523, in _execute_on_connection return connection._execute_clauseelement( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1637, in _execute_clauseelement ret = self._execute_context( ^^^^^^^^^^^^^^^^^^^^^^ File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1840, in _execute_context return self._exec_insertmany_context(dialect, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2122, in _exec_insertmany_context self._handle_dbapi_exception( File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2351, in _handle_dbapi_exception raise sqlalchemy_exception.with_traceback(exc_info[2]) from e File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2114, in _exec_insertmany_context dialect.do_execute( File "/opt/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 943, in do_execute cursor.execute(statement, parameters) File "/opt/app/api/.venv/lib/python3.12/site-packages/psycogreen/gevent.py", line 32, in gevent_wait_callback state = conn.poll() ^^^^^^^^^^^ sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: terminating connection due to administrator command server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. [SQL: INSERT INTO embeddings (model_name, hash, embedding, provider_name) VALUES (%(model_name)s, %(hash)s, %(embedding)s, %(provider_name)s) RETURNING embeddings.id, embeddings.created_at] [parameters: {'model_name': '762-text-embedding-3-large__1', 'hash': '414aa3653a14c1770ee3449e0ee564977ebe6dc2b7bafb80059428ee5f51d26d', 'embedding': <psycopg2.extensions.Binary object at 0x7f76cfc71500>, 'provider_name': 'langgenius/openai_api_compatible/openai_api_compatible'}] (Background on this error at: https://sqlalche.me/e/20/e3q8) ``` ### ✔️ Expected Behavior succeed in embedding task ### ❌ Actual Behavior fail
yindo closed this issue 2026-02-21 19:44:19 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#18162
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?