langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-25 13:36:01 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

KeyError: 'unknown' FATAL: terminating connection due to idle-in-transaction timeout #19842

New Issue
Closed
opened 2026-02-21 20:04:30 -05:00 by yindo · 2 comments
yindo commented 2026-02-21 20:04:30 -05:00
Owner
Copy Link
Copy Source

Originally created by @itning on GitHub (Oct 23, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.9.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

run llm node in workflow

✔️ Expected Behavior

normal

❌ Actual Behavior

nginx-1          | 172.20.0.1 - - [23/Oct/2025:05:47:08 +0000] "GET /console/api/files/upload HTTP/1.1" 200 159 "https://dify.itning.cn/app/323c9754-945c-4032-9657-af3dc971e809/workflow" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36" "192.168.10.222"
db-1             | 2025-10-23 05:47:08.971 UTC [735] FATAL:  terminating connection due to idle-in-transaction timeout
nginx-1          | 172.20.0.1 - - [23/Oct/2025:05:47:09 +0000] "GET /console/api/apps/323c9754-945c-4032-9657-af3dc971e809/workflows/draft HTTP/1.1" 200 51079 "https://dify.itning.cn/app/323c9754-945c-4032-9657-af3dc971e809/workflow" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36" "192.168.10.222"
db-1             | 2025-10-23 05:47:10.283 UTC [758] FATAL:  terminating connection due to idle-in-transaction timeout
plugin_daemon-1  | [GIN] 2025/10/23 - 05:47:12 | 200 |          1m1s |      172.20.0.8 | POST     "/plugin/9df0f4a1-e432-4bf9-a89b-172b2ca380d1/dispatch/llm/invoke"
sandbox-1        | [GIN] 2025/10/23 - 05:47:12 | 200 |  103.454806ms |      172.21.0.6 | POST     "/v1/sandbox/run"
api-1            | 2025-10-23 05:47:12.471 INFO [GraphWorker-0] [_client.py:1038] - HTTP Request: POST http://sandbox:8194/v1/sandbox/run "HTTP/1.1 200 OK"
api-1            | 2025-10-23 05:47:12.473 ERROR [GraphDispatcher] [dispatcher.py:123] - Dispatcher error
api-1            | Traceback (most recent call last):
api-1            |   File "/app/api/core/workflow/graph_engine/orchestration/dispatcher.py", line 105, in _dispatcher_loop
api-1            |     self._event_handler.dispatch(event)
api-1            |   File "/app/api/core/workflow/graph_engine/event_management/event_handlers.py", line 99, in dispatch
api-1            |     return self._dispatch(event)
api-1            |            ^^^^^^^^^^^^^^^^^^^^^
api-1            |   File "/usr/local/lib/python3.12/functools.py", line 949, in _method
api-1            |     return method.__get__(obj, cls)(*args, **kwargs)
api-1            |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
api-1            |   File "/app/api/core/workflow/graph_engine/event_management/event_handlers.py", line 234, in _
api-1            |     result = self._error_handler.handle_node_failure(event)
api-1            |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
api-1            |   File "/app/api/core/workflow/graph_engine/error_handler.py", line 65, in handle_node_failure
api-1            |     node = self._graph.nodes[event.node_id]
api-1            |            ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
api-1            | KeyError: 'unknown'
nginx-1          | 2025/10/23 05:47:12 [error] 20#20: *81 upstream prematurely closed connection while reading upstream, client: 172.20.0.1, server: _, request: "POST /console/api/apps/323c9754-945c-4032-9657-af3dc971e809/workflows/draft/run HTTP/1.1", upstream: "http://172.20.0.8:5001/console/api/apps/323c9754-945c-4032-9657-af3dc971e809/workflows/draft/run", host: "dify.itning.cn", referrer: "https://dify.itning.cn/app/323c9754-945c-4032-9657-af3dc971e809/workflow"
api-1            | [2025-10-23 05:47:12 +0000] [35] [ERROR] Error handling request
api-1            | Traceback (most recent call last):
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1128, in _rollback_impl
api-1            |     self.engine.dialect.do_rollback(self.connection)
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 711, in do_rollback
api-1            |     dbapi_connection.rollback()
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/psycogreen/gevent.py", line 32, in gevent_wait_callback
api-1            |     state = conn.poll()
api-1            |             ^^^^^^^^^^^
api-1            | psycopg2.OperationalError: FATAL:  terminating connection due to idle-in-transaction timeout
api-1            | server closed the connection unexpectedly
api-1            |      This probably means the server terminated abnormally
api-1            |      before or while processing the request.
api-1            |
api-1            |
api-1            | The above exception was the direct cause of the following exception:
api-1            |
api-1            | Traceback (most recent call last):
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/gunicorn/workers/base_async.py", line 114, in handle_request
api-1            |     for item in respiter:
api-1            |                 ^^^^^^^^
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/werkzeug/wsgi.py", line 256, in __next__
api-1            |     return self._next()
api-1            |            ^^^^^^^^^^^^
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/werkzeug/wrappers/response.py", line 32, in _iter_encoded
api-1            |     for item in iterable:
api-1            |                 ^^^^^^^^
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/flask/helpers.py", line 130, in generator
api-1            |     with app_ctx, req_ctx:
api-1            |          ^^^^^^^
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/flask/ctx.py", line 284, in __exit__
api-1            |     self.pop(exc_value)
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/flask/ctx.py", line 262, in pop
api-1            |     self.app.do_teardown_appcontext(exc)
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 1382, in do_teardown_appcontext
api-1            |     self.ensure_sync(func)(exc)
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/flask_sqlalchemy/extension.py", line 448, in _teardown_session
api-1            |     self.session.remove()
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/scoping.py", line 259, in remove
api-1            |     self.registry().close()
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2507, in close
api-1            |     self._close_impl(invalidate=False)
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2576, in _close_impl
api-1            |     transaction.close(invalidate)
api-1            |   File "<string>", line 2, in close
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
api-1            |     ret_value = fn(self, *arg, **kw)
api-1            |                 ^^^^^^^^^^^^^^^^^^^^
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1417, in close
api-1            |     transaction.close()
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2589, in close
api-1            |     self._do_close()
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2727, in _do_close
api-1            |     self._close_impl()
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2713, in _close_impl
api-1            |     self._connection_rollback_impl()
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2705, in _connection_rollback_impl
api-1            |     self.connection._rollback_impl()
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1130, in _rollback_impl
api-1            |     self._handle_dbapi_exception(e, None, None, None, None)
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2355, in _handle_dbapi_exception
api-1            |     raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1128, in _rollback_impl
api-1            |     self.engine.dialect.do_rollback(self.connection)
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 711, in do_rollback
api-1            |     dbapi_connection.rollback()
api-1            |   File "/app/api/.venv/lib/python3.12/site-packages/psycogreen/gevent.py", line 32, in gevent_wait_callback
api-1            |     state = conn.poll()
api-1            |             ^^^^^^^^^^^
api-1            | sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL:  terminating connection due to idle-in-transaction timeout
api-1            | server closed the connection unexpectedly
api-1            |      This probably means the server terminated abnormally
api-1            |      before or while processing the request.
api-1            |
api-1            | (Background on this error at: https://sqlalche.me/e/20/e3q8)
nginx-1          | 172.20.0.1 - - [23/Oct/2025:05:47:12 +0000] "POST /console/api/apps/323c9754-945c-4032-9657-af3dc971e809/workflows/draft/run HTTP/1.1" 200 1576066 "https://dify.itning.cn/app/323c9754-945c-4032-9657-af3dc971e809/workflow" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36" "192.168.10.222"
api-1            | 2025-10-23 05:47:12.585 ERROR [Thread-15 (_generate_worker)] [workflow_entry.py:111] - Unknown Error when workflow entry running
api-1            | Traceback (most recent call last):
api-1            |   File "/app/api/core/workflow/workflow_entry.py", line 107, in run
api-1            |     yield from generator
api-1            |   File "/app/api/core/workflow/graph_engine/graph_engine.py", line 268, in run
api-1            |     raise self._graph_execution.error
api-1            |   File "/app/api/core/workflow/graph_engine/orchestration/dispatcher.py", line 105, in _dispatcher_loop
api-1            |     self._event_handler.dispatch(event)
api-1            |   File "/app/api/core/workflow/graph_engine/event_management/event_handlers.py", line 99, in dispatch
api-1            |     return self._dispatch(event)
api-1            |            ^^^^^^^^^^^^^^^^^^^^^
api-1            |   File "/usr/local/lib/python3.12/functools.py", line 949, in _method
api-1            |     return method.__get__(obj, cls)(*args, **kwargs)
api-1            |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
api-1            |   File "/app/api/core/workflow/graph_engine/event_management/event_handlers.py", line 234, in _
api-1            |     result = self._error_handler.handle_node_failure(event)
api-1            |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
api-1            |   File "/app/api/core/workflow/graph_engine/error_handler.py", line 65, in handle_node_failure
api-1            |     node = self._graph.nodes[event.node_id]
api-1            |            ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
api-1            | KeyError: 'unknown'
Originally created by @itning on GitHub (Oct 23, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.9.2 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce run llm node in workflow ### ✔️ Expected Behavior normal ### ❌ Actual Behavior ``` nginx-1 | 172.20.0.1 - - [23/Oct/2025:05:47:08 +0000] "GET /console/api/files/upload HTTP/1.1" 200 159 "https://dify.itning.cn/app/323c9754-945c-4032-9657-af3dc971e809/workflow" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36" "192.168.10.222" db-1 | 2025-10-23 05:47:08.971 UTC [735] FATAL: terminating connection due to idle-in-transaction timeout nginx-1 | 172.20.0.1 - - [23/Oct/2025:05:47:09 +0000] "GET /console/api/apps/323c9754-945c-4032-9657-af3dc971e809/workflows/draft HTTP/1.1" 200 51079 "https://dify.itning.cn/app/323c9754-945c-4032-9657-af3dc971e809/workflow" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36" "192.168.10.222" db-1 | 2025-10-23 05:47:10.283 UTC [758] FATAL: terminating connection due to idle-in-transaction timeout plugin_daemon-1 | [GIN] 2025/10/23 - 05:47:12 | 200 | 1m1s | 172.20.0.8 | POST "/plugin/9df0f4a1-e432-4bf9-a89b-172b2ca380d1/dispatch/llm/invoke" sandbox-1 | [GIN] 2025/10/23 - 05:47:12 | 200 | 103.454806ms | 172.21.0.6 | POST "/v1/sandbox/run" api-1 | 2025-10-23 05:47:12.471 INFO [GraphWorker-0] [_client.py:1038] - HTTP Request: POST http://sandbox:8194/v1/sandbox/run "HTTP/1.1 200 OK" api-1 | 2025-10-23 05:47:12.473 ERROR [GraphDispatcher] [dispatcher.py:123] - Dispatcher error api-1 | Traceback (most recent call last): api-1 | File "/app/api/core/workflow/graph_engine/orchestration/dispatcher.py", line 105, in _dispatcher_loop api-1 | self._event_handler.dispatch(event) api-1 | File "/app/api/core/workflow/graph_engine/event_management/event_handlers.py", line 99, in dispatch api-1 | return self._dispatch(event) api-1 | ^^^^^^^^^^^^^^^^^^^^^ api-1 | File "/usr/local/lib/python3.12/functools.py", line 949, in _method api-1 | return method.__get__(obj, cls)(*args, **kwargs) api-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-1 | File "/app/api/core/workflow/graph_engine/event_management/event_handlers.py", line 234, in _ api-1 | result = self._error_handler.handle_node_failure(event) api-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-1 | File "/app/api/core/workflow/graph_engine/error_handler.py", line 65, in handle_node_failure api-1 | node = self._graph.nodes[event.node_id] api-1 | ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ api-1 | KeyError: 'unknown' nginx-1 | 2025/10/23 05:47:12 [error] 20#20: *81 upstream prematurely closed connection while reading upstream, client: 172.20.0.1, server: _, request: "POST /console/api/apps/323c9754-945c-4032-9657-af3dc971e809/workflows/draft/run HTTP/1.1", upstream: "http://172.20.0.8:5001/console/api/apps/323c9754-945c-4032-9657-af3dc971e809/workflows/draft/run", host: "dify.itning.cn", referrer: "https://dify.itning.cn/app/323c9754-945c-4032-9657-af3dc971e809/workflow" api-1 | [2025-10-23 05:47:12 +0000] [35] [ERROR] Error handling request api-1 | Traceback (most recent call last): api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1128, in _rollback_impl api-1 | self.engine.dialect.do_rollback(self.connection) api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 711, in do_rollback api-1 | dbapi_connection.rollback() api-1 | File "/app/api/.venv/lib/python3.12/site-packages/psycogreen/gevent.py", line 32, in gevent_wait_callback api-1 | state = conn.poll() api-1 | ^^^^^^^^^^^ api-1 | psycopg2.OperationalError: FATAL: terminating connection due to idle-in-transaction timeout api-1 | server closed the connection unexpectedly api-1 | This probably means the server terminated abnormally api-1 | before or while processing the request. api-1 | api-1 | api-1 | The above exception was the direct cause of the following exception: api-1 | api-1 | Traceback (most recent call last): api-1 | File "/app/api/.venv/lib/python3.12/site-packages/gunicorn/workers/base_async.py", line 114, in handle_request api-1 | for item in respiter: api-1 | ^^^^^^^^ api-1 | File "/app/api/.venv/lib/python3.12/site-packages/werkzeug/wsgi.py", line 256, in __next__ api-1 | return self._next() api-1 | ^^^^^^^^^^^^ api-1 | File "/app/api/.venv/lib/python3.12/site-packages/werkzeug/wrappers/response.py", line 32, in _iter_encoded api-1 | for item in iterable: api-1 | ^^^^^^^^ api-1 | File "/app/api/.venv/lib/python3.12/site-packages/flask/helpers.py", line 130, in generator api-1 | with app_ctx, req_ctx: api-1 | ^^^^^^^ api-1 | File "/app/api/.venv/lib/python3.12/site-packages/flask/ctx.py", line 284, in __exit__ api-1 | self.pop(exc_value) api-1 | File "/app/api/.venv/lib/python3.12/site-packages/flask/ctx.py", line 262, in pop api-1 | self.app.do_teardown_appcontext(exc) api-1 | File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 1382, in do_teardown_appcontext api-1 | self.ensure_sync(func)(exc) api-1 | File "/app/api/.venv/lib/python3.12/site-packages/flask_sqlalchemy/extension.py", line 448, in _teardown_session api-1 | self.session.remove() api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/scoping.py", line 259, in remove api-1 | self.registry().close() api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2507, in close api-1 | self._close_impl(invalidate=False) api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2576, in _close_impl api-1 | transaction.close(invalidate) api-1 | File "<string>", line 2, in close api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go api-1 | ret_value = fn(self, *arg, **kw) api-1 | ^^^^^^^^^^^^^^^^^^^^ api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1417, in close api-1 | transaction.close() api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2589, in close api-1 | self._do_close() api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2727, in _do_close api-1 | self._close_impl() api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2713, in _close_impl api-1 | self._connection_rollback_impl() api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2705, in _connection_rollback_impl api-1 | self.connection._rollback_impl() api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1130, in _rollback_impl api-1 | self._handle_dbapi_exception(e, None, None, None, None) api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2355, in _handle_dbapi_exception api-1 | raise sqlalchemy_exception.with_traceback(exc_info[2]) from e api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1128, in _rollback_impl api-1 | self.engine.dialect.do_rollback(self.connection) api-1 | File "/app/api/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 711, in do_rollback api-1 | dbapi_connection.rollback() api-1 | File "/app/api/.venv/lib/python3.12/site-packages/psycogreen/gevent.py", line 32, in gevent_wait_callback api-1 | state = conn.poll() api-1 | ^^^^^^^^^^^ api-1 | sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: terminating connection due to idle-in-transaction timeout api-1 | server closed the connection unexpectedly api-1 | This probably means the server terminated abnormally api-1 | before or while processing the request. api-1 | api-1 | (Background on this error at: https://sqlalche.me/e/20/e3q8) nginx-1 | 172.20.0.1 - - [23/Oct/2025:05:47:12 +0000] "POST /console/api/apps/323c9754-945c-4032-9657-af3dc971e809/workflows/draft/run HTTP/1.1" 200 1576066 "https://dify.itning.cn/app/323c9754-945c-4032-9657-af3dc971e809/workflow" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36" "192.168.10.222" api-1 | 2025-10-23 05:47:12.585 ERROR [Thread-15 (_generate_worker)] [workflow_entry.py:111] - Unknown Error when workflow entry running api-1 | Traceback (most recent call last): api-1 | File "/app/api/core/workflow/workflow_entry.py", line 107, in run api-1 | yield from generator api-1 | File "/app/api/core/workflow/graph_engine/graph_engine.py", line 268, in run api-1 | raise self._graph_execution.error api-1 | File "/app/api/core/workflow/graph_engine/orchestration/dispatcher.py", line 105, in _dispatcher_loop api-1 | self._event_handler.dispatch(event) api-1 | File "/app/api/core/workflow/graph_engine/event_management/event_handlers.py", line 99, in dispatch api-1 | return self._dispatch(event) api-1 | ^^^^^^^^^^^^^^^^^^^^^ api-1 | File "/usr/local/lib/python3.12/functools.py", line 949, in _method api-1 | return method.__get__(obj, cls)(*args, **kwargs) api-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-1 | File "/app/api/core/workflow/graph_engine/event_management/event_handlers.py", line 234, in _ api-1 | result = self._error_handler.handle_node_failure(event) api-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-1 | File "/app/api/core/workflow/graph_engine/error_handler.py", line 65, in handle_node_failure api-1 | node = self._graph.nodes[event.node_id] api-1 | ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ api-1 | KeyError: 'unknown' ```
yindo added the 🐞 bug label 2026-02-21 20:04:30 -05:00
yindo closed this issue 2026-02-21 20:04:30 -05:00
yindo commented 2026-02-21 20:04:31 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Oct 23, 2025):

Your error is caused by PostgreSQL terminating a connection due to an idle-in-transaction timeout, which leaves the workflow or graph state inconsistent. This results in events with invalid node IDs like 'unknown', leading to a KeyError in Dify's error handler when it tries to access self._graph.nodes[event.node_id] with a key that doesn't exist. The error handler currently doesn't gracefully handle missing or invalid node IDs, so the crash propagates and breaks workflow execution. This is a known issue and has been discussed by other users as well [source].

The root cause is usually that database sessions or transactions are left open too long during long-running or looped operations, causing PostgreSQL to terminate the connection. Recent fixes in Dify refactored database access to always use context-managed sessions (e.g., with Session(db.engine, expire_on_commit=False) as session:) for each operation, ensuring transactions are short-lived and connections are released promptly, which prevents idle-in-transaction timeouts [source].

Recommended actions:

  • Audit your workflow and agent node code to ensure all database queries and updates are wrapped in context-managed sessions.
  • Never share db.session across threads or keep it open across multiple loop iterations.
  • If you have custom code or plugins, refactor them to use the recommended session pattern.
  • As a temporary workaround, you can increase the PostgreSQL idle_in_transaction_session_timeout (e.g., idle_in_transaction_session_timeout = '3600s'), but this only masks the issue and is not recommended for production [source].
  • Consider lowering SQLALCHEMY_POOL_SIZE to help control pool exhaustion.

For the Dify codebase, adding better guarding or logging to the error handler's node access would help surface the root cause and prevent fatal crashes when node IDs are missing or invalid. If you are running a version prior to recent session management improvements, consider updating or backporting those changes.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

@dosubot[bot] commented on GitHub (Oct 23, 2025): <!-- Answer --> Your error is caused by PostgreSQL terminating a connection due to an idle-in-transaction timeout, which leaves the workflow or graph state inconsistent. This results in events with invalid node IDs like `'unknown'`, leading to a `KeyError` in Dify's error handler when it tries to access `self._graph.nodes[event.node_id]` with a key that doesn't exist. The error handler currently doesn't gracefully handle missing or invalid node IDs, so the crash propagates and breaks workflow execution. This is a known issue and has been discussed by other users as well [[source]](https://github.com/langgenius/dify/issues/26563#issuecomment-3428643371). The root cause is usually that database sessions or transactions are left open too long during long-running or looped operations, causing PostgreSQL to terminate the connection. Recent fixes in Dify refactored database access to always use context-managed sessions (e.g., `with Session(db.engine, expire_on_commit=False) as session:`) for each operation, ensuring transactions are short-lived and connections are released promptly, which prevents idle-in-transaction timeouts [[source]](https://github.com/langgenius/dify/issues/27266#issuecomment-3430938229). **Recommended actions:** - Audit your workflow and agent node code to ensure all database queries and updates are wrapped in context-managed sessions. - Never share `db.session` across threads or keep it open across multiple loop iterations. - If you have custom code or plugins, refactor them to use the recommended session pattern. - As a temporary workaround, you can increase the PostgreSQL `idle_in_transaction_session_timeout` (e.g., `idle_in_transaction_session_timeout = '3600s'`), but this only masks the issue and is not recommended for production [[source]](https://github.com/langgenius/dify/issues/26563#issuecomment-3428643371). - Consider lowering `SQLALCHEMY_POOL_SIZE` to help control pool exhaustion. For the Dify codebase, adding better guarding or logging to the error handler's node access would help surface the root cause and prevent fatal crashes when node IDs are missing or invalid. If you are running a version prior to recent session management improvements, consider updating or backporting those changes. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/e1461c89-7c2d-4334-bf73-675660add35b?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/e1461c89-7c2d-4334-bf73-675660add35b?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/e1461c89-7c2d-4334-bf73-675660add35b?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/e1461c89-7c2d-4334-bf73-675660add35b?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/e1461c89-7c2d-4334-bf73-675660add35b?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/e1461c89-7c2d-4334-bf73-675660add35b?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/e1461c89-7c2d-4334-bf73-675660add35b?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/27320)
yindo commented 2026-02-21 20:04:31 -05:00
Author
Owner
Copy Link
Copy Source

@itning commented on GitHub (Oct 23, 2025):

modify

POSTGRES_IDLE_IN_TRANSACTION_SESSION_TIMEOUT:-600000

and work normal

@itning commented on GitHub (Oct 23, 2025): modify `POSTGRES_IDLE_IN_TRANSACTION_SESSION_TIMEOUT:-600000` and work normal
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#19842
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?