langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-08-24 22:11:36 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Parallel embedding request results in timeout #15871

New Issue
Closed
opened 2026-02-21 19:23:45 -05:00 by yindo · 2 comments
yindo commented 2026-02-21 19:23:45 -05:00
Owner
Copy Link
Copy Source

Originally created by @DavideDelbianco on GitHub (Aug 4, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.7.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I have created an MCP tool from a workflow that accepts the payload to embed a document as text into the knowledge.

Whenever I invoke the tool with a parallel = 10 iteration node to send multiple embedding commands, the application freezes and timeouts

✔️ Expected Behavior

Application should handle multiple concurrent API requests

❌ Actual Behavior

    self.__exit__(None, None, None)

  File "/usr/local/lib/python3.12/contextlib.py", line 610, in __exit__

    raise exc_details[1]

  File "/usr/local/lib/python3.12/contextlib.py", line 595, in __exit__

    if cb(*exc_details):

       ^^^^^^^^^^^^^^^^

  File "/app/api/core/mcp/session/base_session.py", line 200, in __exit__

    self._receiver_future.result(timeout=5.0)  # Wait up to 5 seconds

    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result

    return self.__get_result()

           ^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result

    raise self._exception

  File "/app/api/core/tools/mcp_tool/tool.py", line 47, in _invoke

    result = mcp_client.invoke_tool(tool_name=self.entity.identity.name, tool_args=tool_parameters)

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/app/api/core/mcp/mcp_client.py", line 139, in invoke_tool

    return self._session.call_tool(tool_name, tool_args)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/app/api/core/mcp/session/client_session.py", line 252, in call_tool

    return self.send_request(

           ^^^^^^^^^^^^^^^^^^

  File "/app/api/core/mcp/session/base_session.py", line 250, in send_request

    self.check_receiver_status()

  File "/app/api/core/mcp/session/base_session.py", line 189, in check_receiver_status

    self._receiver_future.result()

  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result

    return self.__get_result()

           ^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result

    raise self._exception

  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 59, in run

    result = self.fn(*self.args, **self.kwargs)

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/app/api/core/mcp/session/base_session.py", line 336, in _receive_loop

    self._handle_incoming(message)

  File "/app/api/core/mcp/session/client_session.py", line 356, in _handle_incoming

    self._message_handler(req)

  File "/app/api/core/mcp/session/client_session.py", line 44, in _default_message_handler

    raise ValueError(str(message))

ValueError: timed out
2025-08-04 19:57:33.404 WARNING [Thread-95 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:57:33.807 WARNING [Thread-97 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:57:33.808 WARNING [Thread-97 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:58:33.709 WARNING [Thread-81 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:58:33.709 WARNING [Thread-81 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:58:33.816 WARNING [Thread-83 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:58:33.816 WARNING [Thread-83 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:58:34.147 WARNING [Thread-87 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:58:34.147 WARNING [Thread-87 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:58:34.256 WARNING [Thread-89 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:58:34.257 WARNING [Thread-89 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:58:34.258 WARNING [Thread-78 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/metadata⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/metadata) failed on attempt 1: timed out

2025-08-04 19:58:34.258 WARNING [Thread-78 (_generate_worker)] [node.py:132] - http request node 1754304728365 failed to run: timed out

2025-08-04 19:58:34.262 WARNING [Thread-85 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:58:34.262 WARNING [Thread-85 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:58:34.264 WARNING [Thread-91 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:58:34.264 WARNING [Thread-91 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:58:34.359 WARNING [Thread-93 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:58:34.360 WARNING [Thread-93 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:58:34.534 WARNING [Thread-95 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:58:34.535 WARNING [Thread-95 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:58:35.248 WARNING [Thread-97 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:58:35.249 WARNING [Thread-97 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:59:34.831 WARNING [Thread-81 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:59:34.832 WARNING [Thread-81 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:59:34.937 WARNING [Thread-83 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:59:34.937 WARNING [Thread-83 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:59:35.258 WARNING [Thread-87 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:59:35.258 WARNING [Thread-87 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:59:35.775 WARNING [Thread-78 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/metadata⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/metadata) failed on attempt 1: timed out

2025-08-04 19:59:35.776 WARNING [Thread-78 (_generate_worker)] [node.py:132] - http request node 1754304728365 failed to run: timed out

2025-08-04 19:59:35.778 WARNING [Thread-89 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:59:35.779 WARNING [Thread-89 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:59:35.933 WARNING [Thread-91 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:59:35.934 WARNING [Thread-91 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:59:35.934 WARNING [Thread-85 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:59:35.934 WARNING [Thread-85 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:59:35.935 WARNING [Thread-93 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:59:35.935 WARNING [Thread-93 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:59:35.936 WARNING [Thread-95 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:59:35.936 WARNING [Thread-95 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 19:59:36.366 WARNING [Thread-97 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 19:59:36.366 WARNING [Thread-97 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 20:00:35.945 WARNING [Thread-81 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 20:00:35.945 WARNING [Thread-81 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 20:00:35.976 INFO [ThreadPoolExecutor-104_0] [_client.py:1038] - HTTP Request: POST [http://host.docker.internal/mcp/server/J5aYlMYucALcWirO/mcp⁠](http://host.docker.internal/mcp/server/J5aYlMYucALcWirO/mcp) "HTTP/1.1 200 OK"


[on_tool_execution]

Tool: kb_embed_document

Outputs: {"type":"json","message":{"json_object":{}},"meta":null}


2025-08-04 20:00:36.098 WARNING [Thread-83 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out

2025-08-04 20:00:36.098 WARNING [Thread-83 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out

2025-08-04 20:00:36.178 INFO [ThreadPoolExecutor-102_0] [_client.py:1038] - HTTP Request: POST [http://host.docker.internal/mcp/server/J5aYlMYucALcWirO/mcp⁠](http://host.docker.internal/mcp/server/J5aYlMYucALcWirO/mcp) "HTTP/1.1 200 OK"


[on_tool_execution]

Tool: kb_embed_document

Outputs: {"type":"json","message":{"json_object":{}},"meta":null}
Originally created by @DavideDelbianco on GitHub (Aug 4, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.7.1 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce I have created an MCP tool from a workflow that accepts the payload to embed a document as text into the knowledge. Whenever I invoke the tool with a parallel = 10 iteration node to send multiple embedding commands, the application freezes and timeouts ### ✔️ Expected Behavior Application should handle multiple concurrent API requests ### ❌ Actual Behavior ```bash self.__exit__(None, None, None) File "/usr/local/lib/python3.12/contextlib.py", line 610, in __exit__ raise exc_details[1] File "/usr/local/lib/python3.12/contextlib.py", line 595, in __exit__ if cb(*exc_details): ^^^^^^^^^^^^^^^^ File "/app/api/core/mcp/session/base_session.py", line 200, in __exit__ self._receiver_future.result(timeout=5.0) # Wait up to 5 seconds ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception File "/app/api/core/tools/mcp_tool/tool.py", line 47, in _invoke result = mcp_client.invoke_tool(tool_name=self.entity.identity.name, tool_args=tool_parameters) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/core/mcp/mcp_client.py", line 139, in invoke_tool return self._session.call_tool(tool_name, tool_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/core/mcp/session/client_session.py", line 252, in call_tool return self.send_request( ^^^^^^^^^^^^^^^^^^ File "/app/api/core/mcp/session/base_session.py", line 250, in send_request self.check_receiver_status() File "/app/api/core/mcp/session/base_session.py", line 189, in check_receiver_status self._receiver_future.result() File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 59, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/core/mcp/session/base_session.py", line 336, in _receive_loop self._handle_incoming(message) File "/app/api/core/mcp/session/client_session.py", line 356, in _handle_incoming self._message_handler(req) File "/app/api/core/mcp/session/client_session.py", line 44, in _default_message_handler raise ValueError(str(message)) ValueError: timed out 2025-08-04 19:57:33.404 WARNING [Thread-95 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:57:33.807 WARNING [Thread-97 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:57:33.808 WARNING [Thread-97 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:58:33.709 WARNING [Thread-81 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:58:33.709 WARNING [Thread-81 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:58:33.816 WARNING [Thread-83 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:58:33.816 WARNING [Thread-83 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:58:34.147 WARNING [Thread-87 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:58:34.147 WARNING [Thread-87 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:58:34.256 WARNING [Thread-89 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:58:34.257 WARNING [Thread-89 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:58:34.258 WARNING [Thread-78 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/metadata⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/metadata) failed on attempt 1: timed out 2025-08-04 19:58:34.258 WARNING [Thread-78 (_generate_worker)] [node.py:132] - http request node 1754304728365 failed to run: timed out 2025-08-04 19:58:34.262 WARNING [Thread-85 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:58:34.262 WARNING [Thread-85 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:58:34.264 WARNING [Thread-91 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:58:34.264 WARNING [Thread-91 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:58:34.359 WARNING [Thread-93 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:58:34.360 WARNING [Thread-93 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:58:34.534 WARNING [Thread-95 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:58:34.535 WARNING [Thread-95 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:58:35.248 WARNING [Thread-97 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:58:35.249 WARNING [Thread-97 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:59:34.831 WARNING [Thread-81 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:59:34.832 WARNING [Thread-81 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:59:34.937 WARNING [Thread-83 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:59:34.937 WARNING [Thread-83 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:59:35.258 WARNING [Thread-87 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:59:35.258 WARNING [Thread-87 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:59:35.775 WARNING [Thread-78 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/metadata⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/metadata) failed on attempt 1: timed out 2025-08-04 19:59:35.776 WARNING [Thread-78 (_generate_worker)] [node.py:132] - http request node 1754304728365 failed to run: timed out 2025-08-04 19:59:35.778 WARNING [Thread-89 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:59:35.779 WARNING [Thread-89 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:59:35.933 WARNING [Thread-91 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:59:35.934 WARNING [Thread-91 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:59:35.934 WARNING [Thread-85 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:59:35.934 WARNING [Thread-85 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:59:35.935 WARNING [Thread-93 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:59:35.935 WARNING [Thread-93 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:59:35.936 WARNING [Thread-95 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:59:35.936 WARNING [Thread-95 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 19:59:36.366 WARNING [Thread-97 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 19:59:36.366 WARNING [Thread-97 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 20:00:35.945 WARNING [Thread-81 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 20:00:35.945 WARNING [Thread-81 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 20:00:35.976 INFO [ThreadPoolExecutor-104_0] [_client.py:1038] - HTTP Request: POST [http://host.docker.internal/mcp/server/J5aYlMYucALcWirO/mcp⁠](http://host.docker.internal/mcp/server/J5aYlMYucALcWirO/mcp) "HTTP/1.1 200 OK" [on_tool_execution] Tool: kb_embed_document Outputs: {"type":"json","message":{"json_object":{}},"meta":null} 2025-08-04 20:00:36.098 WARNING [Thread-83 (_generate_worker)] [ssrf_proxy.py:81] - Request to URL [http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text⁠](http://api:5001/v1/datasets/248e9e84-abda-4eeb-a43c-72c345247907/document/create-by-text) failed on attempt 1: timed out 2025-08-04 20:00:36.098 WARNING [Thread-83 (_generate_worker)] [node.py:132] - http request node 1754302492207 failed to run: timed out 2025-08-04 20:00:36.178 INFO [ThreadPoolExecutor-102_0] [_client.py:1038] - HTTP Request: POST [http://host.docker.internal/mcp/server/J5aYlMYucALcWirO/mcp⁠](http://host.docker.internal/mcp/server/J5aYlMYucALcWirO/mcp) "HTTP/1.1 200 OK" [on_tool_execution] Tool: kb_embed_document Outputs: {"type":"json","message":{"json_object":{}},"meta":null} ```
yindo added the 👻 feat:rag label 2026-02-21 19:23:45 -05:00
yindo closed this issue 2026-02-21 19:23:45 -05:00
yindo commented 2026-02-21 19:23:45 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 4, 2025):

This timeout and freezing issue with parallel embedding requests in Dify is a known pain point, especially in self-hosted Docker setups. The main causes are architectural and configuration bottlenecks:

  • The MCP session handling uses a single-threaded executor for message processing, which can become a bottleneck when multiple embedding requests are sent in parallel. Each parallel request creates its own thread/future, but the underlying session only processes one at a time, leading to delays and timeouts under load (source).
  • The default timeouts for HTTP requests and session responses are often too low for embedding operations, especially when running in parallel. These can be tuned via environment variables like SSRF_DEFAULT_TIME_OUT, SSRF_DEFAULT_READ_TIME_OUT, SSRF_DEFAULT_CONNECT_TIME_OUT, and SSRF_DEFAULT_WRITE_TIME_OUT (source).
  • There may be hardcoded timeout values in the codebase (notably in api/core/model_runtime/model_providers/) that override environment variables, so even after increasing timeouts in your .env, you could still hit these limits (source).
  • Database connection pool limits (e.g., PGVECTOR_MAX_CONNECTION, VASTBASE_MAX_CONNECTION) are often set to 5 by default, which is less than your parallel=10 setting. This can cause requests to queue up and eventually timeout (source).
  • Embedding model servers (like Ollama, LocalAI) can crash or become unreachable under parallel load, especially if not exposed correctly to Docker. Make sure your embedding server is listening on 0.0.0.0 and accessed via host.docker.internal (source).
  • Rate limiting and billing decorators on the embedding endpoints may also throttle or reject requests under high concurrency (source).

What you can do:

  1. Lower the parallelism (try parallel=2 or 3) and see if the issue persists. If it works, you’re hitting a resource or concurrency bottleneck.
  2. Increase the timeout-related environment variables in your .env or docker-compose.yaml (e.g., SSRF_DEFAULT_READ_TIME_OUT=30, PLUGIN_MAX_EXECUTION_TIMEOUT=2400, MODEL_EMBEDDING_REQUEST_TIMEOUT=120), then restart your containers (source).
  3. Check your vector DB connection pool settings (PGVECTOR_MAX_CONNECTION, VASTBASE_MAX_CONNECTION) and increase them to match or exceed your parallelism.
  4. Monitor your system’s CPU, memory, and DB connections during parallel embedding to spot resource exhaustion.
  5. Review logs for your embedding model server to ensure it’s not crashing or being throttled.
  6. If you still see timeouts, check for hardcoded timeout values in the codebase, especially in api/core/model_runtime/model_providers/, and increase them if needed (source).
  7. Incrementally increase parallelism and observe at what point failures begin.

If you try these steps and still hit issues, sharing resource usage metrics and logs from both Dify and your embedding model server will help pinpoint the bottleneck.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

@dosubot[bot] commented on GitHub (Aug 4, 2025): <!-- Answer --> This timeout and freezing issue with parallel embedding requests in Dify is a known pain point, especially in self-hosted Docker setups. The main causes are architectural and configuration bottlenecks: - The MCP session handling uses a single-threaded executor for message processing, which can become a bottleneck when multiple embedding requests are sent in parallel. Each parallel request creates its own thread/future, but the underlying session only processes one at a time, leading to delays and timeouts under load ([source](https://github.com/langgenius/dify/blob/a724f356722ec6049be7017c0aa5e43ca43f8617/api/core/mcp/session/base_session.py)). - The default timeouts for HTTP requests and session responses are often too low for embedding operations, especially when running in parallel. These can be tuned via environment variables like `SSRF_DEFAULT_TIME_OUT`, `SSRF_DEFAULT_READ_TIME_OUT`, `SSRF_DEFAULT_CONNECT_TIME_OUT`, and `SSRF_DEFAULT_WRITE_TIME_OUT` ([source](https://github.com/langgenius/dify/blob/a724f356722ec6049be7017c0aa5e43ca43f8617/api/core/helper/ssrf_proxy.py)). - There may be hardcoded timeout values in the codebase (notably in `api/core/model_runtime/model_providers/`) that override environment variables, so even after increasing timeouts in your `.env`, you could still hit these limits ([source](https://github.com/langgenius/dify/issues/19067)). - Database connection pool limits (e.g., `PGVECTOR_MAX_CONNECTION`, `VASTBASE_MAX_CONNECTION`) are often set to 5 by default, which is less than your parallel=10 setting. This can cause requests to queue up and eventually timeout ([source](https://github.com/langgenius/dify/pull/16308)). - Embedding model servers (like Ollama, LocalAI) can crash or become unreachable under parallel load, especially if not exposed correctly to Docker. Make sure your embedding server is listening on `0.0.0.0` and accessed via `host.docker.internal` ([source](https://github.com/langgenius/dify/issues/15679)). - Rate limiting and billing decorators on the embedding endpoints may also throttle or reject requests under high concurrency ([source](https://github.com/langgenius/dify/blob/a724f356722ec6049be7017c0aa5e43ca43f8617/api/controllers/service_api/dataset/document.py)). **What you can do:** 1. Lower the parallelism (try parallel=2 or 3) and see if the issue persists. If it works, you’re hitting a resource or concurrency bottleneck. 2. Increase the timeout-related environment variables in your `.env` or `docker-compose.yaml` (e.g., `SSRF_DEFAULT_READ_TIME_OUT=30`, `PLUGIN_MAX_EXECUTION_TIMEOUT=2400`, `MODEL_EMBEDDING_REQUEST_TIMEOUT=120`), then restart your containers ([source](https://github.com/langgenius/dify/issues/22911)). 3. Check your vector DB connection pool settings (`PGVECTOR_MAX_CONNECTION`, `VASTBASE_MAX_CONNECTION`) and increase them to match or exceed your parallelism. 4. Monitor your system’s CPU, memory, and DB connections during parallel embedding to spot resource exhaustion. 5. Review logs for your embedding model server to ensure it’s not crashing or being throttled. 6. If you still see timeouts, check for hardcoded timeout values in the codebase, especially in `api/core/model_runtime/model_providers/`, and increase them if needed ([source](https://github.com/langgenius/dify/issues/19067)). 7. Incrementally increase parallelism and observe at what point failures begin. If you try these steps and still hit issues, sharing resource usage metrics and logs from both Dify and your embedding model server will help pinpoint the bottleneck. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/f82fbe8d-783f-408d-8c41-4a0793e0a20d?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/f82fbe8d-783f-408d-8c41-4a0793e0a20d?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/f82fbe8d-783f-408d-8c41-4a0793e0a20d?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/f82fbe8d-783f-408d-8c41-4a0793e0a20d?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/f82fbe8d-783f-408d-8c41-4a0793e0a20d?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/f82fbe8d-783f-408d-8c41-4a0793e0a20d?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/f82fbe8d-783f-408d-8c41-4a0793e0a20d?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/23379)
yindo commented 2026-02-21 19:23:46 -05:00
Author
Owner
Copy Link
Copy Source

@DavideDelbianco commented on GitHub (Aug 5, 2025):

Using the same tool as a workflow tool instead of a MCP tool works fine with 10x parallel operations

@DavideDelbianco commented on GitHub (Aug 5, 2025): Using the same tool as a workflow tool instead of a MCP tool works fine with 10x parallel operations
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 👻 feat:rag
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#15871
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?