When the knowledge base uses Ollama embedding to generate vectors, it causes Ollama to crash. #10716

Closed
opened 2026-02-21 18:50:18 -05:00 by yindo · 0 comments
Owner

Originally created by @crazythinking on GitHub (Mar 12, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.0.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

When the knowledge base uses Ollama embedding to generate vectors, it causes Ollama to crash. operator as follow img:

Image

✔️ Expected Behavior

No response

Actual Behavior

error logs as follows:
`

docker-plugin_daemon-1  | [GIN] 2025/03/13 - 01:48:51 | 200 |  1.392324735s |    192.168.48.9 | POST     "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke"
docker-plugin_daemon-1  | [GIN] 2025/03/13 - 01:48:51 | 200 |  1.502842951s |    192.168.48.9 | POST     "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke"
docker-plugin_daemon-1  | [GIN] 2025/03/13 - 01:48:51 | 200 |   1.41124396s |    192.168.48.9 | POST     "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke"
docker-plugin_daemon-1  | [GIN] 2025/03/13 - 01:48:51 | 200 |   1.43269332s |    192.168.48.9 | POST     "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke"
docker-plugin_daemon-1  | [GIN] 2025/03/13 - 01:48:51 | 200 |  1.421329705s |    192.168.48.9 | POST     "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke"
docker-plugin_daemon-1  | [GIN] 2025/03/13 - 01:48:51 | 200 |  1.410274884s |    192.168.48.9 | POST     "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke"
docker-plugin_daemon-1  | [GIN] 2025/03/13 - 01:48:51 | 200 |  1.412063739s |    192.168.48.9 | POST     "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke"
docker-plugin_daemon-1  | [GIN] 2025/03/13 - 01:48:51 | 200 |  1.410747272s |    192.168.48.9 | POST     "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke"
docker-plugin_daemon-1  | [GIN] 2025/03/13 - 01:48:51 | 200 |  1.411868744s |    192.168.48.9 | POST     "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke"
docker-plugin_daemon-1  | [GIN] 2025/03/13 - 01:48:51 | 200 |  1.422839868s |    192.168.48.9 | POST     "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke"
docker-worker-1         | 2025-03-13 01:48:51.463 ERROR [ThreadPoolExecutor-37_0] [cached_embedding.py:98] - Failed to embed documents: %s
docker-worker-1         | Traceback (most recent call last):
docker-worker-1         |   File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents
docker-worker-1         |     embedding_result = self._model_instance.invoke_text_embedding(
docker-worker-1         |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding
docker-worker-1         |     self._round_robin_invoke(
docker-worker-1         |   File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
docker-worker-1         |     return function(*args, **kwargs)
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke
docker-worker-1         |     raise self._transform_invoke_error(e)
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke
docker-worker-1         |     return plugin_model_manager.invoke_text_embedding(
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding
docker-worker-1         |     for resp in response:
docker-worker-1         |                 ^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
docker-worker-1         |     self._handle_plugin_daemon_error(error.error_type, error.message)
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error
docker-worker-1         |     raise InvokeServerUnavailableError(description=args.get("description"))
docker-worker-1         | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
docker-worker-1         | 2025-03-13 01:48:51.464 ERROR [ThreadPoolExecutor-37_5] [cached_embedding.py:98] - Failed to embed documents: %s
docker-worker-1         | Traceback (most recent call last):
docker-worker-1         |   File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents
docker-worker-1         |     embedding_result = self._model_instance.invoke_text_embedding(
docker-worker-1         |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding
docker-worker-1         |     self._round_robin_invoke(
docker-worker-1         |   File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
docker-worker-1         |     return function(*args, **kwargs)
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke
docker-worker-1         |     raise self._transform_invoke_error(e)
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke
docker-worker-1         |     return plugin_model_manager.invoke_text_embedding(
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding
docker-worker-1         |     for resp in response:
docker-worker-1         |                 ^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
docker-worker-1         |     self._handle_plugin_daemon_error(error.error_type, error.message)
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error
docker-worker-1         |     raise InvokeServerUnavailableError(description=args.get("description"))
docker-worker-1         | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
docker-worker-1         | 2025-03-13 01:48:51.465 ERROR [ThreadPoolExecutor-37_3] [cached_embedding.py:98] - Failed to embed documents: %s
docker-worker-1         | Traceback (most recent call last):
docker-worker-1         |   File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents
docker-worker-1         |     embedding_result = self._model_instance.invoke_text_embedding(
docker-worker-1         |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding
docker-worker-1         |     self._round_robin_invoke(
docker-worker-1         |   File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
docker-worker-1         |     return function(*args, **kwargs)
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke
docker-worker-1         |     raise self._transform_invoke_error(e)
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke
docker-worker-1         |     return plugin_model_manager.invoke_text_embedding(
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding
docker-worker-1         |     for resp in response:
docker-worker-1         |                 ^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
docker-worker-1         |     self._handle_plugin_daemon_error(error.error_type, error.message)
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error
docker-worker-1         |     raise InvokeServerUnavailableError(description=args.get("description"))
docker-worker-1         | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
docker-worker-1         | 2025-03-13 01:48:51.466 ERROR [ThreadPoolExecutor-37_7] [cached_embedding.py:98] - Failed to embed documents: %s
docker-worker-1         | Traceback (most recent call last):
docker-worker-1         |   File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents
docker-worker-1         |     embedding_result = self._model_instance.invoke_text_embedding(
docker-worker-1         |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding
docker-worker-1         |     self._round_robin_invoke(
docker-worker-1         |   File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
docker-worker-1         |     return function(*args, **kwargs)
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke
docker-worker-1         |     raise self._transform_invoke_error(e)
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke
docker-worker-1         |     return plugin_model_manager.invoke_text_embedding(
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding
docker-worker-1         |     for resp in response:
docker-worker-1         |                 ^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
docker-worker-1         |     self._handle_plugin_daemon_error(error.error_type, error.message)
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error
docker-worker-1         |     raise InvokeServerUnavailableError(description=args.get("description"))
docker-worker-1         | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
docker-worker-1         | 2025-03-13 01:48:51.467 ERROR [ThreadPoolExecutor-37_6] [cached_embedding.py:98] - Failed to embed documents: %s
docker-worker-1         | Traceback (most recent call last):
docker-worker-1         |   File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents
docker-worker-1         |     embedding_result = self._model_instance.invoke_text_embedding(
docker-worker-1         |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding
docker-worker-1         |     self._round_robin_invoke(
docker-worker-1         |   File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
docker-worker-1         |     return function(*args, **kwargs)
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke
docker-worker-1         |     raise self._transform_invoke_error(e)
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke
docker-worker-1         |     return plugin_model_manager.invoke_text_embedding(
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding
docker-worker-1         |     for resp in response:
docker-worker-1         |                 ^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
docker-worker-1         |     self._handle_plugin_daemon_error(error.error_type, error.message)
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error
docker-worker-1         |     raise InvokeServerUnavailableError(description=args.get("description"))
docker-worker-1         | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
docker-worker-1         | 2025-03-13 01:48:51.468 ERROR [ThreadPoolExecutor-37_8] [cached_embedding.py:98] - Failed to embed documents: %s
docker-worker-1         | Traceback (most recent call last):
docker-worker-1         |   File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents
docker-worker-1         |     embedding_result = self._model_instance.invoke_text_embedding(
docker-worker-1         |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding
docker-worker-1         |     self._round_robin_invoke(
docker-worker-1         |   File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
docker-worker-1         |     return function(*args, **kwargs)
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke
docker-worker-1         |     raise self._transform_invoke_error(e)
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke
docker-worker-1         |     return plugin_model_manager.invoke_text_embedding(
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding
docker-worker-1         |     for resp in response:
docker-worker-1         |                 ^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
docker-worker-1         |     self._handle_plugin_daemon_error(error.error_type, error.message)
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error
docker-worker-1         |     raise InvokeServerUnavailableError(description=args.get("description"))
docker-worker-1         | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
docker-worker-1         | 2025-03-13 01:48:51.469 ERROR [ThreadPoolExecutor-37_2] [cached_embedding.py:98] - Failed to embed documents: %s
docker-worker-1         | Traceback (most recent call last):
docker-worker-1         |   File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents
docker-worker-1         |     embedding_result = self._model_instance.invoke_text_embedding(
docker-worker-1         |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding
docker-worker-1         |     self._round_robin_invoke(
docker-worker-1         |   File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
docker-worker-1         |     return function(*args, **kwargs)
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke
docker-worker-1         |     raise self._transform_invoke_error(e)
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke
docker-worker-1         |     return plugin_model_manager.invoke_text_embedding(
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding
docker-worker-1         |     for resp in response:
docker-worker-1         |                 ^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
docker-worker-1         |     self._handle_plugin_daemon_error(error.error_type, error.message)
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error
docker-worker-1         |     raise InvokeServerUnavailableError(description=args.get("description"))
docker-worker-1         | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
docker-worker-1         | 2025-03-13 01:48:51.470 ERROR [ThreadPoolExecutor-37_9] [cached_embedding.py:98] - Failed to embed documents: %s
docker-worker-1         | Traceback (most recent call last):
docker-worker-1         |   File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents
docker-worker-1         |     embedding_result = self._model_instance.invoke_text_embedding(
docker-worker-1         |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding
docker-worker-1         |     self._round_robin_invoke(
docker-worker-1         |   File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
docker-worker-1         |     return function(*args, **kwargs)
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke
docker-worker-1         |     raise self._transform_invoke_error(e)
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke
docker-worker-1         |     return plugin_model_manager.invoke_text_embedding(
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding
docker-worker-1         |     for resp in response:
docker-worker-1         |                 ^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
docker-worker-1         |     self._handle_plugin_daemon_error(error.error_type, error.message)
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error
docker-worker-1         |     raise InvokeServerUnavailableError(description=args.get("description"))
docker-worker-1         | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
docker-worker-1         | 2025-03-13 01:48:51.471 ERROR [ThreadPoolExecutor-37_1] [cached_embedding.py:98] - Failed to embed documents: %s
docker-worker-1         | Traceback (most recent call last):
docker-worker-1         |   File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents
docker-worker-1         |     embedding_result = self._model_instance.invoke_text_embedding(
docker-worker-1         |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding
docker-worker-1         |     self._round_robin_invoke(
docker-worker-1         |   File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
docker-worker-1         |     return function(*args, **kwargs)
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke
docker-worker-1         |     raise self._transform_invoke_error(e)
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke
docker-worker-1         |     return plugin_model_manager.invoke_text_embedding(
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding
docker-worker-1         |     for resp in response:
docker-worker-1         |                 ^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
docker-worker-1         |     self._handle_plugin_daemon_error(error.error_type, error.message)
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error
docker-worker-1         |     raise InvokeServerUnavailableError(description=args.get("description"))
docker-worker-1         | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
docker-worker-1         | 2025-03-13 01:48:51.474 ERROR [ThreadPoolExecutor-37_4] [cached_embedding.py:98] - Failed to embed documents: %s
docker-worker-1         | Traceback (most recent call last):
docker-worker-1         |   File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents
docker-worker-1         |     embedding_result = self._model_instance.invoke_text_embedding(
docker-worker-1         |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding
docker-worker-1         |     self._round_robin_invoke(
docker-worker-1         |   File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
docker-worker-1         |     return function(*args, **kwargs)
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke
docker-worker-1         |     raise self._transform_invoke_error(e)
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke
docker-worker-1         |     return plugin_model_manager.invoke_text_embedding(
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding
docker-worker-1         |     for resp in response:
docker-worker-1         |                 ^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
docker-worker-1         |     self._handle_plugin_daemon_error(error.error_type, error.message)
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error
docker-worker-1         |     raise InvokeServerUnavailableError(description=args.get("description"))
docker-worker-1         | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
docker-worker-1         | 2025-03-13 01:48:51.474 ERROR [Dummy-32] [indexing_runner.py:96] - consume document failed
docker-worker-1         | Traceback (most recent call last):
docker-worker-1         |   File "/app/api/core/indexing_runner.py", line 80, in run
docker-worker-1         |     self._load(
docker-worker-1         |   File "/app/api/core/indexing_runner.py", line 570, in _load
docker-worker-1         |     tokens += future.result()
docker-worker-1         |               ^^^^^^^^^^^^^^^
docker-worker-1         |   File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 456, in result
docker-worker-1         |     return self.__get_result()
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
docker-worker-1         |     raise self._exception
docker-worker-1         |   File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 59, in run
docker-worker-1         |     result = self.fn(*self.args, **self.kwargs)
docker-worker-1         |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/indexing_runner.py", line 625, in _process_chunk
docker-worker-1         |     index_processor.load(dataset, chunk_documents, with_keywords=False)
docker-worker-1         |   File "/app/api/core/rag/index_processor/processor/paragraph_index_processor.py", line 78, in load
docker-worker-1         |     vector.create(documents)
docker-worker-1         |   File "/app/api/core/rag/datasource/vdb/vector_factory.py", line 156, in create
docker-worker-1         |     embeddings = self._embeddings.embed_documents([document.page_content for document in texts])
docker-worker-1         |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/rag/embedding/cached_embedding.py", line 99, in embed_documents
docker-worker-1         |     raise ex
docker-worker-1         |   File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents
docker-worker-1         |     embedding_result = self._model_instance.invoke_text_embedding(
docker-worker-1         |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding
docker-worker-1         |     self._round_robin_invoke(
docker-worker-1         |   File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
docker-worker-1         |     return function(*args, **kwargs)
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke
docker-worker-1         |     raise self._transform_invoke_error(e)
docker-worker-1         |   File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke
docker-worker-1         |     return plugin_model_manager.invoke_text_embedding(
docker-worker-1         |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding
docker-worker-1         |     for resp in response:
docker-worker-1         |                 ^^^^^^^^
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
docker-worker-1         |     self._handle_plugin_daemon_error(error.error_type, error.message)
docker-worker-1         |   File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error
docker-worker-1         |     raise InvokeServerUnavailableError(description=args.get("description"))
docker-worker-1         | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
docker-worker-1         | 2025-03-13 01:48:51.481 INFO [Dummy-32] [document_indexing_task.py:76] - Processed dataset: 25c00224-e521-480d-bd7d-ce10cf27d426 latency: 3.663817903958261
docker-worker-1         | 2025-03-13 01:48:51.501 INFO [Dummy-32] [trace.py:128] - Task tasks.document_indexing_task.document_indexing_task[116ac803-96a6-49ed-8db2-b838919218be] succeeded in 3.6831789190182462s: None

`
the log of ollama please see crash log : ollamalog.txt

Originally created by @crazythinking on GitHub (Mar 12, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.0.0 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce When the knowledge base uses Ollama embedding to generate vectors, it causes Ollama to crash. operator as follow img: ![Image](https://github.com/user-attachments/assets/0707d5e9-9f67-4261-9754-14eb9808d1c1) ### ✔️ Expected Behavior _No response_ ### ❌ Actual Behavior error logs as follows: ` ``` docker-plugin_daemon-1 | [GIN] 2025/03/13 - 01:48:51 | 200 | 1.392324735s | 192.168.48.9 | POST "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke" docker-plugin_daemon-1 | [GIN] 2025/03/13 - 01:48:51 | 200 | 1.502842951s | 192.168.48.9 | POST "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke" docker-plugin_daemon-1 | [GIN] 2025/03/13 - 01:48:51 | 200 | 1.41124396s | 192.168.48.9 | POST "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke" docker-plugin_daemon-1 | [GIN] 2025/03/13 - 01:48:51 | 200 | 1.43269332s | 192.168.48.9 | POST "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke" docker-plugin_daemon-1 | [GIN] 2025/03/13 - 01:48:51 | 200 | 1.421329705s | 192.168.48.9 | POST "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke" docker-plugin_daemon-1 | [GIN] 2025/03/13 - 01:48:51 | 200 | 1.410274884s | 192.168.48.9 | POST "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke" docker-plugin_daemon-1 | [GIN] 2025/03/13 - 01:48:51 | 200 | 1.412063739s | 192.168.48.9 | POST "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke" docker-plugin_daemon-1 | [GIN] 2025/03/13 - 01:48:51 | 200 | 1.410747272s | 192.168.48.9 | POST "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke" docker-plugin_daemon-1 | [GIN] 2025/03/13 - 01:48:51 | 200 | 1.411868744s | 192.168.48.9 | POST "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke" docker-plugin_daemon-1 | [GIN] 2025/03/13 - 01:48:51 | 200 | 1.422839868s | 192.168.48.9 | POST "/plugin/9e3d2f24-b1a2-4d4a-93ac-92ab508f4517/dispatch/text_embedding/invoke" docker-worker-1 | 2025-03-13 01:48:51.463 ERROR [ThreadPoolExecutor-37_0] [cached_embedding.py:98] - Failed to embed documents: %s docker-worker-1 | Traceback (most recent call last): docker-worker-1 | File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents docker-worker-1 | embedding_result = self._model_instance.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding docker-worker-1 | self._round_robin_invoke( docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke docker-worker-1 | return function(*args, **kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke docker-worker-1 | raise self._transform_invoke_error(e) docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke docker-worker-1 | return plugin_model_manager.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding docker-worker-1 | for resp in response: docker-worker-1 | ^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error docker-worker-1 | raise InvokeServerUnavailableError(description=args.get("description")) docker-worker-1 | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) docker-worker-1 | 2025-03-13 01:48:51.464 ERROR [ThreadPoolExecutor-37_5] [cached_embedding.py:98] - Failed to embed documents: %s docker-worker-1 | Traceback (most recent call last): docker-worker-1 | File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents docker-worker-1 | embedding_result = self._model_instance.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding docker-worker-1 | self._round_robin_invoke( docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke docker-worker-1 | return function(*args, **kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke docker-worker-1 | raise self._transform_invoke_error(e) docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke docker-worker-1 | return plugin_model_manager.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding docker-worker-1 | for resp in response: docker-worker-1 | ^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error docker-worker-1 | raise InvokeServerUnavailableError(description=args.get("description")) docker-worker-1 | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) docker-worker-1 | 2025-03-13 01:48:51.465 ERROR [ThreadPoolExecutor-37_3] [cached_embedding.py:98] - Failed to embed documents: %s docker-worker-1 | Traceback (most recent call last): docker-worker-1 | File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents docker-worker-1 | embedding_result = self._model_instance.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding docker-worker-1 | self._round_robin_invoke( docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke docker-worker-1 | return function(*args, **kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke docker-worker-1 | raise self._transform_invoke_error(e) docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke docker-worker-1 | return plugin_model_manager.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding docker-worker-1 | for resp in response: docker-worker-1 | ^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error docker-worker-1 | raise InvokeServerUnavailableError(description=args.get("description")) docker-worker-1 | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) docker-worker-1 | 2025-03-13 01:48:51.466 ERROR [ThreadPoolExecutor-37_7] [cached_embedding.py:98] - Failed to embed documents: %s docker-worker-1 | Traceback (most recent call last): docker-worker-1 | File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents docker-worker-1 | embedding_result = self._model_instance.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding docker-worker-1 | self._round_robin_invoke( docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke docker-worker-1 | return function(*args, **kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke docker-worker-1 | raise self._transform_invoke_error(e) docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke docker-worker-1 | return plugin_model_manager.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding docker-worker-1 | for resp in response: docker-worker-1 | ^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error docker-worker-1 | raise InvokeServerUnavailableError(description=args.get("description")) docker-worker-1 | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) docker-worker-1 | 2025-03-13 01:48:51.467 ERROR [ThreadPoolExecutor-37_6] [cached_embedding.py:98] - Failed to embed documents: %s docker-worker-1 | Traceback (most recent call last): docker-worker-1 | File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents docker-worker-1 | embedding_result = self._model_instance.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding docker-worker-1 | self._round_robin_invoke( docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke docker-worker-1 | return function(*args, **kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke docker-worker-1 | raise self._transform_invoke_error(e) docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke docker-worker-1 | return plugin_model_manager.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding docker-worker-1 | for resp in response: docker-worker-1 | ^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error docker-worker-1 | raise InvokeServerUnavailableError(description=args.get("description")) docker-worker-1 | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) docker-worker-1 | 2025-03-13 01:48:51.468 ERROR [ThreadPoolExecutor-37_8] [cached_embedding.py:98] - Failed to embed documents: %s docker-worker-1 | Traceback (most recent call last): docker-worker-1 | File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents docker-worker-1 | embedding_result = self._model_instance.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding docker-worker-1 | self._round_robin_invoke( docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke docker-worker-1 | return function(*args, **kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke docker-worker-1 | raise self._transform_invoke_error(e) docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke docker-worker-1 | return plugin_model_manager.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding docker-worker-1 | for resp in response: docker-worker-1 | ^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error docker-worker-1 | raise InvokeServerUnavailableError(description=args.get("description")) docker-worker-1 | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) docker-worker-1 | 2025-03-13 01:48:51.469 ERROR [ThreadPoolExecutor-37_2] [cached_embedding.py:98] - Failed to embed documents: %s docker-worker-1 | Traceback (most recent call last): docker-worker-1 | File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents docker-worker-1 | embedding_result = self._model_instance.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding docker-worker-1 | self._round_robin_invoke( docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke docker-worker-1 | return function(*args, **kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke docker-worker-1 | raise self._transform_invoke_error(e) docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke docker-worker-1 | return plugin_model_manager.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding docker-worker-1 | for resp in response: docker-worker-1 | ^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error docker-worker-1 | raise InvokeServerUnavailableError(description=args.get("description")) docker-worker-1 | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) docker-worker-1 | 2025-03-13 01:48:51.470 ERROR [ThreadPoolExecutor-37_9] [cached_embedding.py:98] - Failed to embed documents: %s docker-worker-1 | Traceback (most recent call last): docker-worker-1 | File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents docker-worker-1 | embedding_result = self._model_instance.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding docker-worker-1 | self._round_robin_invoke( docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke docker-worker-1 | return function(*args, **kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke docker-worker-1 | raise self._transform_invoke_error(e) docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke docker-worker-1 | return plugin_model_manager.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding docker-worker-1 | for resp in response: docker-worker-1 | ^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error docker-worker-1 | raise InvokeServerUnavailableError(description=args.get("description")) docker-worker-1 | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) docker-worker-1 | 2025-03-13 01:48:51.471 ERROR [ThreadPoolExecutor-37_1] [cached_embedding.py:98] - Failed to embed documents: %s docker-worker-1 | Traceback (most recent call last): docker-worker-1 | File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents docker-worker-1 | embedding_result = self._model_instance.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding docker-worker-1 | self._round_robin_invoke( docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke docker-worker-1 | return function(*args, **kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke docker-worker-1 | raise self._transform_invoke_error(e) docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke docker-worker-1 | return plugin_model_manager.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding docker-worker-1 | for resp in response: docker-worker-1 | ^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error docker-worker-1 | raise InvokeServerUnavailableError(description=args.get("description")) docker-worker-1 | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) docker-worker-1 | 2025-03-13 01:48:51.474 ERROR [ThreadPoolExecutor-37_4] [cached_embedding.py:98] - Failed to embed documents: %s docker-worker-1 | Traceback (most recent call last): docker-worker-1 | File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents docker-worker-1 | embedding_result = self._model_instance.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding docker-worker-1 | self._round_robin_invoke( docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke docker-worker-1 | return function(*args, **kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke docker-worker-1 | raise self._transform_invoke_error(e) docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke docker-worker-1 | return plugin_model_manager.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding docker-worker-1 | for resp in response: docker-worker-1 | ^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error docker-worker-1 | raise InvokeServerUnavailableError(description=args.get("description")) docker-worker-1 | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) docker-worker-1 | 2025-03-13 01:48:51.474 ERROR [Dummy-32] [indexing_runner.py:96] - consume document failed docker-worker-1 | Traceback (most recent call last): docker-worker-1 | File "/app/api/core/indexing_runner.py", line 80, in run docker-worker-1 | self._load( docker-worker-1 | File "/app/api/core/indexing_runner.py", line 570, in _load docker-worker-1 | tokens += future.result() docker-worker-1 | ^^^^^^^^^^^^^^^ docker-worker-1 | File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 456, in result docker-worker-1 | return self.__get_result() docker-worker-1 | ^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result docker-worker-1 | raise self._exception docker-worker-1 | File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 59, in run docker-worker-1 | result = self.fn(*self.args, **self.kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/indexing_runner.py", line 625, in _process_chunk docker-worker-1 | index_processor.load(dataset, chunk_documents, with_keywords=False) docker-worker-1 | File "/app/api/core/rag/index_processor/processor/paragraph_index_processor.py", line 78, in load docker-worker-1 | vector.create(documents) docker-worker-1 | File "/app/api/core/rag/datasource/vdb/vector_factory.py", line 156, in create docker-worker-1 | embeddings = self._embeddings.embed_documents([document.page_content for document in texts]) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/rag/embedding/cached_embedding.py", line 99, in embed_documents docker-worker-1 | raise ex docker-worker-1 | File "/app/api/core/rag/embedding/cached_embedding.py", line 61, in embed_documents docker-worker-1 | embedding_result = self._model_instance.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_manager.py", line 221, in invoke_text_embedding docker-worker-1 | self._round_robin_invoke( docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke docker-worker-1 | return function(*args, **kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 53, in invoke docker-worker-1 | raise self._transform_invoke_error(e) docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 42, in invoke docker-worker-1 | return plugin_model_manager.invoke_text_embedding( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 273, in invoke_text_embedding docker-worker-1 | for resp in response: docker-worker-1 | ^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 217, in _handle_plugin_daemon_error docker-worker-1 | raise InvokeServerUnavailableError(description=args.get("description")) docker-worker-1 | core.model_runtime.errors.invoke.InvokeServerUnavailableError: [models] Server Unavailable Error, ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) docker-worker-1 | 2025-03-13 01:48:51.481 INFO [Dummy-32] [document_indexing_task.py:76] - Processed dataset: 25c00224-e521-480d-bd7d-ce10cf27d426 latency: 3.663817903958261 docker-worker-1 | 2025-03-13 01:48:51.501 INFO [Dummy-32] [trace.py:128] - Task tasks.document_indexing_task.document_indexing_task[116ac803-96a6-49ed-8db2-b838919218be] succeeded in 3.6831789190182462s: None ``` ` the log of ollama please see crash log : [ollamalog.txt](https://github.com/user-attachments/files/19220212/ollamalog.txt)
yindo added the 🐞 bug👻 feat:rag labels 2026-02-21 18:50:18 -05:00
yindo closed this issue 2026-02-21 18:50:18 -05:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#10716