docker-worker-1 | core.plugin.manager.exc.PluginInvokeError: PluginInvokeError: {"args":{},"error_type":"RuntimeError","message":"can't start new thread"} #105

Closed
opened 2026-02-16 11:19:46 -05:00 by yindo · 1 comment
Owner

Originally created by @yuyeshuhuai on GitHub (Mar 11, 2025).

The error always comes out after user uses Dify for a while.
Sometimes it happens in the phase of knowledge base indexing( based on LLM for embedding), sometimes in the phase of user communicating with LLM. I am sure that is all relevant to calling LLM.

Once this comes out, I will restart docker containers until it shows up next time.
Did you have this kind of problem? Any suggestion will be appreciated.
Logs in detail as follows:

docker-worker-1 | 2025-03-11 09:56:27.368 ERROR [Dummy-23] [indexing_runner.py:96] - consume document failed
docker-worker-1 | Traceback (most recent call last):
docker-worker-1 | File "/app/api/core/indexing_runner.py", line 73, in run
docker-worker-1 | documents = self._transform(
docker-worker-1 | ^^^^^^^^^^^^^^^^
docker-worker-1 | File "/app/api/core/indexing_runner.py", line 706, in _transform
docker-worker-1 | documents = index_processor.transform(
docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1 | File "/app/api/core/rag/index_processor/processor/parent_child_index_processor.py", line 83, in transform
docker-worker-1 | child_nodes = self._split_child_nodes(
docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1 | File "/app/api/core/rag/index_processor/processor/parent_child_index_processor.py", line 186, in _split_child_nodes
docker-worker-1 | child_documents = child_splitter.split_documents([document_node])
docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1 | File "/app/api/core/rag/splitter/text_splitter.py", line 96, in split_documents
docker-worker-1 | return self.create_documents(texts, metadatas=metadatas)
docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1 | File "/app/api/core/rag/splitter/text_splitter.py", line 81, in create_documents
docker-worker-1 | for chunk in self.split_text(text):
docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^
docker-worker-1 | File "/app/api/core/rag/splitter/fixed_text_splitter.py", line 68, in split_text
docker-worker-1 | chunks_lengths = self._length_function(chunks)
docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1 | File "/app/api/core/rag/splitter/fixed_text_splitter.py", line 38, in _token_encoder
docker-worker-1 | return embedding_model_instance.get_text_embedding_num_tokens(texts=texts)
docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1 | File "/app/api/core/model_manager.py", line 244, in get_text_embedding_num_tokens
docker-worker-1 | self._round_robin_invoke(
docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke
docker-worker-1 | return function(*args, **kwargs)
docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 65, in get_num_tokens
docker-worker-1 | return plugin_model_manager.get_text_embedding_num_tokens(
docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 313, in get_text_embedding_num_tokens
docker-worker-1 | for resp in response:
docker-worker-1 | ^^^^^^^^
docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message)
docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 221, in _handle_plugin_daemon_error
docker-worker-1 | raise PluginInvokeError(description=message)
docker-worker-1 | core.plugin.manager.exc.PluginInvokeError: PluginInvokeError: {"args":{},"error_type":"RuntimeError","message":"can't start new thread"}
docker-worker-1 | 2025-03-11 09:56:27.381 INFO [Dummy-23] [document_indexing_task.py:76] - Processed dataset: fd49a7a6-0946-4512-acbb-9f297a1538ed latency: 0.0918743850197643
docker-plugin_daemon-1 | [GIN] 2025/03/11 - 09:56:27 | 200 | 18.583202ms | 172.19.0.9 | GET "/plugin/2d64604d-dae0-4e51-9cdb-0ebebf24b427/management/models?page=1&page_size=256"

Originally created by @yuyeshuhuai on GitHub (Mar 11, 2025). The error always comes out after user uses Dify for a while. Sometimes it happens in the phase of knowledge base indexing( based on LLM for embedding), sometimes in the phase of user communicating with LLM. I am sure that is all relevant to calling LLM. Once this comes out, I will restart docker containers until it shows up next time. Did you have this kind of problem? Any suggestion will be appreciated. Logs in detail as follows: docker-worker-1 | 2025-03-11 09:56:27.368 ERROR [Dummy-23] [indexing_runner.py:96] - consume document failed docker-worker-1 | Traceback (most recent call last): docker-worker-1 | File "/app/api/core/indexing_runner.py", line 73, in run docker-worker-1 | documents = self._transform( docker-worker-1 | ^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/indexing_runner.py", line 706, in _transform docker-worker-1 | documents = index_processor.transform( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/rag/index_processor/processor/parent_child_index_processor.py", line 83, in transform docker-worker-1 | child_nodes = self._split_child_nodes( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/rag/index_processor/processor/parent_child_index_processor.py", line 186, in _split_child_nodes docker-worker-1 | child_documents = child_splitter.split_documents([document_node]) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/rag/splitter/text_splitter.py", line 96, in split_documents docker-worker-1 | return self.create_documents(texts, metadatas=metadatas) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/rag/splitter/text_splitter.py", line 81, in create_documents docker-worker-1 | for chunk in self.split_text(text): docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/rag/splitter/fixed_text_splitter.py", line 68, in split_text docker-worker-1 | chunks_lengths = self._length_function(chunks) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/rag/splitter/fixed_text_splitter.py", line 38, in _token_encoder docker-worker-1 | return embedding_model_instance.get_text_embedding_num_tokens(texts=texts) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_manager.py", line 244, in get_text_embedding_num_tokens docker-worker-1 | self._round_robin_invoke( docker-worker-1 | File "/app/api/core/model_manager.py", line 370, in _round_robin_invoke docker-worker-1 | return function(*args, **kwargs) docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 65, in get_num_tokens docker-worker-1 | return plugin_model_manager.get_text_embedding_num_tokens( docker-worker-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/model.py", line 313, in get_text_embedding_num_tokens docker-worker-1 | for resp in response: docker-worker-1 | ^^^^^^^^ docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream docker-worker-1 | self._handle_plugin_daemon_error(error.error_type, error.message) docker-worker-1 | File "/app/api/core/plugin/manager/base.py", line 221, in _handle_plugin_daemon_error docker-worker-1 | raise PluginInvokeError(description=message) docker-worker-1 | core.plugin.manager.exc.PluginInvokeError: PluginInvokeError: {"args":{},"error_type":"RuntimeError","message":"can't start new thread"} docker-worker-1 | 2025-03-11 09:56:27.381 INFO [Dummy-23] [document_indexing_task.py:76] - Processed dataset: fd49a7a6-0946-4512-acbb-9f297a1538ed latency: 0.0918743850197643 docker-plugin_daemon-1 | [GIN] 2025/03/11 - 09:56:27 | 200 | 18.583202ms | 172.19.0.9 | GET "/plugin/2d64604d-dae0-4e51-9cdb-0ebebf24b427/management/models?page=1&page_size=256"
yindo added the bug label 2026-02-16 11:19:46 -05:00
yindo closed this issue 2026-02-16 11:19:46 -05:00
Author
Owner

@dosubot[bot] commented on GitHub (Apr 21, 2025):

Hi, @yuyeshuhuai. I'm Dosu, and I'm helping the dify-docs team manage their backlog. I'm marking this issue as stale.

Issue Summary:

  • You reported a recurring threading error, "can't start new thread," in a Docker environment while using Dify.
  • The error occurs during knowledge base indexing or interactions with a language model.
  • You've been temporarily resolving the issue by restarting Docker containers.
  • No comments or developments have been made on this issue yet.

Next Steps:

  • Please let me know if this issue is still relevant to the latest version of the dify-docs repository by commenting here to keep the discussion open.
  • If there is no further activity, the issue will be automatically closed in 7 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Apr 21, 2025): Hi, @yuyeshuhuai. I'm [Dosu](https://dosu.dev), and I'm helping the dify-docs team manage their backlog. I'm marking this issue as stale. **Issue Summary:** - You reported a recurring threading error, "can't start new thread," in a Docker environment while using Dify. - The error occurs during knowledge base indexing or interactions with a language model. - You've been temporarily resolving the issue by restarting Docker containers. - No comments or developments have been made on this issue yet. **Next Steps:** - Please let me know if this issue is still relevant to the latest version of the dify-docs repository by commenting here to keep the discussion open. - If there is no further activity, the issue will be automatically closed in 7 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-docs-archived#105