documents in pipeline keep in status queuing #862

Closed
opened 2026-02-16 10:20:46 -05:00 by yindo · 4 comments
Owner

Originally created by @simpsonyang on GitHub (Dec 9, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues Dify issues & Dify Official Plugins, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.10.1

Plugin version

0.0.4

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I created a pipeline with the markitdown template, then upload 3 files, but the status keeps in Queuing. I have already set FILES_URL=http://api:5001 in my .env file.

✔️ Error log

Exception in thread Thread-106 (_retriever):
Traceback (most recent call last):
File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.12/threading.py", line 1012, in run
self._target(*self._args, **self._kwargs)
File "/app/api/core/rag/retrieval/dataset_retrieval.py", line 652, in _retriever
documents = RetrievalService.retrieve(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/rag/datasource/retrieval_service.py", line 107, in retrieve
raise ValueError(";\n".join(exceptions))
ValueError: Query call with protocol GRPC search failed with message extract target vectors: class Vector_index_0d746ee2_719f_4c3b_b0ed_dd57a6dff939_Node does not have named vector default configured. Available named vectors map[].
2025-12-10 03:45:50.695 INFO [Thread-105 (_retriever)] [_client.py:1038] - HTTP Request: GET http://plugin_daemon:5002/plugin/97098ea7-f8e8-4ba4-a25b-bb1d22ccc1ec/management/models?page=1&page_size=256 "HTTP/1.1 200 OK"
/usr/local/lib/python3.12/json/decoder.py:354: ResourceWarning: unclosed <socket object, fd=57, family=2, type=1, proto=6>
obj, end = self.scan_once(s, idx)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/local/lib/python3.12/json/decoder.py:354: ResourceWarning: unclosed <socket object, fd=53, family=2, type=1, proto=6>
obj, end = self.scan_once(s, idx)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/local/lib/python3.12/json/decoder.py:354: ResourceWarning: unclosed <socket object, fd=49, family=2, type=1, proto=6>
obj, end = self.scan_once(s, idx)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/local/lib/python3.12/json/decoder.py:354: ResourceWarning: unclosed <socket object, fd=48, family=2, type=1, proto=6>
obj, end = self.scan_once(s, idx)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/app/api/core/model_runtime/entities/common_entities.py:12: ResourceWarning: unclosed <socket object, fd=44, family=2, type=1, proto=6>
@model_validator(mode="after")
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/app/api/core/model_runtime/entities/common_entities.py:12: ResourceWarning: unclosed <socket object, fd=50, family=2, type=1, proto=6>
@model_validator(mode="after")
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/app/api/core/model_runtime/entities/common_entities.py:12: ResourceWarning: unclosed <socket object, fd=51, family=2, type=1, proto=6>
@model_validator(mode="after")
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/app/api/core/model_runtime/entities/common_entities.py:12: ResourceWarning: unclosed <socket object, fd=52, family=2, type=1, proto=6>
@model_validator(mode="after")
ResourceWarning: Enable tracemalloc to get the object allocation traceback

Originally created by @simpsonyang on GitHub (Dec 9, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [Dify issues](https://github.com/langgenius/dify/issues) & [Dify Official Plugins](https://github.com/langgenius/dify-official-plugins/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.10.1 ### Plugin version 0.0.4 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce I created a pipeline with the markitdown template, then upload 3 files, but the status keeps in Queuing. I have already set FILES_URL=http://api:5001 in my .env file. ### ✔️ Error log Exception in thread Thread-106 (_retriever): Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1012, in run self._target(*self._args, **self._kwargs) File "/app/api/core/rag/retrieval/dataset_retrieval.py", line 652, in _retriever documents = RetrievalService.retrieve( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/core/rag/datasource/retrieval_service.py", line 107, in retrieve raise ValueError(";\n".join(exceptions)) ValueError: Query call with protocol GRPC search failed with message extract target vectors: class Vector_index_0d746ee2_719f_4c3b_b0ed_dd57a6dff939_Node does not have named vector default configured. Available named vectors map[]. 2025-12-10 03:45:50.695 INFO [Thread-105 (_retriever)] [_client.py:1038] - HTTP Request: GET http://plugin_daemon:5002/plugin/97098ea7-f8e8-4ba4-a25b-bb1d22ccc1ec/management/models?page=1&page_size=256 "HTTP/1.1 200 OK" /usr/local/lib/python3.12/json/decoder.py:354: ResourceWarning: unclosed <socket object, fd=57, family=2, type=1, proto=6> obj, end = self.scan_once(s, idx) ResourceWarning: Enable tracemalloc to get the object allocation traceback /usr/local/lib/python3.12/json/decoder.py:354: ResourceWarning: unclosed <socket object, fd=53, family=2, type=1, proto=6> obj, end = self.scan_once(s, idx) ResourceWarning: Enable tracemalloc to get the object allocation traceback /usr/local/lib/python3.12/json/decoder.py:354: ResourceWarning: unclosed <socket object, fd=49, family=2, type=1, proto=6> obj, end = self.scan_once(s, idx) ResourceWarning: Enable tracemalloc to get the object allocation traceback /usr/local/lib/python3.12/json/decoder.py:354: ResourceWarning: unclosed <socket object, fd=48, family=2, type=1, proto=6> obj, end = self.scan_once(s, idx) ResourceWarning: Enable tracemalloc to get the object allocation traceback /app/api/core/model_runtime/entities/common_entities.py:12: ResourceWarning: unclosed <socket object, fd=44, family=2, type=1, proto=6> @model_validator(mode="after") ResourceWarning: Enable tracemalloc to get the object allocation traceback /app/api/core/model_runtime/entities/common_entities.py:12: ResourceWarning: unclosed <socket object, fd=50, family=2, type=1, proto=6> @model_validator(mode="after") ResourceWarning: Enable tracemalloc to get the object allocation traceback /app/api/core/model_runtime/entities/common_entities.py:12: ResourceWarning: unclosed <socket object, fd=51, family=2, type=1, proto=6> @model_validator(mode="after") ResourceWarning: Enable tracemalloc to get the object allocation traceback /app/api/core/model_runtime/entities/common_entities.py:12: ResourceWarning: unclosed <socket object, fd=52, family=2, type=1, proto=6> @model_validator(mode="after") ResourceWarning: Enable tracemalloc to get the object allocation traceback
yindo added the bug label 2026-02-16 10:20:46 -05:00
yindo closed this issue 2026-02-16 10:20:46 -05:00
Author
Owner

@dosubot[bot] commented on GitHub (Dec 9, 2025):

Your pipeline is stuck in "Queuing" because the vector index in your vector database (like Qdrant) is missing a named vector called "default", which is now required by the gRPC search protocol. Dify's current collection creation logic uses a legacy configuration that doesn't define named vectors, so when the retrieval service tries to search, it fails with the error about the missing "default" named vector, and the pipeline can't proceed.

To fix this, you need to recreate the affected vector collection with a named vector called "default" in its configuration. In Qdrant, this means dropping the problematic collection (e.g., Vector_index_<dataset_id>_Node) and letting Dify re-create it, or manually creating it with the correct named vector schema. After that, re-upload your documents so the embeddings are stored in the new collection. This will allow the retrieval service to work and your pipeline to move past "Queuing" status.

This issue is due to recent changes in Qdrant's gRPC client expectations, and Dify's collection creation logic hasn't yet been updated to always use named vectors in all cases. More details on how the vector index is built and why the error occurs can be found in the Dify codebase and recent pull requests (source).

To reply, just mention @dosu.


How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Dec 9, 2025): <!-- Answer --> Your pipeline is stuck in "Queuing" because the vector index in your vector database (like Qdrant) is missing a named vector called "default", which is now required by the gRPC search protocol. Dify's current collection creation logic uses a legacy configuration that doesn't define named vectors, so when the retrieval service tries to search, it fails with the error about the missing "default" named vector, and the pipeline can't proceed. To fix this, you need to recreate the affected vector collection with a named vector called "default" in its configuration. In Qdrant, this means dropping the problematic collection (e.g., Vector_index_<dataset_id>_Node) and letting Dify re-create it, or manually creating it with the correct named vector schema. After that, re-upload your documents so the embeddings are stored in the new collection. This will allow the retrieval service to work and your pipeline to move past "Queuing" status. This issue is due to recent changes in Qdrant's gRPC client expectations, and Dify's collection creation logic hasn't yet been updated to always use named vectors in all cases. More details on how the vector index is built and why the error occurs can be found in the Dify codebase and recent pull requests ([source](https://github.com/langgenius/dify/blob/51330c0ee631de49f6cc5f7b7436c00cab62d38f/api/core/rag/datasource/vdb/qdrant/qdrant_vector.py#L86-L487)). <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/832fb292-e2a2-4877-a0e9-8134a35b3ba9?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/832fb292-e2a2-4877-a0e9-8134a35b3ba9?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/832fb292-e2a2-4877-a0e9-8134a35b3ba9?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/832fb292-e2a2-4877-a0e9-8134a35b3ba9?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/832fb292-e2a2-4877-a0e9-8134a35b3ba9?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/832fb292-e2a2-4877-a0e9-8134a35b3ba9?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/832fb292-e2a2-4877-a0e9-8134a35b3ba9?feedback_type=other)</sup> [![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=Your%20pipeline%20is%20stuck%20in%20%22Queuing%22%20because%20the%20vector%20index%20in%20your%20vector%20database%20%28like%20Qdrant%29%20is%20missing%20a%20named%20vector%20called%20%22default%22%2C%20which%20is%20now%20required%20by%20the%20gRPC%20search%20protocol.%20Dify%27s%20current%20collection%20creation%20logic%20uses%20a%20legacy%20configuration%20that%20doesn%27t%20define%20named%20vectors%2C%20so%20when%20the%20retrieval%20service%20tries%20to%20search%2C%20it%20fails%20with%20the%20error%20about%20the%20missing%20%22default%22%20named%20vector%2C%20and%20the%20pipeline%20can%27t%20proceed.%0A%0ATo%20fix%20this%2C%20you%20need%20to%20recreate%20the%20affected%20vector%20collection%20with%20a%20named%20vector%20called%20%22default%22%20in%20its%20configuration.%20In%20Qdrant%2C%20this%20means%20dropping%20the%20problematic%20collection%20%28e.g.%2C%20Vector_index_%3Cdataset_id%3E_Node%29%20and%20letting%20Dify%20re-create%20it%2C%20or%20manually%20creating%20it%20with%20the%20correct%20named%20vector%20schema.%20After%20that%2C%20re-upload%20your%20documents%20so%20the%20embeddings%20are%20stored%20in%20the%20new%20collection.%20This%20will%20allow%20the%20retrieval%20service%20to%20work%20and%20your%20pipeline%20to%20move%20past%20%22Queuing%22%20status.%0A%0AThis%20issue%20is%20due%20to%20recent%20changes%20in%20Qdrant%27s%20gRPC%20client%20expectations%2C%20and%20Dify%27s%20collection%20creation%20logic%20hasn%27t%20yet%20been%20updated%20to%20always%20use%20named%20vectors%20in%20all%20cases.%20More%20details%20on%20how%20the%20vector%20index%20is%20built%20and%20why%20the%20error%20occurs%20can%20be%20found%20in%20the%20Dify%20codebase%20and%20recent%20pull%20requests%20%28%5Bsource%5D%28https%3A//github.com/langgenius/dify/blob/51330c0ee631de49f6cc5f7b7436c00cab62d38f/api/core/rag/datasource/vdb/qdrant/qdrant_vector.py%23L86-L487%29%29.)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify-official-plugins/issues/2219)
Author
Owner

@simpsonyang commented on GitHub (Dec 10, 2025):

@dosu, there is no Qdrant in my Dify dockers, I can only see Weaviate.

@simpsonyang commented on GitHub (Dec 10, 2025): @dosu, there is no Qdrant in my Dify dockers, I can only see Weaviate.
Author
Owner

@dosubot[bot] commented on GitHub (Dec 10, 2025):

The error about the missing named vector "default" is a known issue with Weaviate after upgrading Dify or Weaviate. This happens because older Weaviate collections (created before Dify 1.9.2 or Weaviate 1.27.0) use an outdated schema that doesn't include the required "default" named vector, which Dify now expects for search and retrieval. The upgrade process does not automatically migrate these collections, and the vector configuration in Weaviate is immutable, so you can't patch it in place.

To fix this, you need to re-index or re-upload the affected knowledge/documents so the collections are recreated with the correct named vector configuration. Changing the embedding model of the knowledge base can also trigger an automatic index rebuild with the right schema. For large knowledge bases, scripting the export/import or re-upload is recommended. There is no automated migration script—manual intervention is required. If the error persists after re-ingesting, check the Weaviate schema via API or UI to confirm the collection has a named vector "default" configured, and if not, manually delete and recreate the collection. More details are in the official Weaviate migration guide for Dify
(source 1, source 2, source 3).

If your pipeline is still stuck in "Queuing" after fixing the vector schema, double-check that your Celery worker is listening to both the pipeline and priority_pipeline queues, and that all required services (Redis, vector DB, embedding provider) are healthy.

To reply, just mention @dosu.


How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Dec 10, 2025): <!-- Answer --> The error about the missing named vector "default" is a known issue with Weaviate after upgrading Dify or Weaviate. This happens because older Weaviate collections (created before Dify 1.9.2 or Weaviate 1.27.0) use an outdated schema that doesn't include the required "default" named vector, which Dify now expects for search and retrieval. The upgrade process does not automatically migrate these collections, and the vector configuration in Weaviate is immutable, so you can't patch it in place. To fix this, you need to re-index or re-upload the affected knowledge/documents so the collections are recreated with the correct named vector configuration. Changing the embedding model of the knowledge base can also trigger an automatic index rebuild with the right schema. For large knowledge bases, scripting the export/import or re-upload is recommended. There is no automated migration script—manual intervention is required. If the error persists after re-ingesting, check the Weaviate schema via API or UI to confirm the collection has a named vector "default" configured, and if not, manually delete and recreate the collection. More details are in the [official Weaviate migration guide for Dify](https://docs.dify.ai/en/development/migration/weaviate-v4-migration) ([source 1](https://github.com/langgenius/dify/issues/27695), [source 2](https://github.com/langgenius/dify/issues/27291), [source 3](https://github.com/langgenius/dify/discussions/27328)). If your pipeline is still stuck in "Queuing" after fixing the vector schema, double-check that your Celery worker is listening to both the pipeline and priority_pipeline queues, and that all required services (Redis, vector DB, embedding provider) are healthy. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/08ddce17-bfb4-4e85-bae3-966e12bc5b6e?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/08ddce17-bfb4-4e85-bae3-966e12bc5b6e?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/08ddce17-bfb4-4e85-bae3-966e12bc5b6e?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/08ddce17-bfb4-4e85-bae3-966e12bc5b6e?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/08ddce17-bfb4-4e85-bae3-966e12bc5b6e?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/08ddce17-bfb4-4e85-bae3-966e12bc5b6e?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/08ddce17-bfb4-4e85-bae3-966e12bc5b6e?feedback_type=other)</sup> [![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=The%20error%20about%20the%20missing%20named%20vector%20%22default%22%20is%20a%20known%20issue%20with%20Weaviate%20after%20upgrading%20Dify%20or%20Weaviate.%20This%20happens%20because%20older%20Weaviate%20collections%20%28created%20before%20Dify%201.9.2%20or%20Weaviate%201.27.0%29%20use%20an%20outdated%20schema%20that%20doesn%27t%20include%20the%20required%20%22default%22%20named%20vector%2C%20which%20Dify%20now%20expects%20for%20search%20and%20retrieval.%20The%20upgrade%20process%20does%20not%20automatically%20migrate%20these%20collections%2C%20and%20the%20vector%20configuration%20in%20Weaviate%20is%20immutable%2C%20so%20you%20can%27t%20patch%20it%20in%20place.%0A%0ATo%20fix%20this%2C%20you%20need%20to%20re-index%20or%20re-upload%20the%20affected%20knowledge/documents%20so%20the%20collections%20are%20recreated%20with%20the%20correct%20named%20vector%20configuration.%20Changing%20the%20embedding%20model%20of%20the%20knowledge%20base%20can%20also%20trigger%20an%20automatic%20index%20rebuild%20with%20the%20right%20schema.%20For%20large%20knowledge%20bases%2C%20scripting%20the%20export/import%20or%20re-upload%20is%20recommended.%20There%20is%20no%20automated%20migration%20script%E2%80%94manual%20intervention%20is%20required.%20If%20the%20error%20persists%20after%20re-ingesting%2C%20check%20the%20Weaviate%20schema%20via%20API%20or%20UI%20to%20confirm%20the%20collection%20has%20a%20named%20vector%20%22default%22%20configured%2C%20and%20if%20not%2C%20manually%20delete%20and%20recreate%20the%20collection.%20More%20details%20are%20in%20the%20%5Bofficial%20Weaviate%20migration%20guide%20for%20Dify%5D%28https%3A//docs.dify.ai/en/development/migration/weaviate-v4-migration%29%20%20%0A%28%5Bsource%201%5D%28https%3A//github.com/langgenius/dify/issues/27695%29%2C%20%5Bsource%202%5D%28https%3A//github.com/langgenius/dify/issues/27291%29%2C%20%5Bsource%203%5D%28https%3A//github.com/langgenius/dify/discussions/27328%29%29.%0A%0AIf%20your%20pipeline%20is%20still%20stuck%20in%20%22Queuing%22%20after%20fixing%20the%20vector%20schema%2C%20double-check%20that%20your%20Celery%20worker%20is%20listening%20to%20both%20the%20pipeline%20and%20priority_pipeline%20queues%2C%20and%20that%20all%20required%20services%20%28Redis%2C%20vector%20DB%2C%20embedding%20provider%29%20are%20healthy.)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify-official-plugins/issues/2219)
Author
Owner

@simpsonyang commented on GitHub (Dec 10, 2025):

意思明白了,还是找DS给了详细的操作步骤,解决了。

@simpsonyang commented on GitHub (Dec 10, 2025): 意思明白了,还是找DS给了详细的操作步骤,解决了。
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#862