langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-08-27 11:11:21 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Always display "In Queue" after upload the file for knowledgebase #20453

New Issue
Closed
opened 2026-02-21 20:07:30 -05:00 by yindo · 2 comments
yindo commented 2026-02-21 20:07:30 -05:00
Owner
Copy Link
Copy Source

Originally created by @ultradawn on GitHub (Nov 20, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.10

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

Why it's always show "In queue", after I uploaded the file. I start the backend API and web service by code. No error in API backend and the middleware docker container.

Image

2025-11-20 13:13:09,672 INFO [_client.py:1038] 6642906b19 HTTP Request: GET http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/management/models?page=1&page_size=256 "HTTP/1.1 200 OK"
2025-11-20 13:13:09,701 INFO [_client.py:1038] 6642906b19 HTTP Request: GET http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/asset/bfff83a66922c09cb5a5aa68829742b8b4f4e818579db42f53c7b8a30912cd8b.svg "HTTP/1.1 200 OK"
2025-11-20 13:13:09,701 INFO [_client.py:1038] 35bccfdb05 HTTP Request: GET http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/management/models?page=1&page_size=256 "HTTP/1.1 200 OK"
2025-11-20 13:13:09,703 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:09] "GET /console/api/workspaces/100392f6-3771-4bd3-b605-ec4fc857844a/model-providers/langgenius/ollama/ollama/icon_small/zh_Hans HTTP/1.1" 200 -
2025-11-20 13:13:09,710 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:09] "GET /console/api/workspaces/current/default-model?model_type=rerank HTTP/1.1" 200 -
2025-11-20 13:13:11,549 INFO [_client.py:1038] 6d9c3e024e HTTP Request: GET http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/management/models?page=1&page_size=256 "HTTP/1.1 200 OK"
2025-11-20 13:13:11,573 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:11] "POST /console/api/datasets/indexing-estimate HTTP/1.1" 200 -
2025-11-20 13:13:15,76 INFO [document_indexing_task_proxy.py:64] 26a7bdb9f8 dispatch args: 100392f6-3771-4bd3-b605-ec4fc857844a - False - sandbox
2025-11-20 13:13:15,76 INFO [document_indexing_task_proxy.py:27] 26a7bdb9f8 send dataset 647b3f59-c023-4d61-b497-3fbba92c5e5e to direct queue
2025-11-20 13:13:15,98 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:15] "POST /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e/documents HTTP/1.1" 200 -
2025-11-20 13:13:15,128 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:15] "GET /console/api/datasets/process-rule?document_id=807394c7-97df-48f2-9d45-8acd44b96d2b HTTP/1.1" 200 -
2025-11-20 13:13:15,128 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:15] "GET /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e/batch/20251120131315371167/indexing-status HTTP/1.1" 200 -
2025-11-20 13:13:15,142 INFO [_client.py:1038] aab72b6403 HTTP Request: GET http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/management/models?page=1&page_size=256 "HTTP/1.1 200 OK"
2025-11-20 13:13:15,145 INFO [_client.py:1038] 40a1666db4 HTTP Request: GET http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/management/models?page=1&page_size=256 "HTTP/1.1 200 OK"
2025-11-20 13:13:15,170 INFO [_client.py:1038] aab72b6403 HTTP Request: POST http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/dispatch/model/schema "HTTP/1.1 200 OK"
2025-11-20 13:13:15,172 INFO [_client.py:1038] 40a1666db4 HTTP Request: POST http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/dispatch/model/schema "HTTP/1.1 200 OK"
2025-11-20 13:13:15,174 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:15] "GET /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e HTTP/1.1" 200 -
2025-11-20 13:13:15,180 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:15] "GET /console/api/datasets?limit=30&page=1 HTTP/1.1" 200 -
2025-11-20 13:13:17,674 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:17] "GET /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e/batch/20251120131315371167/indexing-status HTTP/1.1" 200 -
2025-11-20 13:13:20,199 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:20] "GET /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e/batch/20251120131315371167/indexing-status HTTP/1.1" 200 -
2025-11-20 13:13:22,728 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:22] "GET /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e/batch/20251120131315371167/indexing-status HTTP/1.1" 200 -
2025-11-20 13:13:25,267 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:25] "GET /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e/batch/20251120131315371167/indexing-status HTTP/1.1" 200 -

✔️ Expected Behavior

Expected it could perform the index

❌ Actual Behavior

No response

Originally created by @ultradawn on GitHub (Nov 20, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.10 ### Cloud or Self Hosted Self Hosted (Source) ### Steps to reproduce Why it's always show "In queue", after I uploaded the file. I start the backend API and web service by code. No error in API backend and the middleware docker container. <img width="1007" height="273" alt="Image" src="https://github.com/user-attachments/assets/c53a4e3e-37fe-4482-9266-e8b77d6d43dd" /> ------------------------------------------------------------- 2025-11-20 13:13:09,672 INFO [_client.py:1038] 6642906b19 HTTP Request: GET http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/management/models?page=1&page_size=256 "HTTP/1.1 200 OK" 2025-11-20 13:13:09,701 INFO [_client.py:1038] 6642906b19 HTTP Request: GET http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/asset/bfff83a66922c09cb5a5aa68829742b8b4f4e818579db42f53c7b8a30912cd8b.svg "HTTP/1.1 200 OK" 2025-11-20 13:13:09,701 INFO [_client.py:1038] 35bccfdb05 HTTP Request: GET http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/management/models?page=1&page_size=256 "HTTP/1.1 200 OK" 2025-11-20 13:13:09,703 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:09] "GET /console/api/workspaces/100392f6-3771-4bd3-b605-ec4fc857844a/model-providers/langgenius/ollama/ollama/icon_small/zh_Hans HTTP/1.1" 200 - 2025-11-20 13:13:09,710 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:09] "GET /console/api/workspaces/current/default-model?model_type=rerank HTTP/1.1" 200 - 2025-11-20 13:13:11,549 INFO [_client.py:1038] 6d9c3e024e HTTP Request: GET http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/management/models?page=1&page_size=256 "HTTP/1.1 200 OK" 2025-11-20 13:13:11,573 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:11] "POST /console/api/datasets/indexing-estimate HTTP/1.1" 200 - 2025-11-20 13:13:15,76 INFO [document_indexing_task_proxy.py:64] 26a7bdb9f8 dispatch args: 100392f6-3771-4bd3-b605-ec4fc857844a - False - sandbox 2025-11-20 13:13:15,76 INFO [document_indexing_task_proxy.py:27] 26a7bdb9f8 send dataset 647b3f59-c023-4d61-b497-3fbba92c5e5e to direct queue 2025-11-20 13:13:15,98 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:15] "POST /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e/documents HTTP/1.1" 200 - 2025-11-20 13:13:15,128 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:15] "GET /console/api/datasets/process-rule?document_id=807394c7-97df-48f2-9d45-8acd44b96d2b HTTP/1.1" 200 - 2025-11-20 13:13:15,128 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:15] "GET /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e/batch/20251120131315371167/indexing-status HTTP/1.1" 200 - 2025-11-20 13:13:15,142 INFO [_client.py:1038] aab72b6403 HTTP Request: GET http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/management/models?page=1&page_size=256 "HTTP/1.1 200 OK" 2025-11-20 13:13:15,145 INFO [_client.py:1038] 40a1666db4 HTTP Request: GET http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/management/models?page=1&page_size=256 "HTTP/1.1 200 OK" 2025-11-20 13:13:15,170 INFO [_client.py:1038] aab72b6403 HTTP Request: POST http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/dispatch/model/schema "HTTP/1.1 200 OK" 2025-11-20 13:13:15,172 INFO [_client.py:1038] 40a1666db4 HTTP Request: POST http://127.0.0.1:5002/plugin/100392f6-3771-4bd3-b605-ec4fc857844a/dispatch/model/schema "HTTP/1.1 200 OK" 2025-11-20 13:13:15,174 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:15] "GET /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e HTTP/1.1" 200 - 2025-11-20 13:13:15,180 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:15] "GET /console/api/datasets?limit=30&page=1 HTTP/1.1" 200 - 2025-11-20 13:13:17,674 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:17] "GET /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e/batch/20251120131315371167/indexing-status HTTP/1.1" 200 - 2025-11-20 13:13:20,199 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:20] "GET /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e/batch/20251120131315371167/indexing-status HTTP/1.1" 200 - 2025-11-20 13:13:22,728 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:22] "GET /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e/batch/20251120131315371167/indexing-status HTTP/1.1" 200 - 2025-11-20 13:13:25,267 INFO [_internal.py:97] 127.0.0.1 - - [20/Nov/2025 13:13:25] "GET /console/api/datasets/647b3f59-c023-4d61-b497-3fbba92c5e5e/batch/20251120131315371167/indexing-status HTTP/1.1" 200 - ### ✔️ Expected Behavior Expected it could perform the index ### ❌ Actual Behavior _No response_
yindo added the 🐞 bug label 2026-02-21 20:07:30 -05:00
yindo closed this issue 2026-02-21 20:07:30 -05:00
yindo commented 2026-02-21 20:07:30 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Nov 20, 2025):

The "In Queue" status after uploading a knowledgebase file usually means the background indexing task hasn't started. This is almost always caused by the Celery worker not running, not listening to the correct queues, or a misconfiguration in your self-hosted setup. For Dify v1.10, make sure your Celery worker is started with both the pipeline and priority_pipeline queues, like this:

celery -A api.tasks.celery worker -Q pipeline,priority_pipeline --concurrency=1 --loglevel=INFO

If you use Docker Compose, check that the worker service includes both queues in its command or environment. If you modified the entrypoint or environment, ensure priority_pipeline is included in CELERY_QUEUES (some users fixed the issue by adding it) (example).

Also, confirm all required services are running: api, worker, web, weaviate, db, redis, and nginx. Use docker compose ps or your process manager to check. If you hit the rate limit for your embedding model API key, files can also get stuck—check your API usage and limits (reference).

If everything looks correct but files are still stuck, try restarting the worker service and check its logs for errors or warnings about unconsumed queues. Upgrading to the latest Dify version is recommended, as several queue and indexing reliability fixes have been made (reference).

If you need to recover stuck files, Dify has backend recovery tasks (like recover_document_indexing_task) that can be triggered via API or admin scripts (reference).

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

@dosubot[bot] commented on GitHub (Nov 20, 2025): <!-- Answer --> The "In Queue" status after uploading a knowledgebase file usually means the background indexing task hasn't started. This is almost always caused by the Celery worker not running, not listening to the correct queues, or a misconfiguration in your self-hosted setup. For Dify v1.10, make sure your Celery worker is started with both the `pipeline` and `priority_pipeline` queues, like this: ```bash celery -A api.tasks.celery worker -Q pipeline,priority_pipeline --concurrency=1 --loglevel=INFO ``` If you use Docker Compose, check that the worker service includes both queues in its command or environment. If you modified the entrypoint or environment, ensure `priority_pipeline` is included in `CELERY_QUEUES` (some users fixed the issue by adding it) [(example)](https://github.com/langgenius/dify/issues/27473). Also, confirm all required services are running: `api`, `worker`, `web`, `weaviate`, `db`, `redis`, and `nginx`. Use `docker compose ps` or your process manager to check. If you hit the rate limit for your embedding model API key, files can also get stuck—check your API usage and limits [(reference)](https://github.com/langgenius/dify/issues/12959). If everything looks correct but files are still stuck, try restarting the worker service and check its logs for errors or warnings about unconsumed queues. Upgrading to the latest Dify version is recommended, as several queue and indexing reliability fixes have been made [(reference)](https://github.com/langgenius/dify/issues/18386). If you need to recover stuck files, Dify has backend recovery tasks (like `recover_document_indexing_task`) that can be triggered via API or admin scripts [(reference)](https://github.com/langgenius/dify/issues/20327). <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/6fa65b75-e239-4611-bced-74756f9482bc?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/6fa65b75-e239-4611-bced-74756f9482bc?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/6fa65b75-e239-4611-bced-74756f9482bc?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/6fa65b75-e239-4611-bced-74756f9482bc?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/6fa65b75-e239-4611-bced-74756f9482bc?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/6fa65b75-e239-4611-bced-74756f9482bc?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/6fa65b75-e239-4611-bced-74756f9482bc?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/28468)
yindo commented 2026-02-21 20:07:31 -05:00
Author
Owner
Copy Link
Copy Source

@ultradawn commented on GitHub (Nov 20, 2025):

I solved the issue with following cmd. Thanks

uv run celery -A app.celery worker -P threads -c 2 --loglevel INFO -Q dataset,priority_dataset,priority_pipeline,pipeline,mail,ops_trace,app_deletion,plugin,workflow_storage,conversation,workflow,schedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_executor

@ultradawn commented on GitHub (Nov 20, 2025): I solved the issue with following cmd. Thanks ---------------------- uv run celery -A app.celery worker -P threads -c 2 --loglevel INFO -Q dataset,priority_dataset,priority_pipeline,pipeline,mail,ops_trace,app_deletion,plugin,workflow_storage,conversation,workflow,schedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_executor
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#20453
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?