langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Parallel nodes get blocked during workflow execution. #16955

New Issue
Closed
opened 2026-02-21 19:29:31 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 19:29:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @easongfan on GitHub (Sep 9, 2025).

Originally assigned to: @Nov1c444 on GitHub.

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

Version 1.8.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Image

Description

I'm encountering a concurrency issue with parallel nodes during workflow execution, specifically when the nodes are LLM calls. While the workflow completes successfully and produces the correct final result, the execution process itself is not running with the expected level of parallelism.

Actual Behavior

During execution, when the workflow reaches a set of parallel branches containing LLM nodes (as indicated by the red lines in the screenshot), a maximum of only 5 LLM nodes run concurrently.

All other parallel LLM nodes remain in a blocked state. They only start processing after one of the currently active nodes has finished its execution. This creates a bottleneck and significantly slows down the workflow.

Expected Behavior

Based on my configuration, I expect a much higher number of parallel LLM nodes to execute simultaneously, as defined by settings like MAX_PARALLEL_LIMIT, CELERY_WORKER_AMOUNT, and CELERY_MAX_WORKERS. The concurrency should not be hard-capped at 5.

Troubleshooting Steps Taken

I have tried to resolve this by adjusting the following configuration parameters, but it had no effect. The concurrency limit remains at 5.

Workflow runtime configuration

WORKFLOW_MAX_EXECUTION_STEPS=500
WORKFLOW_MAX_EXECUTION_TIME=1200
WORKFLOW_CALL_MAX_DEPTH=500
MAX_VARIABLE_SIZE=204800
WORKFLOW_PARALLEL_DEPTH_LIMIT=300
WORKFLOW_FILE_UPLOAD_LIMIT=10

The maximum number of tools that can be used in the agent.

MAX_TOOLS_NUM=100

Maximum number of Parallelism branches in the workflow

MAX_PARALLEL_LIMIT=500

The maximum number of iterations for agent setting

MAX_ITERATIONS_NUM=999

The number of Celery workers. The default is 1, and can be set as needed.

CELERY_WORKER_AMOUNT=10

Flag indicating whether to enable autoscaling of Celery workers.

CELERY_AUTO_SCALE=true

The maximum number of Celery workers that can be autoscaled.

CELERY_MAX_WORKERS=15

The minimum number of Celery workers that can be autoscaled.

CELERY_MIN_WORKERS=5

It seems there might be another configuration or a hard-coded limit that is throttling the execution of parallel LLM nodes specifically.

✔️ Expected Behavior

All parallel nodes should be able to run concurrently.

❌ Actual Behavior

The concurrent processing gets blocked when it encounters LLM nodes.

Originally created by @easongfan on GitHub (Sep 9, 2025). Originally assigned to: @Nov1c444 on GitHub. ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version Version 1.8.1 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce <img width="885" height="697" alt="Image" src="https://github.com/user-attachments/assets/0e81af93-abeb-4f66-9cb5-61f3d9c4f23f" /> Description I'm encountering a concurrency issue with parallel nodes during workflow execution, specifically when the nodes are LLM calls. While the workflow completes successfully and produces the correct final result, the execution process itself is not running with the expected level of parallelism. Actual Behavior During execution, when the workflow reaches a set of parallel branches containing LLM nodes (as indicated by the red lines in the screenshot), a maximum of only 5 LLM nodes run concurrently. All other parallel LLM nodes remain in a blocked state. They only start processing after one of the currently active nodes has finished its execution. This creates a bottleneck and significantly slows down the workflow. Expected Behavior Based on my configuration, I expect a much higher number of parallel LLM nodes to execute simultaneously, as defined by settings like MAX_PARALLEL_LIMIT, CELERY_WORKER_AMOUNT, and CELERY_MAX_WORKERS. The concurrency should not be hard-capped at 5. Troubleshooting Steps Taken I have tried to resolve this by adjusting the following configuration parameters, but it had no effect. The concurrency limit remains at 5. # Workflow runtime configuration WORKFLOW_MAX_EXECUTION_STEPS=500 WORKFLOW_MAX_EXECUTION_TIME=1200 WORKFLOW_CALL_MAX_DEPTH=500 MAX_VARIABLE_SIZE=204800 WORKFLOW_PARALLEL_DEPTH_LIMIT=300 WORKFLOW_FILE_UPLOAD_LIMIT=10 # The maximum number of tools that can be used in the agent. MAX_TOOLS_NUM=100 # Maximum number of Parallelism branches in the workflow MAX_PARALLEL_LIMIT=500 # The maximum number of iterations for agent setting MAX_ITERATIONS_NUM=999 # The number of Celery workers. The default is 1, and can be set as needed. CELERY_WORKER_AMOUNT=10 # Flag indicating whether to enable autoscaling of Celery workers. CELERY_AUTO_SCALE=true # The maximum number of Celery workers that can be autoscaled. CELERY_MAX_WORKERS=15 # The minimum number of Celery workers that can be autoscaled. CELERY_MIN_WORKERS=5 It seems there might be another configuration or a hard-coded limit that is throttling the execution of parallel LLM nodes specifically. ### ✔️ Expected Behavior All parallel nodes should be able to run concurrently. ### ❌ Actual Behavior The concurrent processing gets blocked when it encounters LLM nodes.
yindo added the 🐞 bug label 2026-02-21 19:29:31 -05:00
yindo closed this issue 2026-02-21 19:29:31 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#16955
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?