langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 12:36:14 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Slow Responding in Agent Node #21783

New Issue
Closed
opened 2026-02-21 20:14:17 -05:00 by yindo · 8 comments
yindo commented 2026-02-21 20:14:17 -05:00
Owner
Copy Link
Copy Source

Originally created by @zio525 on GitHub (Jan 19, 2026).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.11.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

As shown in images, it takes about 11 seconds from workflow beginning to the first piece of message. And the log also shows that it takes about 11 seconds from beginning to the end of first round of the Agent Node.
It seems that the Agent start responding after the end of single round, instead of outputting the text first and waiting for tool response. The model of the Agent Node is GPT-5.

I remember that it works well when i use deepseek model.

Image

Image

Image

✔️ Expected Behavior

responding streaming text first

❌ Actual Behavior

The Agent will wait for the end of single round, and then reply the text

Originally created by @zio525 on GitHub (Jan 19, 2026). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.11.2 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce As shown in images, it takes about 11 seconds from workflow beginning to the first piece of message. And the log also shows that it takes about 11 seconds from beginning to the end of first round of the Agent Node. It seems that the Agent start responding after the end of single round, instead of outputting the text first and waiting for tool response. The model of the Agent Node is GPT-5. I remember that it works well when i use deepseek model. <img width="2559" height="1405" alt="Image" src="https://github.com/user-attachments/assets/e1f5600e-9e7f-4ec3-87d4-df3738c7b197" /> ![Image](https://github.com/user-attachments/assets/d58cc242-b010-4b8f-b4f3-6c93f2ee45e7) ![Image](https://github.com/user-attachments/assets/9b9d8f3a-76e4-43b0-a6ca-a3f7f7a3835f) ### ✔️ Expected Behavior responding streaming text first ### ❌ Actual Behavior The Agent will wait for the end of single round, and then reply the text
yindo added the 🐞 bug🤖 feat:agent labels 2026-02-21 20:14:17 -05:00
yindo closed this issue 2026-02-21 20:14:17 -05:00
yindo commented 2026-02-21 20:14:17 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 19, 2026):

The delayed (non-streaming) response from the GPT-5 Agent Node is likely due to one or more of these factors:

  1. Model plugin features: For agent node streaming to work, the model/plugin must declare the stream-tool-call feature in its capabilities. If your GPT-5 plugin YAML/config does not include this, Dify will not stream tool calls—even if the model API supports streaming. Compare with DeepSeek, which likely advertises this feature, enabling streaming in your setup. You can check your plugin config and add:

    - type: radio
  variable: function_calling_type
  default: function_call
  options: [no_call, function_call, tool_call]
- type: radio
  variable: stream_function_calling
  default: supported
  options: [supported, not_supported]

    This signals to Dify that the model supports streaming tool calls. Without it, agent node streaming is disabled source.

  2. Agent strategy mode: Streaming is only supported in Function Calling mode, not ReAct mode. If your agent is set to ReAct, it will always wait for the round to finish before outputting any text source.

  3. Workflow design: If your workflow graph requires merging or aggregation after the agent node (e.g., conditional branches, variable aggregators), streaming is lost because the engine waits for all branches to finish before emitting results source.

  4. Recent code changes: A recent update (PR #30832, Jan 2026) changed Dify to defer streaming until referenced variables are updated. If your workflow or agent node updates conversation variables, streaming will be delayed until those updates complete source.

  5. Reverse proxy buffering: If you use nginx or a similar reverse proxy, make sure buffering is disabled and HTTP/2 is not interfering, as these can block streaming responses source.

What to do next:

  • Check your GPT-5 plugin YAML for the stream-tool-call feature as shown above.
  • Make sure your agent is using Function Calling mode, not ReAct.
  • Review your workflow for any merges/aggregations after the agent node.
  • If you recently updated Dify, consider whether variable assignment timing is affecting streaming.
  • If you want, share your GPT-5 plugin YAML and workflow config for a more targeted review.

Let me know if you want help checking your config or troubleshooting further!

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Chat with Dosu Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Jan 19, 2026): <!-- Answer --> The delayed (non-streaming) response from the GPT-5 Agent Node is likely due to one or more of these factors: 1. **Model plugin features:** For agent node streaming to work, the model/plugin must declare the `stream-tool-call` feature in its capabilities. If your GPT-5 plugin YAML/config does not include this, Dify will not stream tool calls—even if the model API supports streaming. Compare with DeepSeek, which likely advertises this feature, enabling streaming in your setup. You can check your plugin config and add: ```yaml - type: radio variable: function_calling_type default: function_call options: [no_call, function_call, tool_call] - type: radio variable: stream_function_calling default: supported options: [supported, not_supported] ``` This signals to Dify that the model supports streaming tool calls. Without it, agent node streaming is disabled [`source`](https://github.com/langgenius/dify/issues/27028#issuecomment-3413692901). 2. **Agent strategy mode:** Streaming is only supported in Function Calling mode, not ReAct mode. If your agent is set to ReAct, it will always wait for the round to finish before outputting any text [`source`](https://github.com/langgenius/dify/issues/20952). 3. **Workflow design:** If your workflow graph requires merging or aggregation after the agent node (e.g., conditional branches, variable aggregators), streaming is lost because the engine waits for all branches to finish before emitting results [`source`](https://github.com/langgenius/dify/issues/15351). 4. **Recent code changes:** A recent update (PR #30832, Jan 2026) changed Dify to defer streaming until referenced variables are updated. If your workflow or agent node updates conversation variables, streaming will be delayed until those updates complete [`source`](https://github.com/langgenius/dify/pull/30832). 5. **Reverse proxy buffering:** If you use nginx or a similar reverse proxy, make sure buffering is disabled and HTTP/2 is not interfering, as these can block streaming responses [`source`](https://github.com/langgenius/dify/issues/28258). **What to do next:** - Check your GPT-5 plugin YAML for the `stream-tool-call` feature as shown above. - Make sure your agent is using Function Calling mode, not ReAct. - Review your workflow for any merges/aggregations after the agent node. - If you recently updated Dify, consider whether variable assignment timing is affecting streaming. - If you want, share your GPT-5 plugin YAML and workflow config for a more targeted review. Let me know if you want help checking your config or troubleshooting further! <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/d93745d2-1a25-47b8-9fed-8262e85df033?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/d93745d2-1a25-47b8-9fed-8262e85df033?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/d93745d2-1a25-47b8-9fed-8262e85df033?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/d93745d2-1a25-47b8-9fed-8262e85df033?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/d93745d2-1a25-47b8-9fed-8262e85df033?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/d93745d2-1a25-47b8-9fed-8262e85df033?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/d93745d2-1a25-47b8-9fed-8262e85df033?feedback_type=other)</sup> [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=The%20delayed%20%28non-streaming%29%20response%20from%20the%20GPT-5%20Agent%20Node%20is%20likely%20due%20to%20one%20or%20more%20of%20these%20factors%3A%0A%0A1.%20%2A%2AModel%20plugin%20features%3A%2A%2A%20For%20agent%20node%20streaming%20to%20work%2C%20the%20model/plugin%20must%20declare%20the%20%60stream-tool-call%60%20feature%20in%20its%20capabilities.%20If%20your%20GPT-5%20plugin%20YAML/config%20does%20not%20include%20this%2C%20Dify%20will%20not%20stream%20tool%20calls%E2%80%94even%20if%20the%20model%20API%20supports%20streaming.%20Compare%20with%20DeepSeek%2C%20which%20likely%20advertises%20this%20feature%2C%20enabling%20streaming%20in%20your%20setup.%20You%20can%20check%20your%20plugin%20config%20and%20add%3A%0A%20%20%20%60%60%60yaml%0A%20%20%20-%20type%3A%20radio%0A%20%20%20%20%20variable%3A%20function_calling_type%0A%20%20%20%20%20default%3A%20function_call%0A%20%20%20%20%20options%3A%20%5Bno_call%2C%20function_call%2C%20tool_call%5D%0A%20%20%20-%20type%3A%20radio%0A%20%20%20%20%20variable%3A%20stream_function_calling%0A%20%20%20%20%20default%3A%20supported%0A%20%20%20%20%20options%3A%20%5Bsupported%2C%20not_supported%5D%0A%20%20%20%60%60%60%0A%20%20%20This%20signals%20to%20Dify%20that%20the%20model%20supports%20streaming%20tool%20calls.%20Without%20it%2C%20agent%20node%20streaming%20is%20disabled%20%5B%60source%60%5D%28https%3A//github.com/langgenius/dify/issues/27028%23issuecomment-3413692901%29.%0A%0A2.%20%2A%2AAgent%20strategy%20mode%3A%2A%2A%20Streaming%20is%20only%20supported%20in%20Function%20Calling%20mode%2C%20not%20ReAct%20mode.%20If%20your%20agent%20is%20set%20to%20ReAct%2C%20it%20will%20always%20wait%20for%20the%20round%20to%20finish%20before%20outputting%20any%20text%20%5B%60source%60%5D%28https%3A//github.com/langgenius/dify/issues/20952%29.%0A%0A3.%20%2A%2AWorkflow%20design%3A%2A%2A%20If%20your%20workflow%20graph%20requires%20merging%20or%20aggregation%20after%20the%20agent%20node%20%28e.g.%2C%20conditional%20branches%2C%20variable%20aggregators%29%2C%20streaming%20is%20lost%20because%20the%20engine%20waits%20for%20all%20branches%20to%20finish%20before%20emitting%20results%20%5B%60source%60%5D%28https%3A//github.com/langgenius/dify/issues/15351%29.%0A%0A4.%20%2A%2ARecent%20code%20changes%3A%2A%2A%20A%20recent%20update%20%28PR%20%2330832%2C%20Jan%202026%29%20changed%20Dify%20to%20defer%20streaming%20until%20referenced%20variables%20are%20updated.%20If%20your%20workflow%20or%20agent%20node%20updates%20conversation%20variables%2C%20streaming%20will%20be%20delayed%20until%20those%20updates%20complete%20%5B%60source%60%5D%28https%3A//github.com/langgenius/dify/pull/30832%29.%0A%0A5.%20%2A%2AReverse%20proxy%20buffering%3A%2A%2A%20If%20you%20use%20nginx%20or%20a%20similar%20reverse%20proxy%2C%20make%20sure%20buffering%20is%20disabled%20and%20HTTP/2%20is%20not%20interfering%2C%20as%20these%20can%20block%20streaming%20responses%20%5B%60source%60%5D%28https%3A//github.com/langgenius/dify/issues/28258%29.%0A%0A%2A%2AWhat%20to%20do%20next%3A%2A%2A%20%20%0A-%20Check%20your%20GPT-5%20plugin%20YAML%20for%20the%20%60stream-tool-call%60%20feature%20as%20shown%20above.%0A-%20Make%20sure%20your%20agent%20is%20using%20Function%20Calling%20mode%2C%20not%20ReAct.%0A-%20Review%20your%20workflow%20for%20any%20merges/aggregations%20after%20the%20agent%20node.%0A-%20If%20you%20recently%20updated%20Dify%2C%20consider%20whether%20variable%20assignment%20timing%20is%20affecting%20streaming.%0A-%20If%20you%20want%2C%20share%20your%20GPT-5%20plugin%20YAML%20and%20workflow%20config%20for%20a%20more%20targeted%20review.%0A%0ALet%20me%20know%20if%20you%20want%20help%20checking%20your%20config%20or%20troubleshooting%20further%21)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/31259)
yindo commented 2026-02-21 20:14:17 -05:00
Author
Owner
Copy Link
Copy Source

@fatelei commented on GitHub (Jan 20, 2026):

you can use opentelemetry to dig why it slow

enable config in .env

ENABLE_OTEL=true
OTEL_SAMPLING_RATE=1.0

docker network create otel || true
docker run -d --name jaeger --network otel \
  -p 16686:16686 -p 4317:4317 -p 4318:4318 \
  jaegertracing/all-in-one:latest
@fatelei commented on GitHub (Jan 20, 2026): you can use opentelemetry to dig why it slow enable config in .env ``` ENABLE_OTEL=true OTEL_SAMPLING_RATE=1.0 ``` ``` docker network create otel || true docker run -d --name jaeger --network otel \ -p 16686:16686 -p 4317:4317 -p 4318:4318 \ jaegertracing/all-in-one:latest ```
yindo commented 2026-02-21 20:14:17 -05:00
Author
Owner
Copy Link
Copy Source

@zio525 commented on GitHub (Jan 20, 2026):

you can use opentelemetry to dig why it slow

enable config in .env

ENABLE_OTEL=true
OTEL_SAMPLING_RATE=1.0

docker network create otel || true
docker run -d --name jaeger --network otel \
  -p 16686:16686 -p 4317:4317 -p 4318:4318 \
  jaegertracing/all-in-one:latest

I set up these env variable in .env file. And I installed the jaeger in dokcer. Then I restart the dify. Now the jaeger can receive trace request, but there is no trace about dify. it seems that dify don't send trace request. The jaeger and dify are in a same network now, and I also set OTLP_BASE_ENDPOINT=http://172.19.0.14:4318 in .env file.

@zio525 commented on GitHub (Jan 20, 2026): > you can use opentelemetry to dig why it slow > > enable config in .env > > ``` > ENABLE_OTEL=true > OTEL_SAMPLING_RATE=1.0 > ``` > > ``` > docker network create otel || true > docker run -d --name jaeger --network otel \ > -p 16686:16686 -p 4317:4317 -p 4318:4318 \ > jaegertracing/all-in-one:latest > ``` I set up these env variable in .env file. And I installed the jaeger in dokcer. Then I restart the dify. Now the jaeger can receive trace request, but there is no trace about dify. it seems that dify don't send trace request. The jaeger and dify are in a same network now, and I also set OTLP_BASE_ENDPOINT=http://172.19.0.14:4318 in .env file.
yindo commented 2026-02-21 20:14:17 -05:00
Author
Owner
Copy Link
Copy Source

@zio525 commented on GitHub (Jan 20, 2026):

I found that it cannot be streamed even in a simple workflow with one LLM Node.

Image

Image

@zio525 commented on GitHub (Jan 20, 2026): I found that it cannot be streamed even in a simple workflow with one LLM Node. <img width="2559" height="1399" alt="Image" src="https://github.com/user-attachments/assets/67776a0b-f6b5-47a8-8a35-d76056735d67" /> ![Image](https://github.com/user-attachments/assets/f83bd223-d995-4d20-898e-f21e899162f8)
yindo commented 2026-02-21 20:14:17 -05:00
Author
Owner
Copy Link
Copy Source

@zio525 commented on GitHub (Jan 20, 2026):

the apikey that im using is verified for organization, and i test it can reply streaming when i directly request the api.

@zio525 commented on GitHub (Jan 20, 2026): the apikey that im using is verified for organization, and i test it can reply streaming when i directly request the api.
yindo commented 2026-02-21 20:14:17 -05:00
Author
Owner
Copy Link
Copy Source

@fatelei commented on GitHub (Jan 20, 2026):

use the opentelemetry, you will find why it slow

@fatelei commented on GitHub (Jan 20, 2026): use the opentelemetry, you will find why it slow
yindo commented 2026-02-21 20:14:17 -05:00
Author
Owner
Copy Link
Copy Source

@zio525 commented on GitHub (Jan 27, 2026):

use the opentelemetry, you will find why it slow

I tested in a simple chatflow with a start node , a agent node, and a reply node. it streams reply when I use Deep Seek Model in Agent Node, but it fails when I use GPT-5.

@zio525 commented on GitHub (Jan 27, 2026): > use the opentelemetry, you will find why it slow I tested in a simple chatflow with a start node , a agent node, and a reply node. it streams reply when I use Deep Seek Model in Agent Node, but it fails when I use GPT-5.
yindo commented 2026-02-21 20:14:17 -05:00
Author
Owner
Copy Link
Copy Source

@zio525 commented on GitHub (Jan 28, 2026):

I figure out why the Agent node seems that cannot output streaming when I use GPT model.

  1. Open AI doesn't open the thought content of its model, which means there is only the text output part that can be replied as "messaage" in every round.
  2. Agent nodes don't always output the text in every round. It will "forget" to output the text in some round, especially when it got lot of tools and a long prompt.

The above reasons can lead to a scene that the first "message" event may be later several round or more than the corresponding tool calling round.

@zio525 commented on GitHub (Jan 28, 2026): I figure out why the Agent node seems that cannot output streaming when I use GPT model. 1. Open AI doesn't open the thought content of its model, which means there is only the text output part that can be replied as "messaage" in every round. 2. Agent nodes don't always output the text in every round. It will "forget" to output the text in some round, especially when it got lot of tools and a long prompt. The above reasons can lead to a scene that the first "message" event may be later several round or more than the corresponding tool calling round.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug 🤖 feat:agent
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#21783
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?