langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 04:25:29 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

LLM node not streaming #12321

New Issue
Closed
opened 2026-02-21 19:06:54 -05:00 by yindo · 10 comments
yindo commented 2026-02-21 19:06:54 -05:00
Owner
Copy Link
Copy Source

Originally created by @DavideDelbianco on GitHub (Mar 28, 2025).

Originally assigned to: @laipz8200, @Nov1c444 on GitHub.

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.1.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I've seen the issue #13626 where the bug is supposedly fixed
I've seen the issue #15700 where the solution seems to be to remove the IF/ELSE node before the Question classifier node
I've seen the issue #15700 where the bug seems to be relate to crossing Question Classifier nodes
I've seen the issue #16882 where the solution is to place the streaming LLM+Answer nodes inside a Loop Node

In the video you can see me:

  1. Use the streaming node inside the Loop node (it works)
  2. Use the same flow with the streaming node outside the Loop node (it does not work)
  3. Remove the branch coming from a non executed branch of the first question classifier (it works even if there's an IF/ELSE node before the second Classifier node)
  4. Execute a simple flow that seems to work in a freshly created chatflow
  5. Tear down my flow (upgraded from previous versions) to make it exactly like the working example (it does not work)
  6. Demonstrate that adding an IF/ELSE node before the Classifer nodes that merge into the LLM node breaks the streaming output.

It seems to me that there are multiple issue related to the Question Classifier node + IF/ELSE node + merging branches before a LLM node, but only if one of the branches remains not executed, and possibly even something broken in old DSL because of point 4 and 5 yelding different results

https://drive.google.com/file/d/1uwQ142aKNAQKbwCex2dDgFQ8-G_hmNUS/view?usp=drive_link

✔️ Expected Behavior

LLM nodes should stream the output independendantly of what happened in previous branches with Classier nodes

❌ Actual Behavior

LLM node does not stream the output in more than one condition when a Classifier node was previously used

Originally created by @DavideDelbianco on GitHub (Mar 28, 2025). Originally assigned to: @laipz8200, @Nov1c444 on GitHub. ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.1.3 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce I've seen the issue #13626 where the bug is supposedly fixed I've seen the issue #15700 where the solution seems to be to remove the IF/ELSE node before the Question classifier node I've seen the issue #15700 where the bug seems to be relate to crossing Question Classifier nodes I've seen the issue #16882 where the solution is to place the streaming LLM+Answer nodes inside a Loop Node In the video you can see me: 1. Use the streaming node inside the Loop node (it works) 2. Use the same flow with the streaming node outside the Loop node (it does not work) 3. Remove the branch coming from a non executed branch of the first question classifier (it works even if there's an IF/ELSE node before the second Classifier node) 4. Execute a simple flow that seems to work in a freshly created chatflow 5. Tear down my flow (upgraded from previous versions) to make it exactly like the working example (it does not work) 6. Demonstrate that adding an IF/ELSE node before the Classifer nodes that merge into the LLM node breaks the streaming output. It seems to me that there are multiple issue related to the Question Classifier node + IF/ELSE node + merging branches before a LLM node, but only if one of the branches remains not executed, and possibly even something broken in old DSL because of point 4 and 5 yelding different results https://drive.google.com/file/d/1uwQ142aKNAQKbwCex2dDgFQ8-G_hmNUS/view?usp=drive_link ### ✔️ Expected Behavior LLM nodes should stream the output independendantly of what happened in previous branches with Classier nodes ### ❌ Actual Behavior LLM node does not stream the output in more than one condition when a Classifier node was previously used
yindo added the 🐞 bug label 2026-02-21 19:06:54 -05:00
yindo closed this issue 2026-02-21 19:06:54 -05:00
yindo commented 2026-02-21 19:06:54 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Mar 28, 2025):

There are a few relevant discussions and issues related to the problem you're experiencing with the LLM node not streaming output when used with Classifier and IF/ELSE nodes:

  1. Discussion on Slow Response with Streaming: There is an ongoing discussion about slow response times when using the Dify chatflow's API with streaming enabled [1].

  2. Issue with Streaming Output Failing: A closed issue addressed a problem where streaming output failed when LLM nodes enabled error handling. This was resolved by a pull request that fixed the stream output error [2].

  3. Classifier Node and IF/ELSE Node Issues: There is a known limitation in version v0.15.3 where the parallel chatflow gets stuck at the Question Classifier node when there is a connection between the Variable Assigner and If/Else node. This is expected to be fixed in the upcoming v1.0.0 release [3].

  4. Nested IF/ELSE Workflow Issue: The LLM node not streaming output in certain conditions, especially with nested IF/ELSE nodes, was due to an issue in the _fetch_node_ids_in_reachable_branch method. This has been addressed in a recent update through PR #12105 in Dify version 0.14.2 [4].

It seems like your issue might be related to these known problems, particularly with the handling of branches and the execution order in workflows involving Classifier and IF/ELSE nodes. You might want to check if your setup aligns with the solutions provided in these discussions and issues. If the problem persists, consider whether any recent updates or patches might address your specific scenario.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Mar 28, 2025): <!-- Answer --> There are a few relevant discussions and issues related to the problem you're experiencing with the LLM node not streaming output when used with Classifier and IF/ELSE nodes: 1. **Discussion on Slow Response with Streaming**: There is an ongoing discussion about slow response times when using the Dify chatflow's API with streaming enabled <sup>[[1]](https://github.com/langgenius/dify/discussions/16602)</sup>. 2. **Issue with Streaming Output Failing**: A closed issue addressed a problem where streaming output failed when LLM nodes enabled error handling. This was resolved by a pull request that fixed the stream output error <sup>[[2]](https://github.com/langgenius/dify/issues/15626)</sup>. 3. **Classifier Node and IF/ELSE Node Issues**: There is a known limitation in version v0.15.3 where the parallel chatflow gets stuck at the Question Classifier node when there is a connection between the Variable Assigner and If/Else node. This is expected to be fixed in the upcoming v1.0.0 release <sup>[[3]](https://github.com/langgenius/dify/issues/13628)</sup>. 4. **Nested IF/ELSE Workflow Issue**: The LLM node not streaming output in certain conditions, especially with nested IF/ELSE nodes, was due to an issue in the `_fetch_node_ids_in_reachable_branch` method. This has been addressed in a recent update through PR #12105 in Dify version 0.14.2 <sup>[[4]](https://github.com/langgenius/dify/issues/15162)</sup>. It seems like your issue might be related to these known problems, particularly with the handling of branches and the execution order in workflows involving Classifier and IF/ELSE nodes. You might want to check if your setup aligns with the solutions provided in these discussions and issues. If the problem persists, consider whether any recent updates or patches might address your specific scenario. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/ed86aae1-4ade-4607-81e6-fedbb249bb08?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/ed86aae1-4ade-4607-81e6-fedbb249bb08?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/ed86aae1-4ade-4607-81e6-fedbb249bb08?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/ed86aae1-4ade-4607-81e6-fedbb249bb08?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/ed86aae1-4ade-4607-81e6-fedbb249bb08?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/ed86aae1-4ade-4607-81e6-fedbb249bb08?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/ed86aae1-4ade-4607-81e6-fedbb249bb08?feedback_type=other)</sup>
yindo commented 2026-02-21 19:06:54 -05:00
Author
Owner
Copy Link
Copy Source

@ft4710403 commented on GitHub (Mar 29, 2025):

我找到一个解决方法，把流输出有问题的节点，套在一个循环组件里面，只循环1次，在里面放入llm和回复组件，即可恢复正常的流输出~~~

Image

@ft4710403 commented on GitHub (Mar 29, 2025): 我找到一个解决方法，把流输出有问题的节点，套在一个循环组件里面，只循环1次，在里面放入llm和回复组件，即可恢复正常的流输出~~~ ![Image](https://github.com/user-attachments/assets/0d835327-8dfa-4702-9593-1f18cbed6d83)
yindo commented 2026-02-21 19:06:55 -05:00
Author
Owner
Copy Link
Copy Source

@DavideDelbianco commented on GitHub (Mar 29, 2025):

@ft4710403 it's among the tests that I've done, but the issue needs to be fixed 😊

@DavideDelbianco commented on GitHub (Mar 29, 2025): @ft4710403 it's among the tests that I've done, but the issue needs to be fixed 😊
yindo commented 2026-02-21 19:06:55 -05:00
Author
Owner
Copy Link
Copy Source

@saurabhy27 commented on GitHub (Apr 24, 2025):

@DavideDelbianco I’ve tested using the streaming node inside a Loop node — it works only if there are no failed nodes attached to the streaming node.
If there’s a failed node connected to the streaming node, streaming does not work as expected.
Removing the failed node restores normal streaming behavior.

@saurabhy27 commented on GitHub (Apr 24, 2025): @DavideDelbianco I’ve tested using the streaming node inside a Loop node — it works only if there are no failed nodes attached to the streaming node. If there’s a failed node connected to the streaming node, streaming does not work as expected. Removing the failed node restores normal streaming behavior.
yindo commented 2026-02-21 19:06:55 -05:00
Author
Owner
Copy Link
Copy Source

@Nov1c444 commented on GitHub (May 13, 2025):

I want to verify if the mentioned PR has fixed this issue, but unfortunately I cannot replay the video. Could you provide something for me to verify whether this PR has fixed the issue?

@Nov1c444 commented on GitHub (May 13, 2025): I want to verify if the mentioned PR has fixed this issue, but unfortunately I cannot replay the video. Could you provide something for me to verify whether this PR has fixed the issue?
yindo commented 2026-02-21 19:06:55 -05:00
Author
Owner
Copy Link
Copy Source

@DavideDelbianco commented on GitHub (May 13, 2025):

I have restored the video,let me know if the link does not work

@DavideDelbianco commented on GitHub (May 13, 2025): I have restored the video,let me know if the link does not work
yindo commented 2026-02-21 19:06:55 -05:00
Author
Owner
Copy Link
Copy Source

@Nov1c444 commented on GitHub (May 13, 2025):

It works, but the issue of streaming output when one node is connected to multiple condition nodes, which I mentioned above, has still not been resolved by the PR. I will do my best to solve this problem soon.

@Nov1c444 commented on GitHub (May 13, 2025): It works, but the issue of streaming output when one node is connected to multiple condition nodes, which I mentioned above, has still not been resolved by the PR. I will do my best to solve this problem soon.
yindo commented 2026-02-21 19:06:55 -05:00
Author
Owner
Copy Link
Copy Source

@DavideDelbianco commented on GitHub (Jun 26, 2025):

Is the PR a valid one?
Does this issue still exist?

@DavideDelbianco commented on GitHub (Jun 26, 2025): Is the PR a valid one? Does this issue still exist?
yindo commented 2026-02-21 19:06:55 -05:00
Author
Owner
Copy Link
Copy Source

@Nov1c444 commented on GitHub (Jun 27, 2025):

Fixed by https://github.com/langgenius/dify/pull/20880

Is the PR a valid one? Does this issue still exist?

@Nov1c444 commented on GitHub (Jun 27, 2025): Fixed by https://github.com/langgenius/dify/pull/20880 > Is the PR a valid one? Does this issue still exist?
yindo commented 2026-02-21 19:06:55 -05:00
Author
Owner
Copy Link
Copy Source

@LXXiaogege commented on GitHub (Dec 11, 2025):

我找到一个解决方法，把流输出有问题的节点，套在一个循环组件里面，只循环1次，在里面放入llm和回复组件，即可恢复正常的流输出~~~

Image

Good！！！

@LXXiaogege commented on GitHub (Dec 11, 2025): > 我找到一个解决方法，把流输出有问题的节点，套在一个循环组件里面，只循环1次，在里面放入llm和回复组件，即可恢复正常的流输出~~~ > > ![Image](https://github.com/user-attachments/assets/0d835327-8dfa-4702-9593-1f18cbed6d83) Good！！！
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#12321
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?