langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-08-26 18:36:52 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

In the API streaming request execution mode, an unexpected disconnection of the client can lead to the interruption of the current workflow execution, which will continuously display a "running" status #16009

New Issue
Closed
opened 2026-02-21 19:24:18 -05:00 by yindo · 8 comments
yindo commented 2026-02-21 19:24:18 -05:00
Owner
Copy Link
Copy Source

Originally created by @LZW-Andrewlu on GitHub (Aug 8, 2025).

Originally assigned to: @laipz8200 on GitHub.

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.7.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

BugReport: API request for run workflow will fall down when client closed unexcepted.

In the API streaming request execution mode, an unexpected disconnection of the client can lead to the interruption of the current workflow execution, which will continuously display a "running" status.

My application scenario:

I am currently designing a backend service application that executes some automated processes through API-calling workflows. When the application is making some API streaming requests, I restart the application (in a hot update/deployment scenario). After the restart, I call the workflow execution progress query interface to check the task execution results. I found that the interrupted tasks remain in the running state and do not continue to run

✔️ Expected Behavior

Expected result:

In the interface invocation mode, the execution of the workflow should not be unexpectedly halted due to a disconnection from the client. It should be able to continue executing and return results through the progress query interface

Temporary solution:

To address the aforementioned issues, I devised a temporary solution by designing an interface relay proxy service. This service forwards API requests to the Dify service via a pass-through proxy, and correctly determines whether the client has closed the connection in the response, thereby deciding whether to forward the response. As long as the proxy service does not terminate abnormally, it will not cause the requested interface to be interrupted unexpectedly

Example code:


    const response = await axiosInstance(config);

    // 设置响应头
    Object.keys(response.headers).forEach(key => {
      res.setHeader(key, response.headers[key]);
    });

    res.status(response.status);

    if (isStream) {
      // 处理流式响应
      logger.info(`Streaming response from: ${url},${response.status}`);
      response.data.on('data', (chunk) => {
        try {
          !res.closed && res.write(chunk);
          if (res.closed) {
            logger.info(`Response stream closed for request`);
          }
        } catch (error) {
          logger.error(`Error writing chunk to response: ${error.message}`);
        }
      });
      response.data.on('end', () => {
        try {
          !res.closed && res.end();
        } catch (error) {
          logger.error(`Error ending response stream: ${error.message}`);
        }
      });
    } else {
      // 处理普通响应
      res.json(response.data);
      logger.info(`response to: ${url}, body: ${JSON.stringify(response.data)}`);
    }

❌ Actual Behavior

when i restart my app, the request with "/workflows/run" will fall down ,and will not complete the task with correct response. it will continuely response "running" status for me with progress query api.

Originally created by @LZW-Andrewlu on GitHub (Aug 8, 2025). Originally assigned to: @laipz8200 on GitHub. ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.7.1 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce # BugReport: API request for run workflow will fall down when client closed unexcepted. In the API streaming request execution mode, an unexpected disconnection of the client can lead to the interruption of the current workflow execution, which will continuously display a "running" status. # My application scenario: I am currently designing a backend service application that executes some automated processes through API-calling workflows. When the application is making some API streaming requests, I restart the application (in a hot update/deployment scenario). After the restart, I call the workflow execution progress query interface to check the task execution results. I found that the interrupted tasks remain in the running state and do not continue to run ### ✔️ Expected Behavior # Expected result: In the interface invocation mode, the execution of the workflow should not be unexpectedly halted due to a disconnection from the client. It should be able to continue executing and return results through the progress query interface # Temporary solution: To address the aforementioned issues, I devised a temporary solution by designing an interface relay proxy service. This service forwards API requests to the Dify service via a pass-through proxy, and correctly determines whether the client has closed the connection in the response, thereby deciding whether to forward the response. As long as the proxy service does not terminate abnormally, it will not cause the requested interface to be interrupted unexpectedly # Example code: ```js const response = await axiosInstance(config); // 设置响应头 Object.keys(response.headers).forEach(key => { res.setHeader(key, response.headers[key]); }); res.status(response.status); if (isStream) { // 处理流式响应 logger.info(`Streaming response from: ${url},${response.status}`); response.data.on('data', (chunk) => { try { !res.closed && res.write(chunk); if (res.closed) { logger.info(`Response stream closed for request`); } } catch (error) { logger.error(`Error writing chunk to response: ${error.message}`); } }); response.data.on('end', () => { try { !res.closed && res.end(); } catch (error) { logger.error(`Error ending response stream: ${error.message}`); } }); } else { // 处理普通响应 res.json(response.data); logger.info(`response to: ${url}, body: ${JSON.stringify(response.data)}`); } ``` ### ❌ Actual Behavior when i restart my app, the request with "/workflows/run" will fall down ,and will not complete the task with correct response. it will continuely response "running" status for me with progress query api.
yindo added the 🐞 bugstale labels 2026-02-21 19:24:18 -05:00
yindo closed this issue 2026-02-21 19:24:19 -05:00
yindo commented 2026-02-21 19:24:19 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 8, 2025):

This is a known architectural limitation in Dify (including 1.7.1): if a client disconnects unexpectedly during a streaming workflow execution (like /workflows/run), the workflow execution thread may be killed, leaving the workflow stuck in a "running" state in the database—even if the backend execution completes. This happens because workflow status updates and logging are performed in the main thread responsible for streaming the response; if that thread is terminated by a disconnect, those updates never occur, so the workflow never transitions to "succeeded" or "failed" [source].

There is no built-in mechanism to detect client disconnects and mark the workflow as failed or completed. Only explicit API calls (like the stop endpoint) or successful completion events trigger status transitions. This issue has been reported in multiple GitHub issues and affects both cloud and self-hosted deployments, especially in streaming mode [source].

Your proxy workaround is effective for guarding against client disconnects, but it won't help if the server process itself crashes or is restarted. The recommended architectural solution is to decouple database operations (status updates) from the main thread handling the streaming response—such as by moving DB updates to a separate thread or using a queue/worker system (like Celery)—to ensure workflow status updates continue regardless of client connection state [source].

There are config options for workflow execution timeouts (e.g., WORKFLOW_MAX_EXECUTION_TIME), but these do not address the issue of stuck "running" workflows due to disconnects. No recent pull requests have addressed this problem, so it remains unresolved in the latest versions.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

@dosubot[bot] commented on GitHub (Aug 8, 2025): <!-- Answer --> This is a known architectural limitation in Dify (including 1.7.1): if a client disconnects unexpectedly during a streaming workflow execution (like `/workflows/run`), the workflow execution thread may be killed, leaving the workflow stuck in a "running" state in the database—even if the backend execution completes. This happens because workflow status updates and logging are performed in the main thread responsible for streaming the response; if that thread is terminated by a disconnect, those updates never occur, so the workflow never transitions to "succeeded" or "failed" [[source](https://github.com/langgenius/dify/issues/12798)]. There is no built-in mechanism to detect client disconnects and mark the workflow as failed or completed. Only explicit API calls (like the stop endpoint) or successful completion events trigger status transitions. This issue has been reported in multiple GitHub issues and affects both cloud and self-hosted deployments, especially in streaming mode [[source](https://github.com/langgenius/dify/issues/14362)]. Your proxy workaround is effective for guarding against client disconnects, but it won't help if the server process itself crashes or is restarted. The recommended architectural solution is to decouple database operations (status updates) from the main thread handling the streaming response—such as by moving DB updates to a separate thread or using a queue/worker system (like Celery)—to ensure workflow status updates continue regardless of client connection state [[source](https://github.com/langgenius/dify/issues/14362)]. There are config options for workflow execution timeouts (e.g., `WORKFLOW_MAX_EXECUTION_TIME`), but these do not address the issue of stuck "running" workflows due to disconnects. No recent pull requests have addressed this problem, so it remains unresolved in the latest versions. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/871b6a88-7c14-4d40-af62-ce473d4d8c35?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/871b6a88-7c14-4d40-af62-ce473d4d8c35?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/871b6a88-7c14-4d40-af62-ce473d4d8c35?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/871b6a88-7c14-4d40-af62-ce473d4d8c35?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/871b6a88-7c14-4d40-af62-ce473d4d8c35?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/871b6a88-7c14-4d40-af62-ce473d4d8c35?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/871b6a88-7c14-4d40-af62-ce473d4d8c35?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/23654)
yindo commented 2026-02-21 19:24:19 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Aug 9, 2025):

Similar to

@crazywoola commented on GitHub (Aug 9, 2025): Similar to - https://github.com/langgenius/dify/issues/23291 - https://github.com/langgenius/dify/issues/17300 - https://github.com/langgenius/dify/issues/15500 - https://github.com/langgenius/dify/issues/23179 - https://github.com/langgenius/dify/issues/23174 - https://github.com/langgenius/dify/issues/21770
yindo commented 2026-02-21 19:24:19 -05:00
Author
Owner
Copy Link
Copy Source

@LZW-Andrewlu commented on GitHub (Aug 11, 2025):

I expect that in API mode, only asynchronous task submission is performed without waiting for the task to complete execution. Instead, the workflow_run_id is used to actively query the task status afterwards. However, currently, there is no such mode that can be adapted @dosu

@LZW-Andrewlu commented on GitHub (Aug 11, 2025): I expect that in API mode, only asynchronous task submission is performed without waiting for the task to complete execution. Instead, the workflow_run_id is used to actively query the task status afterwards. However, currently, there is no such mode that can be adapted @dosu
yindo commented 2026-02-21 19:24:19 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Aug 15, 2025):

@LZW-Andrewlu We are trying to achieve this by moving the logic into message queues.

@crazywoola commented on GitHub (Aug 15, 2025): @LZW-Andrewlu We are trying to achieve this by moving the logic into message queues.
yindo commented 2026-02-21 19:24:19 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Sep 15, 2025):

Hi, @LZW-Andrewlu. I'm Dosu, and I'm helping the Dify team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You reported that in API streaming mode, unexpected client disconnects cause workflow executions to be interrupted but stuck in a "running" state.
  • You implemented a proxy workaround to detect disconnects and prevent interruptions.
  • The root cause is a known architectural limitation where status updates rely on the streaming thread, which is terminated on disconnect.
  • A recommended fix involves decoupling database updates from the streaming thread using queues or workers.
  • Community efforts are ongoing to move logic into message queues, but no built-in async task submission mode exists yet.

Next Steps:

  • Please let me know if this issue is still relevant with the latest version of Dify by commenting here to keep the discussion open.
  • Otherwise, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Sep 15, 2025): Hi, @LZW-Andrewlu. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog and am marking this issue as stale. **Issue Summary:** - You reported that in API streaming mode, unexpected client disconnects cause workflow executions to be interrupted but stuck in a "running" state. - You implemented a proxy workaround to detect disconnects and prevent interruptions. - The root cause is a known architectural limitation where status updates rely on the streaming thread, which is terminated on disconnect. - A recommended fix involves decoupling database updates from the streaming thread using queues or workers. - Community efforts are ongoing to move logic into message queues, but no built-in async task submission mode exists yet. **Next Steps:** - Please let me know if this issue is still relevant with the latest version of Dify by commenting here to keep the discussion open. - Otherwise, this issue will be automatically closed in 15 days. Thank you for your understanding and contribution!
yindo commented 2026-02-21 19:24:19 -05:00
Author
Owner
Copy Link
Copy Source

@miiiz commented on GitHub (Sep 25, 2025):

Version: v1.8.1

This is what i had several times recently.
When running a Chatflow app, all nodes in the flow complete successfully, but:
• The run detail page in the console still shows the workflow as Running.
• Chat and conversation history does not show any the final answer.
• Essentially, the chatflow execution never transitions into a “finished” state even though no node is pending.

Image Image
@miiiz commented on GitHub (Sep 25, 2025): Version: v1.8.1 This is what i had several times recently. When running a Chatflow app, all nodes in the flow complete successfully, but: • The run detail page in the console still shows the workflow as Running. • Chat and conversation history does not show any the final answer. • Essentially, the chatflow execution never transitions into a “finished” state even though no node is pending. <img width="953" height="431" alt="Image" src="https://github.com/user-attachments/assets/579604e3-6180-4fcb-8ab1-090ef7798a22" /> <img width="944" height="478" alt="Image" src="https://github.com/user-attachments/assets/048e78f8-b6c2-4141-9fb7-3b00590f7329" />
yindo commented 2026-02-21 19:24:19 -05:00
Author
Owner
Copy Link
Copy Source

@laipz8200 commented on GitHub (Oct 22, 2025):

Hi @miiiz, thanks a lot for your feedback! Could you please share more details about how to reliably reproduce the issue where the flow gets stuck in the 'Running' status (for example: environment, step-by-step actions, or under what conditions it happens)? We'd like to clarify the scenario so we can better investigate—thanks again!

@laipz8200 commented on GitHub (Oct 22, 2025): Hi @miiiz, thanks a lot for your feedback! Could you please share more details about how to reliably reproduce the issue where the flow gets stuck in the 'Running' status (for example: environment, step-by-step actions, or under what conditions it happens)? We'd like to clarify the scenario so we can better investigate—thanks again!
yindo commented 2026-02-21 19:24:19 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 21, 2026):

Hi, @LZW-Andrewlu. I'm Dosu, and I'm helping the Dify team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You reported that in API streaming mode, unexpected client disconnects cause workflow executions to get stuck in a "running" state.
  • This happens because status updates rely on the streaming thread, which is terminated on disconnect.
  • A suggested fix involves decoupling database updates using queues or worker processes.
  • Community members have noted ongoing efforts to address this via message queues, and similar issues persist in version 1.8.1.
  • The maintainers have asked for more reproduction details but have not received updates recently.

Next Steps:

  • Please let me know if this issue is still relevant with the latest version of Dify by commenting here to keep the discussion open.
  • Otherwise, I will automatically close this issue in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Jan 21, 2026): Hi, @LZW-Andrewlu. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog and am marking this issue as stale. **Issue Summary:** - You reported that in API streaming mode, unexpected client disconnects cause workflow executions to get stuck in a "running" state. - This happens because status updates rely on the streaming thread, which is terminated on disconnect. - A suggested fix involves decoupling database updates using queues or worker processes. - Community members have noted ongoing efforts to address this via message queues, and similar issues persist in version 1.8.1. - The maintainers have asked for more reproduction details but have not received updates recently. **Next Steps:** - Please let me know if this issue is still relevant with the latest version of Dify by commenting here to keep the discussion open. - Otherwise, I will automatically close this issue in 15 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label stale 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#16009
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?