langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-08-27 21:10:04 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

chat-messages API is taking more time compared to test run in dify #5634

New Issue
Closed
opened 2026-02-21 18:11:58 -05:00 by yindo · 4 comments
yindo commented 2026-02-21 18:11:58 -05:00
Owner
Copy Link
Copy Source

Originally created by @AugNSo on GitHub (Sep 14, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.8.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Build a workflow that involves a LLM node.
Access the workflow via chat-message API.
[So far we observe this problem regardless of which LLM used or response mode]

✔️ Expected Behavior

The total response time is expected to be about the same to how much preview run takes in dify.

❌ Actual Behavior

The total response time can be up to two times of how much preview run takes.
This problem is very obvious when the LLM is given a long prompt such as performing RAG task.

Originally created by @AugNSo on GitHub (Sep 14, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version 0.8.0 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce Build a workflow that involves a LLM node. Access the workflow via chat-message API. [So far we observe this problem regardless of which LLM used or response mode] ### ✔️ Expected Behavior The total response time is expected to be about the same to how much preview run takes in dify. ### ❌ Actual Behavior The total response time can be up to two times of how much preview run takes. This problem is very obvious when the LLM is given a long prompt such as performing RAG task.
yindo added the 🐞 bug label 2026-02-21 18:11:58 -05:00
yindo closed this issue 2026-02-21 18:11:58 -05:00
yindo commented 2026-02-21 18:11:59 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Sep 14, 2024):

@AugNSo Hello there! I'm here to help you with bug reports, questions, and becoming a contributor. Let's work together to solve any issues you encounter while waiting for a human maintainer.

I found a similar unsolved discussion and some related issues that might help you understand and address the performance discrepancy you're experiencing with the chat-messages API.

  1. Similar Discussion:

  2. Related Issues:

The chat-messages API might be taking more time due to the create_chat_message method in the ChatClient class, which does not handle long prompts efficiently. The method sends a POST request to the /chat-messages endpoint, and if the response_mode is set to "blocking", it waits for the entire response to be processed before returning, which can be time-consuming for long prompts [4].

Additionally, there are known limitations and performance issues documented for the /chat-messages endpoint, such as the blocking mode potentially causing interruptions if the process takes too long due to Cloudflare restrictions [5][6][7].

Consider using the streaming response mode to improve performance and responsiveness.

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Sep 14, 2024): <!-- Greeting --> @AugNSo Hello there! I'm here to help you with bug reports, questions, and becoming a contributor. Let's work together to solve any issues you encounter while waiting for a human maintainer. <!-- Answer --> I found a similar unsolved discussion and some related issues that might help you understand and address the performance discrepancy you're experiencing with the `chat-messages` API. 1. **Similar Discussion**: - [Help: The performance of calling the API interface exposed through the workflow defined by dify is only half of the performance of directly calling the Chat interface](https://github.com/langgenius/dify/discussions/6936) <sup>[[1]](https://github.com/langgenius/dify/discussions/6936)</sup>. 2. **Related Issues**: - **Open Issue**: [Agent mode does not use streaming output, so the conversation experience seems slow](https://github.com/langgenius/dify/issues/2280) <sup>[[2]](https://github.com/langgenius/dify/issues/2280)</sup>. - **Suggested Solution**: Implement streaming output in the `invoke_llm` method to improve responsiveness. ```python llm_result: LLMResult = model_instance.invoke_llm( prompt_messages=prompt_messages, model_parameters=app_orchestration_config.model_config.parameters, tools=[], stop=app_orchestration_config.model_config.stop, stream=True, # Enable streaming user=self.user_id, callbacks=[], ) ``` - **Closed Issue**: [Dify assistant app stops generating when it takes long](https://github.com/langgenius/dify/issues/2278) <sup>[[3]](https://github.com/langgenius/dify/issues/2278)</sup>. - **Suggested Solution**: Increase the timeout value in the frontend code and consider implementing a reconnect feature to handle disconnections. The `chat-messages` API might be taking more time due to the `create_chat_message` method in the `ChatClient` class, which does not handle long prompts efficiently. The method sends a POST request to the `/chat-messages` endpoint, and if the `response_mode` is set to "blocking", it waits for the entire response to be processed before returning, which can be time-consuming for long prompts <sup>[[4]](https://github.com/langgenius/dify/blob/main/sdks/python-client/dify_client/client.py)</sup>. Additionally, there are known limitations and performance issues documented for the `/chat-messages` endpoint, such as the blocking mode potentially causing interruptions if the process takes too long due to Cloudflare restrictions <sup>[[5]](https://github.com/langgenius/dify/blob/main/web/app/components/develop/template/template_chat.zh.mdx)</sup><sup>[[6]](https://github.com/langgenius/dify/blob/main/web/app/components/develop/template/template_advanced_chat.zh.mdx)</sup><sup>[[7]](https://github.com/langgenius/dify/blob/main/web/app/components/develop/template/template_chat.en.mdx)</sup>. Consider using the streaming response mode to improve performance and responsiveness. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:11:59 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Sep 14, 2024):

Try set auto_generate_name to false.

@crazywoola commented on GitHub (Sep 14, 2024): Try set `auto_generate_name ` to false.
yindo commented 2026-02-21 18:11:59 -05:00
Author
Owner
Copy Link
Copy Source

@yg1031 commented on GitHub (Apr 25, 2025):

Try set auto_generate_name to false.

Not working, the chat_messages API takes more than 5 seconds before the first token come when conversation_id is set.
However, if conversation_id is set to empty, it takes less than 2 seconds, but without chat history.
Moreover, auto_generate_name does not affect first token delay. Dify version is 1.1.3, deployed on docker.

@yg1031 commented on GitHub (Apr 25, 2025): > Try set `auto_generate_name ` to false. Not working, the `chat_messages` API takes more than 5 seconds before the first token come when `conversation_id` is set. However, if `conversation_id` is set to empty, it takes less than 2 seconds, but without chat history. Moreover, `auto_generate_name` does not affect first token delay. Dify version is 1.1.3, deployed on docker.
yindo commented 2026-02-21 18:11:59 -05:00
Author
Owner
Copy Link
Copy Source

@AugNSo commented on GitHub (Apr 25, 2025):

Try set auto_generate_name to false.

Not working, the chat_messages API takes more than 5 seconds before the first token come when conversation_id is set. However, if conversation_id is set to empty, it takes less than 2 seconds, but without chat history. Moreover, auto_generate_name does not affect first token delay. Dify version is 1.1.3, deployed on docker.

I would suggest that you open a separate issue. You are replying to an issue that was opened a long time ago, was properly addressed at the time, and has a different cause. Please make it easier for developers to track issues.

@AugNSo commented on GitHub (Apr 25, 2025): > > Try set `auto_generate_name ` to false. > > Not working, the `chat_messages` API takes more than 5 seconds before the first token come when `conversation_id` is set. However, if `conversation_id` is set to empty, it takes less than 2 seconds, but without chat history. Moreover, `auto_generate_name` does not affect first token delay. Dify version is 1.1.3, deployed on docker. I would suggest that you open a separate issue. You are replying to an issue that was opened a long time ago, was properly addressed at the time, and has a different cause. Please make it easier for developers to track issues.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#5634
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?