langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

azure openai api Stream options UI CHAT ERROR #6289

New Issue
Closed
opened 2026-02-21 18:15:00 -05:00 by yindo · 4 comments
yindo commented 2026-02-21 18:15:00 -05:00
Owner
Copy Link
Copy Source

Originally created by @dale22 on GitHub (Oct 24, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.10.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

use api can be set streaming . but ui not set , so is error
model name:gpt-4o-2024-05-13
azure openai api version:2024-05-01-preview
step
1,set model is ok，so API Key and Endpoint and deployment name is ok
image
2,but dify chat is error, and ui not set streaming false， docker-api logs:403
image
image

How I can ui chat ,close streaming

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

Originally created by @dale22 on GitHub (Oct 24, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version 0.10.0 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce use api can be set streaming . but ui not set , so is error model name:gpt-4o-2024-05-13 azure openai api version:2024-05-01-preview step 1,set model is ok，so API Key and Endpoint and deployment name is ok ![image](https://github.com/user-attachments/assets/46058437-7c27-4ab0-ae89-3b17be7096b1) 2,but dify chat is error, and ui not set streaming false， docker-api logs:403 ![image](https://github.com/user-attachments/assets/ee8e1f26-356b-4c98-b16d-a4f2b5c35340) ![image](https://github.com/user-attachments/assets/bfc6e0b0-7296-42ac-8c80-6bc5ee64e9de) How I can ui chat ,close streaming ### ✔️ Expected Behavior _No response_ ### ❌ Actual Behavior _No response_
yindo closed this issue 2026-02-21 18:15:00 -05:00
yindo commented 2026-02-21 18:15:00 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Oct 24, 2024):

I found some similar discussions and issues related to the Azure OpenAI API and streaming options in Dify:

  1. Azure openai api Stream options setting - This discussion is still open and was last updated on October 23, 2024.
  2. How to make the conversational application created using dify use streaming output? - This discussion is also open and was last updated on April 24, 2024.
  3. Stream options are not available for Azure openAI - This issue is closed. The suggested solution was that the error might be due to the model itself not supporting streaming, similar to issues encountered with structured output, regardless of whether it's OpenAI or Azure OpenAI.

To address your issue of disabling streaming in the Dify UI, it seems that Dify differentiates between models that support streaming and those that do not by using a boolean parameter stream in the inference function. If stream is set to False, it indicates that the model does not support streaming [1].

Additionally, Dify's UI indicates whether a model supports streaming through the stream_function_calling variable. This variable is shown when the model type is llm and is a select input with options for supported and not_supported. The default value is not_supported, indicating that streaming is not supported unless explicitly set to supported [2].

You might want to check the configuration in your Dify UI to ensure that the streaming option is set to not_supported for models that do not support it.

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Oct 24, 2024): <!-- Answer --> I found some similar discussions and issues related to the Azure OpenAI API and streaming options in Dify: 1. [Azure openai api Stream options setting](https://github.com/langgenius/dify/discussions/9715) - This discussion is still open and was last updated on October 23, 2024. 2. [How to make the conversational application created using dify use streaming output?](https://github.com/langgenius/dify/discussions/3767) - This discussion is also open and was last updated on April 24, 2024. 3. [Stream options are not available for Azure openAI](https://github.com/langgenius/dify/issues/9653) - This issue is closed. The suggested solution was that the error might be due to the model itself not supporting streaming, similar to issues encountered with structured output, regardless of whether it's OpenAI or Azure OpenAI. To address your issue of disabling streaming in the Dify UI, it seems that Dify differentiates between models that support streaming and those that do not by using a boolean parameter `stream` in the `inference` function. If `stream` is set to `False`, it indicates that the model does not support streaming <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/core/model_runtime/model_providers/sagemaker/llm/llm.py)</sup>. Additionally, Dify's UI indicates whether a model supports streaming through the `stream_function_calling` variable. This variable is shown when the model type is `llm` and is a select input with options for `supported` and `not_supported`. The default value is `not_supported`, indicating that streaming is not supported unless explicitly set to `supported` <sup>[[2]](https://github.com/langgenius/dify/blob/main/api/core/model_runtime/model_providers/openai_api_compatible/openai_api_compatible.yaml)</sup>. You might want to check the configuration in your Dify UI to ensure that the streaming option is set to `not_supported` for models that do not support it. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:15:00 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Oct 24, 2024):

This error is more like the model itself does not support streaming, and I remember that if I used structure output, there would be a similar error, no matter it's OpenAI or Azure OpenAI

Need to check if you are using structured output.

@crazywoola commented on GitHub (Oct 24, 2024): > This error is more like the model itself does not support streaming, and I remember that if I used structure output, there would be a similar error, no matter it's OpenAI or Azure OpenAI Need to check if you are using structured output.
yindo commented 2026-02-21 18:15:00 -05:00
Author
Owner
Copy Link
Copy Source

@dale22 commented on GitHub (Oct 24, 2024):

I don't need structural output , set response text, also this error
image

@dale22 commented on GitHub (Oct 24, 2024): I don't need structural output , set response text, also this error ![image](https://github.com/user-attachments/assets/f9a7693d-69c8-43d0-bc56-4a77ea6bb071)
yindo commented 2026-02-21 18:15:01 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Nov 24, 2024):

Hi, @dale22. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

  • The issue involves a bug with the Azure OpenAI API's streaming options in the Dify UI chat when using a self-hosted Docker setup.
  • Despite setting the API to streaming, the UI does not reflect this, resulting in an error and a 403 log.
  • I suggested checking the configuration to ensure the streaming option is set to not_supported for models that do not support it.
  • @crazywoola mentioned the error might be due to the model not supporting streaming, and you confirmed not using structured output but still encountering the error.

Next Steps:

  • Please confirm if this issue is still relevant to the latest version of the Dify repository. If so, you can keep the discussion open by commenting on the issue.
  • Otherwise, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Nov 24, 2024): Hi, @dale22. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog. I'm marking this issue as stale. **Issue Summary:** - The issue involves a bug with the Azure OpenAI API's streaming options in the Dify UI chat when using a self-hosted Docker setup. - Despite setting the API to streaming, the UI does not reflect this, resulting in an error and a 403 log. - I suggested checking the configuration to ensure the streaming option is set to `not_supported` for models that do not support it. - @crazywoola mentioned the error might be due to the model not supporting streaming, and you confirmed not using structured output but still encountering the error. **Next Steps:** - Please confirm if this issue is still relevant to the latest version of the Dify repository. If so, you can keep the discussion open by commenting on the issue. - Otherwise, this issue will be automatically closed in 15 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#6289
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?