langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 04:25:29 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Models of TONGYI cannot be added (Dify 1.0.0) #8938

New Issue
Closed
opened 2026-02-21 18:28:43 -05:00 by yindo · 17 comments
yindo commented 2026-02-21 18:28:43 -05:00
Owner
Copy Link
Copy Source

Originally created by @hardyfu on GitHub (Mar 3, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.0.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

go to Model Provider and install TONGYI. after installing, click setting and enter API.

Image

✔️ Expected Behavior

All the model of TONGYI should be able to add.

❌ Actual Behavior

Most of models of TONGYI cannot be added in Model Provider, such as qwen-coder-plus, qwen-turbo.

Error msg when adding model like qwen-turbo:
parameter incremental_output only support stream call

Image
Originally created by @hardyfu on GitHub (Mar 3, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.0.0 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce go to Model Provider and install TONGYI. after installing, click setting and enter API. <img width="870" alt="Image" src="https://github.com/user-attachments/assets/3bb04310-739f-4e2c-a9af-10e3c9283cfd" /> ### ✔️ Expected Behavior All the model of TONGYI should be able to add. ### ❌ Actual Behavior Most of models of TONGYI cannot be added in Model Provider, such as qwen-coder-plus, qwen-turbo. Error msg when adding model like qwen-turbo: parameter incremental_output only support stream call <img width="1319" alt="Image" src="https://github.com/user-attachments/assets/727ce835-b232-46f7-9f60-5cc95c16f9c0" />
yindo added the 🐞 bug label 2026-02-21 18:28:43 -05:00
yindo closed this issue 2026-02-21 18:28:43 -05:00
yindo commented 2026-02-21 18:28:43 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Mar 3, 2025):

@hardyfu Hello there! I'm here to help you with any bugs, questions, or contributions you may have regarding Dify. I'm looking forward to assisting you!

I found a similar unsolved discussion regarding the "incremental_output Only Supports Stream Call" error in Dify API settings. You can view the discussion here [1].

Regarding the error message "parameter incremental_output only support stream call," it seems that the TONGYI models like qwen-coder-plus and qwen-turbo may require specific configurations to support streaming. Dify's Model Provider interface handles streaming calls through the _invoke method, which supports both streaming and synchronous returns. The stream parameter in this method determines whether the output is in a streaming manner. If stream is set to True, the method returns a generator that yields LLMResultChunk objects, allowing for streaming output [2][3].

To resolve this issue, ensure that the models are configured to handle streaming calls by implementing the _invoke method with the capability to handle the stream parameter. This might involve checking the model's YAML configuration and ensuring that it supports the necessary parameters for streaming.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Mar 3, 2025): <!-- Greeting --> @hardyfu Hello there! I'm here to help you with any bugs, questions, or contributions you may have regarding Dify. I'm looking forward to assisting you! <!-- Answer --> I found a similar unsolved discussion regarding the "incremental_output Only Supports Stream Call" error in Dify API settings. You can view the discussion [here](https://github.com/langgenius/dify/discussions/13605) <sup>[[1]](https://github.com/langgenius/dify/discussions/13605)</sup>. Regarding the error message "parameter incremental_output only support stream call," it seems that the TONGYI models like qwen-coder-plus and qwen-turbo may require specific configurations to support streaming. Dify's Model Provider interface handles streaming calls through the `_invoke` method, which supports both streaming and synchronous returns. The `stream` parameter in this method determines whether the output is in a streaming manner. If `stream` is set to `True`, the method returns a generator that yields `LLMResultChunk` objects, allowing for streaming output <sup>[[2]](https://github.com/langgenius/dify/blob/main/api/core/model_runtime/docs/en_US/interfaces.md)</sup><sup>[[3]](https://github.com/langgenius/dify-docs/blob/main/en/guides/model-configuration/interfaces.md)</sup>. To resolve this issue, ensure that the models are configured to handle streaming calls by implementing the `_invoke` method with the capability to handle the `stream` parameter. This might involve checking the model's YAML configuration and ensuring that it supports the necessary parameters for streaming. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/faacf26b-59b7-4ae0-a09f-6e680fbfc6c4?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/faacf26b-59b7-4ae0-a09f-6e680fbfc6c4?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/faacf26b-59b7-4ae0-a09f-6e680fbfc6c4?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/faacf26b-59b7-4ae0-a09f-6e680fbfc6c4?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/faacf26b-59b7-4ae0-a09f-6e680fbfc6c4?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/faacf26b-59b7-4ae0-a09f-6e680fbfc6c4?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/faacf26b-59b7-4ae0-a09f-6e680fbfc6c4?feedback_type=other)</sup>
yindo commented 2026-02-21 18:28:44 -05:00
Author
Owner
Copy Link
Copy Source

@hardyfu commented on GitHub (Mar 3, 2025):

@dosu thanks. I've read the discussion, but I don't know how to change incremental-output=stream in core/model_runtime/moder_viewers/tongyi/llm/llm.py, I can't find it.

@hardyfu commented on GitHub (Mar 3, 2025): @dosu thanks. I've read the discussion, but I don't know how to change incremental-output=stream in core/model_runtime/moder_viewers/tongyi/llm/llm.py, I can't find it.
yindo commented 2026-02-21 18:28:44 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Mar 3, 2025):

The incremental_output parameter is set in the _generate method within the llm.py file located in the core/model_runtime/model_providers/tongyi/llm directory. It is set to the value of the stream parameter, which means if stream is True, incremental_output will also be True [1].

Here's a snippet of how it is set:

response = Generation.call(**params, result_format="message", stream=stream, incremental_output=stream)

To modify this, you would need to ensure that when you call this method, the stream parameter is set to True to enable streaming and thus set incremental_output to True. If you are unable to find the file, make sure you are looking in the correct directory path as mentioned above.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Mar 3, 2025): <!-- Answer --> The `incremental_output` parameter is set in the `_generate` method within the `llm.py` file located in the `core/model_runtime/model_providers/tongyi/llm` directory. It is set to the value of the `stream` parameter, which means if `stream` is `True`, `incremental_output` will also be `True` <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/core/model_runtime/model_providers/tongyi/llm/llm.py)</sup>. Here's a snippet of how it is set: ```python response = Generation.call(**params, result_format="message", stream=stream, incremental_output=stream) ``` To modify this, you would need to ensure that when you call this method, the `stream` parameter is set to `True` to enable streaming and thus set `incremental_output` to `True`. If you are unable to find the file, make sure you are looking in the correct directory path as mentioned above. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/c7fd27af-4330-4c67-b8ec-e01e0b87ed34?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/c7fd27af-4330-4c67-b8ec-e01e0b87ed34?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/c7fd27af-4330-4c67-b8ec-e01e0b87ed34?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/c7fd27af-4330-4c67-b8ec-e01e0b87ed34?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/c7fd27af-4330-4c67-b8ec-e01e0b87ed34?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/c7fd27af-4330-4c67-b8ec-e01e0b87ed34?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/c7fd27af-4330-4c67-b8ec-e01e0b87ed34?feedback_type=other)</sup>
yindo commented 2026-02-21 18:28:44 -05:00
Author
Owner
Copy Link
Copy Source

@hardyfu commented on GitHub (Mar 3, 2025):

@dosu thanks. I found the file and make the change accordingly, and restart dify and re-install TONGYI. but the issue is still exist.

@hardyfu commented on GitHub (Mar 3, 2025): @dosu thanks. I found the file and make the change accordingly, and restart dify and re-install TONGYI. but the issue is still exist.
yindo commented 2026-02-21 18:28:44 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Mar 3, 2025):

Ensure that the TONGYI models are configured to support streaming in their YAML configuration files. The incremental_output parameter should be set to True by ensuring the stream parameter is True in the _invoke and _generate methods. Double-check that the changes are correctly saved and that the application is fully restarted after making these changes.

If the issue persists, verify that there are no additional configurations or dependencies required for TONGYI models to handle streaming calls. Unfortunately, specific configurations for TONGYI models to support streaming are not detailed in the available documentation, so reviewing any related documentation or comments in the codebase might provide further insights.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Mar 3, 2025): <!-- Answer --> Ensure that the TONGYI models are configured to support streaming in their YAML configuration files. The `incremental_output` parameter should be set to `True` by ensuring the `stream` parameter is `True` in the `_invoke` and `_generate` methods. Double-check that the changes are correctly saved and that the application is fully restarted after making these changes. If the issue persists, verify that there are no additional configurations or dependencies required for TONGYI models to handle streaming calls. Unfortunately, specific configurations for TONGYI models to support streaming are not detailed in the available documentation, so reviewing any related documentation or comments in the codebase might provide further insights. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/617504c9-ad3a-47f5-92df-4ea0d673dc21?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/617504c9-ad3a-47f5-92df-4ea0d673dc21?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/617504c9-ad3a-47f5-92df-4ea0d673dc21?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/617504c9-ad3a-47f5-92df-4ea0d673dc21?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/617504c9-ad3a-47f5-92df-4ea0d673dc21?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/617504c9-ad3a-47f5-92df-4ea0d673dc21?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/617504c9-ad3a-47f5-92df-4ea0d673dc21?feedback_type=other)</sup>
yindo commented 2026-02-21 18:28:44 -05:00
Author
Owner
Copy Link
Copy Source

@lisontowind commented on GitHub (Mar 3, 2025):

@laipz8200
The 0.04 version of the Tongyi plugin does not have this issue.
If the API key was already entered when using the 0.04 version, upgrading to version 0.05 does not require entering the key again.
However, this leads to new problems, causing the model to be unusable.
This is due to the addition of the deepseek-r1 model, and I hope the developers can address this issue.
Image

@lisontowind commented on GitHub (Mar 3, 2025): @laipz8200 The 0.04 version of the Tongyi plugin does not have this issue. If the API key was already entered when using the 0.04 version, upgrading to version 0.05 does not require entering the key again. However, this leads to new problems, causing the model to be unusable. This is due to the addition of the deepseek-r1 model, and I hope the developers can address this issue. ![Image](https://github.com/user-attachments/assets/65aa3950-865b-4034-9de3-f3c54e833e2c)
yindo commented 2026-02-21 18:28:44 -05:00
Author
Owner
Copy Link
Copy Source

@vindia9 commented on GitHub (Mar 3, 2025):

I'm also experiencing the same issue.

After upgrading from version 0.0.4 to 0.0.5, the Qwen models are no longer working. The error message is:

[tongyi] Error: PluginInvokeError: {"args":{},"error_type":"UnboundLocalError","message":"cannot access local variable 'is_reasoning_started' where it is not associated with a value"}

When I tried deleting "tongyi" from the model list and re-entering the API key, I still couldn't add it successfully. The error message that pops up in the top right corner of the page is:

parameter incremental_output only support stream call

Dify version: 1.0.0

@vindia9 commented on GitHub (Mar 3, 2025): I'm also experiencing the same issue. After upgrading from version 0.0.4 to 0.0.5, the Qwen models are no longer working. The error message is: ```text [tongyi] Error: PluginInvokeError: {"args":{},"error_type":"UnboundLocalError","message":"cannot access local variable 'is_reasoning_started' where it is not associated with a value"} ``` When I tried deleting "tongyi" from the model list and re-entering the API key, I still couldn't add it successfully. The error message that pops up in the top right corner of the page is: ```text parameter incremental_output only support stream call ``` Dify version: 1.0.0
yindo commented 2026-02-21 18:28:45 -05:00
Author
Owner
Copy Link
Copy Source

@hardyfu commented on GitHub (Mar 3, 2025):

yes, actually, I encountered the similiar issue when doing knowledge embedding :(

{
"code": "indexing_estimate_error",
"message": "PluginInvokeError: {"error_type":"unmarshal_error","message":"unmarshal json failed: json: cannot unmarshal number into Go struct field GetTextEmbeddingNumTokensResponse.num_tokens of type []int"}",
"status": 500
}

It seems like a lot of bugs in this new version.

@hardyfu commented on GitHub (Mar 3, 2025): yes, actually, I encountered the similiar issue when doing knowledge embedding :( { "code": "indexing_estimate_error", "message": "PluginInvokeError: {\"error_type\":\"unmarshal_error\",\"message\":\"unmarshal json failed: json: cannot unmarshal number into Go struct field GetTextEmbeddingNumTokensResponse.num_tokens of type []int\"}", "status": 500 } It seems like a lot of bugs in this new version.
yindo commented 2026-02-21 18:28:45 -05:00
Author
Owner
Copy Link
Copy Source

@hardyfu commented on GitHub (Mar 3, 2025):

anyone can share the older version (v0.15.3) docker-compose.yaml?

@hardyfu commented on GitHub (Mar 3, 2025): anyone can share the older version (v0.15.3) docker-compose.yaml?
yindo commented 2026-02-21 18:28:45 -05:00
Author
Owner
Copy Link
Copy Source

@lisontowind commented on GitHub (Mar 3, 2025):

@hardyfu You don't need to download v0.15.3; you just need to uninstall the 0.05 version of the Tongyi plugin and then go to the plugin market to download and install the 0.04 version of the plugin.
You can obtain version 0.15.3 through the following command:
git clone https://github.com/langgenius/dify.git --branch 0.15.3

@lisontowind commented on GitHub (Mar 3, 2025): @hardyfu You don't need to download v0.15.3; you just need to uninstall the 0.05 version of the Tongyi plugin and then go to the plugin market to download and install the 0.04 version of the plugin. You can obtain version 0.15.3 through the following command: `git clone https://github.com/langgenius/dify.git --branch 0.15.3`
yindo commented 2026-02-21 18:28:45 -05:00
Author
Owner
Copy Link
Copy Source

@hardyfu commented on GitHub (Mar 3, 2025):

thanks @lisontowind

@hardyfu commented on GitHub (Mar 3, 2025): thanks @lisontowind
yindo commented 2026-02-21 18:28:45 -05:00
Author
Owner
Copy Link
Copy Source

@HuChundong commented on GitHub (Mar 3, 2025):

yes, actually, I encountered the similiar issue when doing knowledge embedding :(

{ "code": "indexing_estimate_error", "message": "PluginInvokeError: {"error_type":"unmarshal_error","message":"unmarshal json failed: json: cannot unmarshal number into Go struct field GetTextEmbeddingNumTokensResponse.num_tokens of type []int"}", "status": 500 }

It seems like a lot of bugs in this new version.

not a lot, looks like full of bug.

@HuChundong commented on GitHub (Mar 3, 2025): > yes, actually, I encountered the similiar issue when doing knowledge embedding :( > > { "code": "indexing_estimate_error", "message": "PluginInvokeError: {"error_type":"unmarshal_error","message":"unmarshal json failed: json: cannot unmarshal number into Go struct field GetTextEmbeddingNumTokensResponse.num_tokens of type []int"}", "status": 500 } > > It seems like a lot of bugs in this new version. not a lot, looks like full of bug.
yindo commented 2026-02-21 18:28:45 -05:00
Author
Owner
Copy Link
Copy Source

@hardyfu commented on GitHub (Mar 3, 2025):

anyway, thanks guys for your kind help. I'm using 0.15.3 now...

@hardyfu commented on GitHub (Mar 3, 2025): anyway, thanks guys for your kind help. I'm using 0.15.3 now...
yindo commented 2026-02-21 18:28:45 -05:00
Author
Owner
Copy Link
Copy Source

@okokcc5 commented on GitHub (Mar 3, 2025):

Finally, by modifying the get_num_tokens method in text_embedding_model.py, it now appears to be functioning normally without any issues and can run smoothly.

@okokcc5 commented on GitHub (Mar 3, 2025): Finally, by modifying the get_num_tokens method in text_embedding_model.py, it now appears to be functioning normally without any issues and can run smoothly.
yindo commented 2026-02-21 18:28:45 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Mar 4, 2025):

Duplicated https://github.com/langgenius/dify/issues/14736

@crazywoola commented on GitHub (Mar 4, 2025): Duplicated https://github.com/langgenius/dify/issues/14736
yindo commented 2026-02-21 18:28:45 -05:00
Author
Owner
Copy Link
Copy Source

@zhangjianlongdba commented on GitHub (Mar 4, 2025):

On the page https://marketplace.dify.ai/plugins/langgenius/tongyi, click on the version history, download version 0.4 to your local machine, and then install it via the local plugin installation method. This will resolve the issue.

@zhangjianlongdba commented on GitHub (Mar 4, 2025): On the page https://marketplace.dify.ai/plugins/langgenius/tongyi, click on the version history, download version 0.4 to your local machine, and then install it via the local plugin installation method. This will resolve the issue.
yindo commented 2026-02-21 18:28:45 -05:00
Author
Owner
Copy Link
Copy Source

@choiking commented on GitHub (Mar 6, 2025):

On the page https://marketplace.dify.ai/plugins/langgenius/tongyi, click on the version history, download version 0.4 to your local machine, and then install it via the local plugin installation method. This will resolve the issue.

where is the option to install it via the local plugin installation method? I am locally running dify

@choiking commented on GitHub (Mar 6, 2025): > On the page https://marketplace.dify.ai/plugins/langgenius/tongyi, click on the version history, download version 0.4 to your local machine, and then install it via the local plugin installation method. This will resolve the issue. where is the option to install it via the local plugin installation method? I am locally running dify
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#8938
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?