langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-21 17:45:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

Qwen3 Switching between thinking and non thinking #252

New Issue
Closed
opened 2026-02-16 10:18:36 -05:00 by yindo · 4 comments
yindo commented 2026-02-16 10:18:36 -05:00
Owner
Copy Link
Copy Source

Originally created by @chunxingque on GitHub (May 5, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.2.0

Cloud or Self Hosted

Self Hosted (Source), Self Hosted (Docker)

Steps to reproduce

I deploy the latest Qwen3 model using sglang
(https://www.modelscope.cn/models/Qwen/Qwen3-32B)

run shell：

python3 -m sglang.launch_server \
--model-path /data/models/Qwen3-32B \
--host 0.0.0.0 \
--port 8000 \
--trust-remote-code \
--tp 2 \
--mem-fraction-static 0.95 \
--max-total-tokens  32768 \
--chunked-prefill-size 8192 \
--api-key mykey \
--reasoning-parser qwen3

Using OpenAI-API compatible plugin to use the model
model repo: [https://www.modelscope.cn/models/Qwen/Qwen3-32B]
Qwen3 can switch between thinking and non thinking modes, as follows:
We provide a soft switch mechanism that allows users to dynamically control the model's behavior when enable_thinking=True. Specifically, you can add /think and /no_think to user prompts or system messages to switch the model's thinking mode from turn to turn. The model will follow the most recent instruction in multi-turn conversations.

✔️ Expected Behavior

When I use the/no_think tag, dify still returns the tag for thinking

{
  "text": "<think>\n\n\n\n</think>\n\n我是通义千问，阿里巴巴最新推出的通义千问大语言模型，是Qwen3。我基于更全面的知识语料，拥有更丰富的语言理解和表达能力。"
}

I want to remove the content of <think>\n\n\n\n</think>\n\n How can I achieve this?
or，How to add the parameter chat_template_kwargs": {"enable_thinking": false} in the OpenAI-API compatible plugin?

example：

curl http://localhost:30000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "Qwen/Qwen3-8B",
  "messages": [
    {"role": "user", "content": "Give me a short introduction to large language models."}
  ],
  "temperature": 0.7,
  "top_p": 0.8,
  "top_k": 20,
  "max_tokens": 8192,
  "presence_penalty": 1.5,
  "chat_template_kwargs": {"enable_thinking": false}
}'

❌ Actual Behavior

No response

Originally created by @chunxingque on GitHub (May 5, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.2.0 ### Cloud or Self Hosted Self Hosted (Source), Self Hosted (Docker) ### Steps to reproduce I deploy the latest Qwen3 model using sglang (https://www.modelscope.cn/models/Qwen/Qwen3-32B) run shell： ``` python3 -m sglang.launch_server \ --model-path /data/models/Qwen3-32B \ --host 0.0.0.0 \ --port 8000 \ --trust-remote-code \ --tp 2 \ --mem-fraction-static 0.95 \ --max-total-tokens 32768 \ --chunked-prefill-size 8192 \ --api-key mykey \ --reasoning-parser qwen3 ``` Using OpenAI-API compatible plugin to use the model model repo: [https://www.modelscope.cn/models/Qwen/Qwen3-32B] Qwen3 can switch between thinking and non thinking modes, as follows: We provide a soft switch mechanism that allows users to dynamically control the model's behavior when enable_thinking=True. Specifically, you can add /think and /no_think to user prompts or system messages to switch the model's thinking mode from turn to turn. The model will follow the most recent instruction in multi-turn conversations. ### ✔️ Expected Behavior When I use the/no_think tag, dify still returns the tag for thinking ``` { "text": "<think>\n\n\n\n</think>\n\n我是通义千问，阿里巴巴最新推出的通义千问大语言模型，是Qwen3。我基于更全面的知识语料，拥有更丰富的语言理解和表达能力。" } ``` I want to remove the content of `<think>\n\n\n\n</think>\n\n` How can I achieve this? or，How to add the parameter `chat_template_kwargs": {"enable_thinking": false}` in the OpenAI-API compatible plugin? example： ``` curl http://localhost:30000/v1/chat/completions -H "Content-Type: application/json" -d '{ "model": "Qwen/Qwen3-8B", "messages": [ {"role": "user", "content": "Give me a short introduction to large language models."} ], "temperature": 0.7, "top_p": 0.8, "top_k": 20, "max_tokens": 8192, "presence_penalty": 1.5, "chat_template_kwargs": {"enable_thinking": false} }' ``` ### ❌ Actual Behavior _No response_
yindo closed this issue 2026-02-16 10:18:36 -05:00
yindo commented 2026-02-16 10:18:36 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (May 5, 2025):

Hello @chunxingque! I'm Dosu, your friendly bot here to help you solve bugs, answer questions, and assist with becoming a contributor. I'm here while you wait for a human maintainer. How can I assist you today?

I couldn't find specific information on how to configure the chat_template_kwargs parameter in the OpenAI-API compatible plugin within Dify. However, I did find a similar unsolved discussion that might be relevant to your issue: Help how can I split the thought content? I just want the final answer. [1].

For your issue with the <think> tags, Dify processes and returns these tags by rendering them on the frontend instead of the backend. This change prevents HTML tags from being inserted into the API response content while still allowing the display of the thinking process. The functionality is managed by the preprocessThinkTag function in web/app/components/base/markdown.tsx. In Dify v0.15.3 and above, the thinking process and answer are automatically separated using HTML details tags, which improves the rendering of the separation in the frontend. However, in Dify v0.15.3, the <think> tag is not returned in the response when using the OpenAI-API-compatible interface, unlike when accessing the native OpenAI API directly. This feature has been restored in a recent pull request (PR langgenius/dify#13392) and will be available in versions after v0.15.3 [2][3][4][5].

To remove the <think> content, you might need to handle it on the client side by stripping out these tags from the response before displaying it. If you are looking to configure the chat_template_kwargs parameter, you might need to explore the plugin's documentation or source code for more details, as I couldn't find specific guidance on this.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (May 5, 2025): <!-- Greeting --> Hello @chunxingque! I'm [Dosu](https://dosu.dev), your friendly bot here to help you solve bugs, answer questions, and assist with becoming a contributor. I'm here while you wait for a human maintainer. How can I assist you today? <!-- Answer --> I couldn't find specific information on how to configure the `chat_template_kwargs` parameter in the OpenAI-API compatible plugin within Dify. However, I did find a similar unsolved discussion that might be relevant to your issue: [Help how can I split the thought content? I just want the final answer.](https://github.com/langgenius/dify/discussions/18244) <sup>[[1]](https://github.com/langgenius/dify/discussions/18244)</sup>. For your issue with the `<think>` tags, Dify processes and returns these tags by rendering them on the frontend instead of the backend. This change prevents HTML tags from being inserted into the API response content while still allowing the display of the thinking process. The functionality is managed by the `preprocessThinkTag` function in `web/app/components/base/markdown.tsx`. In Dify v0.15.3 and above, the thinking process and answer are automatically separated using HTML details tags, which improves the rendering of the separation in the frontend. However, in Dify v0.15.3, the `<think>` tag is not returned in the response when using the OpenAI-API-compatible interface, unlike when accessing the native OpenAI API directly. This feature has been restored in a recent pull request (PR langgenius/dify#13392) and will be available in versions after v0.15.3 <sup>[[2]](https://github.com/langgenius/dify/issues/13419)</sup><sup>[[3]](https://github.com/langgenius/dify/issues/14594)</sup><sup>[[4]](https://github.com/langgenius/dify/issues/13621)</sup><sup>[[5]](https://github.com/langgenius/dify/issues/14762)</sup>. To remove the `<think>` content, you might need to handle it on the client side by stripping out these tags from the response before displaying it. If you are looking to configure the `chat_template_kwargs` parameter, you might need to explore the plugin's documentation or source code for more details, as I couldn't find specific guidance on this. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/d0524b38-1024-4d77-b784-f93f7798a689?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/d0524b38-1024-4d77-b784-f93f7798a689?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/d0524b38-1024-4d77-b784-f93f7798a689?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/d0524b38-1024-4d77-b784-f93f7798a689?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/d0524b38-1024-4d77-b784-f93f7798a689?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/d0524b38-1024-4d77-b784-f93f7798a689?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/d0524b38-1024-4d77-b784-f93f7798a689?feedback_type=other)</sup>
yindo commented 2026-02-16 10:18:36 -05:00
Author
Owner
Copy Link
Copy Source

@quicksandznzn commented on GitHub (May 5, 2025):

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.[x] I have searched for existing issues search for existing issues, including closed ones.[x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).[x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)[x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.2.0

Cloud or Self Hosted

Self Hosted (Source), Self Hosted (Docker)

Steps to reproduce

I deploy the latest Qwen3 model using sglang (https://www.modelscope.cn/models/Qwen/Qwen3-32B)

run shell：

python3 -m sglang.launch_server \
--model-path /data/models/Qwen3-32B \
--host 0.0.0.0 \
--port 8000 \
--trust-remote-code \
--tp 2 \
--mem-fraction-static 0.95 \
--max-total-tokens  32768 \
--chunked-prefill-size 8192 \
--api-key mykey \
--reasoning-parser qwen3

Using OpenAI-API compatible plugin to use the model model repo: [https://www.modelscope.cn/models/Qwen/Qwen3-32B] Qwen3 can switch between thinking and non thinking modes, as follows: We provide a soft switch mechanism that allows users to dynamically control the model's behavior when enable_thinking=True. Specifically, you can add /think and /no_think to user prompts or system messages to switch the model's thinking mode from turn to turn. The model will follow the most recent instruction in multi-turn conversations.

✔️ Expected Behavior

When I use the/no_think tag, dify still returns the tag for thinking

{
  "text": "<think>\n\n\n\n</think>\n\n我是通义千问，阿里巴巴最新推出的通义千问大语言模型，是Qwen3。我基于更全面的知识语料，拥有更丰富的语言理解和表达能力。"
}

I want to remove the content of <think>\n\n\n\n</think>\n\n How can I achieve this? or，How to add the parameter chat_template_kwargs": {"enable_thinking": false} in the OpenAI-API compatible plugin?

example：

curl http://localhost:30000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "Qwen/Qwen3-8B",
  "messages": [
    {"role": "user", "content": "Give me a short introduction to large language models."}
  ],
  "temperature": 0.7,
  "top_p": 0.8,
  "top_k": 20,
  "max_tokens": 8192,
  "presence_penalty": 1.5,
  "chat_template_kwargs": {"enable_thinking": false}
}'

❌ Actual Behavior

No response

update OpenAI-API-compatible to 0.0.14

@quicksandznzn commented on GitHub (May 5, 2025): > ### Self Checks > * [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).[x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.[x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)).[x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)[x] Please do not modify this template :) and fill in all the required fields. > > ### Dify version > 1.2.0 > > ### Cloud or Self Hosted > Self Hosted (Source), Self Hosted (Docker) > > ### Steps to reproduce > I deploy the latest Qwen3 model using sglang (https://www.modelscope.cn/models/Qwen/Qwen3-32B) > > run shell： > > ``` > python3 -m sglang.launch_server \ > --model-path /data/models/Qwen3-32B \ > --host 0.0.0.0 \ > --port 8000 \ > --trust-remote-code \ > --tp 2 \ > --mem-fraction-static 0.95 \ > --max-total-tokens 32768 \ > --chunked-prefill-size 8192 \ > --api-key mykey \ > --reasoning-parser qwen3 > ``` > > Using OpenAI-API compatible plugin to use the model model repo: [https://www.modelscope.cn/models/Qwen/Qwen3-32B] Qwen3 can switch between thinking and non thinking modes, as follows: We provide a soft switch mechanism that allows users to dynamically control the model's behavior when enable_thinking=True. Specifically, you can add /think and /no_think to user prompts or system messages to switch the model's thinking mode from turn to turn. The model will follow the most recent instruction in multi-turn conversations. > > ### ✔️ Expected Behavior > When I use the/no_think tag, dify still returns the tag for thinking > > ``` > { > "text": "<think>\n\n\n\n</think>\n\n我是通义千问，阿里巴巴最新推出的通义千问大语言模型，是Qwen3。我基于更全面的知识语料，拥有更丰富的语言理解和表达能力。" > } > ``` > > I want to remove the content of `<think>\n\n\n\n</think>\n\n` How can I achieve this? or，How to add the parameter `chat_template_kwargs": {"enable_thinking": false}` in the OpenAI-API compatible plugin? > > example： > > ``` > curl http://localhost:30000/v1/chat/completions -H "Content-Type: application/json" -d '{ > "model": "Qwen/Qwen3-8B", > "messages": [ > {"role": "user", "content": "Give me a short introduction to large language models."} > ], > "temperature": 0.7, > "top_p": 0.8, > "top_k": 20, > "max_tokens": 8192, > "presence_penalty": 1.5, > "chat_template_kwargs": {"enable_thinking": false} > }' > ``` > > ### ❌ Actual Behavior > _No response_ update OpenAI-API-compatible to 0.0.14
yindo commented 2026-02-16 10:18:36 -05:00
Author
Owner
Copy Link
Copy Source

@chunxingque commented on GitHub (May 6, 2025):

找到问题了,流返回的第二条返回"reasoning_content":"\n\n"，所以dify返回<think>\n\n\n\n</think>\n\n 内容。

b'data: {"id":"93df841df4f04146ad639a8c49c54f30","object":"chat.completion.chunk","created":1746518229,"model":"szzsfc-llm2","choices":[{"index":0,"delta":{"role":"assistant","content":null,"reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}\n\n'
b'data: {"id":"93df841df4f04146ad639a8c49c54f30","object":"chat.completion.chunk","created":1746518229,"model":"szzsfc-llm2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":"\\n\\n","tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}\n\n'
b'data: {"id":"93df841df4f04146ad639a8c49c54f30","object":"chat.completion.chunk","created":1746518229,"model":"szzsfc-llm2","choices":[{"index":0,"delta":{"role":null,"content":"\\n\\n","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}\n\n'
192.168.15.46 - - [06/May/2025 15:57:09] "POST /v1/chat/completions HTTP/1.1" 200 -
b'data: {"id":"93df841df4f04146ad639a8c49c54f30","object":"chat.completion.chunk","created":1746518229,"model":"szzsfc-llm2","choices":[{"index":0,"delta":{"role":null,"content":"\xe6\x88\x91\xe6\x98\xaf","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}\n\n'
@chunxingque commented on GitHub (May 6, 2025): 找到问题了,流返回的第二条返回"reasoning_content":"\\n\\n"，所以dify返回`<think>\n\n\n\n</think>\n\n `内容。 ``` b'data: {"id":"93df841df4f04146ad639a8c49c54f30","object":"chat.completion.chunk","created":1746518229,"model":"szzsfc-llm2","choices":[{"index":0,"delta":{"role":"assistant","content":null,"reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}\n\n' b'data: {"id":"93df841df4f04146ad639a8c49c54f30","object":"chat.completion.chunk","created":1746518229,"model":"szzsfc-llm2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":"\\n\\n","tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}\n\n' b'data: {"id":"93df841df4f04146ad639a8c49c54f30","object":"chat.completion.chunk","created":1746518229,"model":"szzsfc-llm2","choices":[{"index":0,"delta":{"role":null,"content":"\\n\\n","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}\n\n' 192.168.15.46 - - [06/May/2025 15:57:09] "POST /v1/chat/completions HTTP/1.1" 200 - b'data: {"id":"93df841df4f04146ad639a8c49c54f30","object":"chat.completion.chunk","created":1746518229,"model":"szzsfc-llm2","choices":[{"index":0,"delta":{"role":null,"content":"\xe6\x88\x91\xe6\x98\xaf","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}\n\n' ```
yindo commented 2026-02-16 10:18:36 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 19, 2025):

Hi, @chunxingque. I'm Dosu, and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You reported that using the /no_think tag with the Qwen3-32B model still results in output wrapped in tags.
  • I clarified that tags are processed on the frontend and recent updates have improved their handling.
  • Another user provided reproduction steps and inquired about disabling thinking via chat_template_kwargs, but no definitive solution was found.
  • You discovered that the reasoning_content field returns empty newlines, which causes the tags to appear, pointing to the model's response format as the root cause.

Next Steps:

  • Please confirm if this issue is still relevant with the latest version of the dify-official-plugins repository by commenting here.
  • If no further updates are provided, this issue will be automatically closed in 5 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Aug 19, 2025): Hi, @chunxingque. I'm [Dosu](https://dosu.dev), and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale. **Issue Summary:** - You reported that using the /no_think tag with the Qwen3-32B model still results in output wrapped in <think> tags. - I clarified that <think> tags are processed on the frontend and recent updates have improved their handling. - Another user provided reproduction steps and inquired about disabling thinking via chat_template_kwargs, but no definitive solution was found. - You discovered that the reasoning_content field returns empty newlines, which causes the <think> tags to appear, pointing to the model's response format as the root cause. **Next Steps:** - Please confirm if this issue is still relevant with the latest version of the dify-official-plugins repository by commenting here. - If no further updates are provided, this issue will be automatically closed in 5 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#252
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?