Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3933] Qwen3 /chat without streaming requires enable_thinking=false for providers? #2504

New Issue
Closed
opened 2026-02-22 18:29:58 -05:00 by yindo · 6 comments
yindo commented 2026-02-22 18:29:58 -05:00
Owner
Copy Link
Copy Source

Originally created by @HarrisonZhang on GitHub (Jun 2, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3933

How are you running AnythingLLM?

Docker (local)

What happened?

Description:

When calling the /api/v1/workspace/{slug}/chat API endpoint (which is intended for non-streaming responses), I am consistently receiving a 400 error with the message: "parameter.enable_thinking must be set to false for non-streaming calls". This error occurs even though I am explicitly setting enable_thinking: false in my JSON request body.

Steps to Reproduce:

Make a POST request to the /api/v1/workspace/{workspace_slug}/chat endpoint.

Include the Authorization header with a valid Bearer token.

Include the Content-Type: application/json header.

Use the following JSON payload (or similar), ensuring enable_thinking is false:

JSON

{
"message": "What is AnythingLLM?",
"mode": "chat",
"sessionId": "some-session-id",
"enable_thinking": false
}
Example cURL command:

Bash

curl -X 'POST'
'http://YOUR_ANYTHINGLLM_HOST/api/v1/workspace/YOUR_WORKSPACE_SLUG/chat'
-H 'accept: application/json'
-H 'Authorization: Bearer YOUR_API_TOKEN'
-H 'Content-Type: application/json'
-d '{
"message": "What is AnythingLLM?",
"mode": "chat",
"sessionId": "some-session-id",
"enable_thinking": false
}'
Expected Behavior:

The API should accept the request and return a successful response (e.g., HTTP 200 OK) with the chat completion, as enable_thinking is correctly set to false for a non-streaming call.

Actual Behavior:

The API returns an HTTP 500 Internal Server Error, with the following JSON error response in the body (the HTTP status code in the UI screenshot was 500, but the error message inside the JSON payload indicates a 400-type parameter validation issue):

JSON

{
"id": "...", // Some unique ID
"type": "abort",
"textResponse": null,
"sources": [],
"error": "400 parameter.enable_thinking must be set to false for non-streaming calls"
}
(Attached screenshot from my API client tool also shows this behavior)

Troubleshooting Steps Taken:

Confirmed enable_thinking is explicitly set to false in the request body.
Verified the Content-Type header is application/json.
Tried simplifying the request body to minimal fields, still with enable_thinking: false.

Image

Are there known steps to reproduce?

No response

Originally created by @HarrisonZhang on GitHub (Jun 2, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3933 ### How are you running AnythingLLM? Docker (local) ### What happened? Description: When calling the /api/v1/workspace/{slug}/chat API endpoint (which is intended for non-streaming responses), I am consistently receiving a 400 error with the message: "parameter.enable_thinking must be set to false for non-streaming calls". This error occurs even though I am explicitly setting enable_thinking: false in my JSON request body. Steps to Reproduce: Make a POST request to the /api/v1/workspace/{workspace_slug}/chat endpoint. Include the Authorization header with a valid Bearer token. Include the Content-Type: application/json header. Use the following JSON payload (or similar), ensuring enable_thinking is false: JSON { "message": "What is AnythingLLM?", "mode": "chat", "sessionId": "some-session-id", "enable_thinking": false } Example cURL command: Bash curl -X 'POST' \ 'http://YOUR_ANYTHINGLLM_HOST/api/v1/workspace/YOUR_WORKSPACE_SLUG/chat' \ -H 'accept: application/json' \ -H 'Authorization: Bearer YOUR_API_TOKEN' \ -H 'Content-Type: application/json' \ -d '{ "message": "What is AnythingLLM?", "mode": "chat", "sessionId": "some-session-id", "enable_thinking": false }' Expected Behavior: The API should accept the request and return a successful response (e.g., HTTP 200 OK) with the chat completion, as enable_thinking is correctly set to false for a non-streaming call. Actual Behavior: The API returns an HTTP 500 Internal Server Error, with the following JSON error response in the body (the HTTP status code in the UI screenshot was 500, but the error message inside the JSON payload indicates a 400-type parameter validation issue): JSON { "id": "...", // Some unique ID "type": "abort", "textResponse": null, "sources": [], "error": "400 parameter.enable_thinking must be set to false for non-streaming calls" } (Attached screenshot from my API client tool also shows this behavior) Troubleshooting Steps Taken: Confirmed enable_thinking is explicitly set to false in the request body. Verified the Content-Type header is application/json. Tried simplifying the request body to minimal fields, still with enable_thinking: false. ![Image](https://github.com/user-attachments/assets/3b71a734-a2fe-49b8-a041-9c3bc7072206) ### Are there known steps to reproduce? _No response_
yindo added the possible bugneeds info / can't replicate labels 2026-02-22 18:29:58 -05:00
yindo closed this issue 2026-02-22 18:29:58 -05:00
yindo commented 2026-02-22 18:29:58 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Jun 2, 2025):

What model and provider are you using here for this? We actually do not pass all body params passed via the API unless we explicitly define them so when you add "enable_thinking": false to the body of the API call our backend is not passing this to your provider in the way you are expecting it. If you can provide me with more information on how I can replicate this, we can definitely make sure this gets fixed.

@shatfield4 commented on GitHub (Jun 2, 2025): What model and provider are you using here for this? We actually do not pass all body params passed via the API unless we explicitly define them so when you add `"enable_thinking": false` to the body of the API call our backend is not passing this to your provider in the way you are expecting it. If you can provide me with more information on how I can replicate this, we can definitely make sure this gets fixed.
yindo commented 2026-02-22 18:29:59 -05:00
Author
Owner
Copy Link
Copy Source

@HarrisonZhang commented on GitHub (Jun 2, 2025):

What model and provider are you using here for this? We actually do not pass all body params passed via the API unless we explicitly define them so when you add "enable_thinking": false to the body of the API call our backend is not passing this to your provider in the way you are expecting it. If you can provide me with more information on how I can replicate this, we can definitely make sure this gets fixed.

Thanks for your reply. I have installed the latest version of AnythingLLM. When I call the API 'http://localhost:3001/api/v1/workspace/{slug}/chat', it keeps prompting "400 parameter.enable_thinking must be set to false for non-streaming calls".

Image

However, when I call the endpoint 'http://localhost:3001/api/v1/workspace/{slug}/stream-chat', everything is normal.

Image

In AnythingLLM, the model I am using is qwen3. On the web page (UI), everything looks normal.

@HarrisonZhang commented on GitHub (Jun 2, 2025): > What model and provider are you using here for this? We actually do not pass all body params passed via the API unless we explicitly define them so when you add `"enable_thinking": false` to the body of the API call our backend is not passing this to your provider in the way you are expecting it. If you can provide me with more information on how I can replicate this, we can definitely make sure this gets fixed. Thanks for your reply. I have installed the latest version of AnythingLLM. When I call the API 'http://localhost:3001/api/v1/workspace/{slug}/chat', it keeps prompting "400 parameter.enable_thinking must be set to false for non-streaming calls". ![Image](https://github.com/user-attachments/assets/3474dac6-7eb6-40b1-a25e-3bc4e3327a24) However, when I call the endpoint 'http://localhost:3001/api/v1/workspace/{slug}/stream-chat', everything is normal. ![Image](https://github.com/user-attachments/assets/007401da-9aab-41bc-89e1-80310127b69e) In AnythingLLM, the model I am using is qwen3. On the web page (UI), everything looks normal.
yindo commented 2026-02-22 18:29:59 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 3, 2025):

When he asks for the provider he means like OpenAI, Ollama, LMStudio, etc, not the model. The issue is that we do not pass provider arguments via the body in the request. It seems like your provider does not allow non-streaming with thinking and is something specific with your LLM provider and how those properties propagate.

Which is odd to have as a requirement since thinking or not should really not be tied to streaming.

Which provider are you using so we can attempt to replicate since until then this is provider specific

@timothycarambat commented on GitHub (Jun 3, 2025): When he asks for the provider he means like OpenAI, Ollama, LMStudio, etc, not the model. The issue is that we do not pass provider arguments via the body in the request. It seems like your provider does not allow non-streaming _with thinking_ and is something specific with your LLM provider and how those properties propagate. Which is odd to have as a requirement since thinking or not should really not be tied to streaming. Which provider are you using so we can attempt to replicate since until then this is provider specific
yindo commented 2026-02-22 18:29:59 -05:00
Author
Owner
Copy Link
Copy Source

@HarrisonZhang commented on GitHub (Jun 4, 2025):

Thank you for your help. I now understand why this problem is happening.

When I use the Deepseek model, the API can be accessed normally. However, when I use the 'Generic OpenAI' provider to connect to the qwen3 model, calling the /chat API fails.

It's worth mentioning that the qwen3 model works normally for Q&A in the web UI, but not via the API endpoint.

Image

@HarrisonZhang commented on GitHub (Jun 4, 2025): Thank you for your help. I now understand why this problem is happening. When I use the Deepseek model, the API can be accessed normally. However, when I use the 'Generic OpenAI' provider to connect to the qwen3 model, calling the /chat API fails. It's worth mentioning that the qwen3 model works normally for Q&A in the web UI, but not via the API endpoint. ![Image](https://github.com/user-attachments/assets/d072a747-bebe-4fbe-9828-d08eed96d550)
yindo commented 2026-02-22 18:29:59 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 4, 2025):

When I use the Deepseek model, the API can be accessed normally. However, when I use the 'Generic OpenAI' provider to connect to the qwen3 model, calling the /chat API fails.

This makes sense, since Qwen3 has thinking_disabled or /no_think as a property you can use to disable thoughts. It is weird however that you have to disable that to do non-streaming output. We dont see that with any other Qwen3 providers :/

We dont have access to Ali-cloud since we are US-based. Ill have to see if this is replicable on OpenRouter or something to see if that is a common limitation

It's worth mentioning that the qwen3 model works normally for Q&A in the web UI, but not via the API endpoint.

Ah, that is because was also using the stream: true in the UI - since that is what people expect. That being said if streaming is off then calling @agent should also break in this cirumstance since that is streaming disabled as well - can you verify that?

@timothycarambat commented on GitHub (Jun 4, 2025): > When I use the Deepseek model, the API can be accessed normally. However, when I use the 'Generic OpenAI' provider to connect to the qwen3 model, calling the /chat API fails. This makes sense, since Qwen3 has `thinking_disabled` or `/no_think` as a property you can use to disable thoughts. It is weird however that you have to disable that to do non-streaming output. We dont see that with _any other_ Qwen3 providers :/ We dont have access to Ali-cloud since we are US-based. Ill have to see if this is replicable on OpenRouter or something to see if that is a common limitation > It's worth mentioning that the qwen3 model works normally for Q&A in the web UI, but not via the API endpoint. Ah, that is because was also using the `stream: true` in the UI - since that is what people expect. That being said if streaming is off then calling `@agent` should also break in this cirumstance since that is streaming disabled as well - can you verify that?
yindo commented 2026-02-22 18:29:59 -05:00
Author
Owner
Copy Link
Copy Source

@sizhongyibanhts commented on GitHub (Nov 24, 2025):

answer with think

aliyun

extra_body={"enable_thinking": True}

from openai import OpenAI
import os

# 初始化OpenAI客户端
client = OpenAI(
    # 如果没有配置环境变量，请用阿里云百炼API Key替换：api_key="sk-xxx"
    # 新加坡和北京地域的API Key不同。获取API Key：https://help.aliyun.com/zh/model-studio/get-api-key
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # 以下是北京地域base_url，如果使用新加坡地域的模型，需要将base_url替换为：https://dashscope-intl.aliyuncs.com/compatible-mode/v1
    base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)

messages = [{"role": "user", "content": "你是谁"}]

completion = client.chat.completions.create(
    model="qwen-plus",  # 您可以按需更换为其它深度思考模型
    messages=messages,
    extra_body={"enable_thinking": True},
    stream=True,
    stream_options={
        "include_usage": True
    },
)

reasoning_content = ""  # 完整思考过程
answer_content = ""  # 完整回复
is_answering = False  # 是否进入回复阶段
print("\n" + "=" * 20 + "思考过程" + "=" * 20 + "\n")

for chunk in completion:
    if not chunk.choices:
        print("\nUsage:")
        print(chunk.usage)
        continue

    delta = chunk.choices[0].delta

    # 只收集思考内容
    if hasattr(delta, "reasoning_content") and delta.reasoning_content is not None:
        if not is_answering:
            print(delta.reasoning_content, end="", flush=True)
        reasoning_content += delta.reasoning_content

    # 收到content，开始进行回复
    if hasattr(delta, "content") and delta.content:
        if not is_answering:
            print("\n" + "=" * 20 + "完整回复" + "=" * 20 + "\n")
            is_answering = True
        print(delta.content, end="", flush=True)
        answer_content += delta.content

result:


====================思考过程====================

好的，用户问“你是谁”，我需要给出一个准确且友好的回答。首先，我要确认自己的身份，即通义千问，由阿里巴巴集团旗下的通义实验室研发。接下来，应该说明我的主要功能，比如回答问题、创作文字、逻辑推理等。同时，要保持语气亲切，避免过于技术化，让用户感觉轻松。还要注意不要使用复杂术语，确保回答简洁明了。另外，可能需要加入一些互动元素，邀请用户提问，促进进一步交流。最后，检查是否有遗漏的重要信息，比如我的中文名称“通义千问”和英文名称“Qwen”，以及所属公司和实验室。确保回答全面且符合用户期望。
====================完整回复====================

你好！我是通义千问，是阿里巴巴集团旗下的通义实验室自主研发的超大规模语言模型。我可以回答问题、创作文字、进行逻辑推理、编程等，旨在为用户提供高质量的信息和服务。你可以叫我Qwen，或者直接叫我通义千问。有什么我可以帮你的吗？

answer without think

aliyun

import os
from openai import OpenAI

# 1. 准备工作：初始化客户端
client = OpenAI(
    # 建议通过环境变量配置API Key，避免硬编码。
    api_key=os.environ["DASHSCOPE_API_KEY"],
    # API Key与地域强绑定，请确保base_url与API Key的地域一致。
    base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)

# 2. 发起流式请求
completion = client.chat.completions.create(
    model="qwen-plus",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "请介绍一下自己"}
    ],
    stream=True,
    stream_options={"include_usage": True}
)

# 3. 处理流式响应
# 用列表暂存响应片段，最后 join 比逐次 += 字符串更高效
content_parts = []
print("AI: ", end="", flush=True)

for chunk in completion:
    if chunk.choices:
        content = chunk.choices[0].delta.content or ""
        print(content, end="", flush=True)
        content_parts.append(content)
    elif chunk.usage:
        print("\n--- 请求用量 ---")
        print(f"输入 Tokens: {chunk.usage.prompt_tokens}")
        print(f"输出 Tokens: {chunk.usage.completion_tokens}")
        print(f"总计 Tokens: {chunk.usage.total_tokens}")

full_response = "".join(content_parts)
# print(f"\n--- 完整回复 ---\n{full_response}")

result:

AI: 你好！我是Qwen，是阿里巴巴集团旗下的通义实验室自主研发的超大规模语言模型。我能够回答问题、创作文字，比如写故事、写公文、写邮件、写剧本、逻辑推理、编程等等，还能表达观点，玩游戏等。我支持多种语言，包括但不限于中文、英文、德语、法语、西班牙语等。如果你有任何问题或需要帮助，欢迎随时告诉我！
--- 请求用量 ---
输入 Tokens: 26
输出 Tokens: 87
总计 Tokens: 113
@sizhongyibanhts commented on GitHub (Nov 24, 2025): # answer with think [aliyun](https://bailian.console.aliyun.com/?utm_content=se_1021228191&gclid=Cj0KCQiAoZDJBhC0ARIsAERP-F8HnOWbk7G-BoNogxDtD83gCGiV2hUC4_2rF5Xk7QLqzrs9ZsmpCWEaAuQcEALw_wcB&tab=doc#/doc/?type=model&url=2870973) extra_body={"enable_thinking": True} ``` python from openai import OpenAI import os # 初始化OpenAI客户端 client = OpenAI( # 如果没有配置环境变量，请用阿里云百炼API Key替换：api_key="sk-xxx" # 新加坡和北京地域的API Key不同。获取API Key：https://help.aliyun.com/zh/model-studio/get-api-key api_key=os.getenv("DASHSCOPE_API_KEY"), # 以下是北京地域base_url，如果使用新加坡地域的模型，需要将base_url替换为：https://dashscope-intl.aliyuncs.com/compatible-mode/v1 base_url="https://dashscope.aliyuncs.com/compatible-mode/v1", ) messages = [{"role": "user", "content": "你是谁"}] completion = client.chat.completions.create( model="qwen-plus", # 您可以按需更换为其它深度思考模型 messages=messages, extra_body={"enable_thinking": True}, stream=True, stream_options={ "include_usage": True }, ) reasoning_content = "" # 完整思考过程 answer_content = "" # 完整回复 is_answering = False # 是否进入回复阶段 print("\n" + "=" * 20 + "思考过程" + "=" * 20 + "\n") for chunk in completion: if not chunk.choices: print("\nUsage:") print(chunk.usage) continue delta = chunk.choices[0].delta # 只收集思考内容 if hasattr(delta, "reasoning_content") and delta.reasoning_content is not None: if not is_answering: print(delta.reasoning_content, end="", flush=True) reasoning_content += delta.reasoning_content # 收到content，开始进行回复 if hasattr(delta, "content") and delta.content: if not is_answering: print("\n" + "=" * 20 + "完整回复" + "=" * 20 + "\n") is_answering = True print(delta.content, end="", flush=True) answer_content += delta.content ``` result: ``` python ====================思考过程==================== 好的，用户问“你是谁”，我需要给出一个准确且友好的回答。首先，我要确认自己的身份，即通义千问，由阿里巴巴集团旗下的通义实验室研发。接下来，应该说明我的主要功能，比如回答问题、创作文字、逻辑推理等。同时，要保持语气亲切，避免过于技术化，让用户感觉轻松。还要注意不要使用复杂术语，确保回答简洁明了。另外，可能需要加入一些互动元素，邀请用户提问，促进进一步交流。最后，检查是否有遗漏的重要信息，比如我的中文名称“通义千问”和英文名称“Qwen”，以及所属公司和实验室。确保回答全面且符合用户期望。 ====================完整回复==================== 你好！我是通义千问，是阿里巴巴集团旗下的通义实验室自主研发的超大规模语言模型。我可以回答问题、创作文字、进行逻辑推理、编程等，旨在为用户提供高质量的信息和服务。你可以叫我Qwen，或者直接叫我通义千问。有什么我可以帮你的吗？ ``` # answer without think [aliyun](https://bailian.console.aliyun.com/?utm_content=se_1021228191&gclid=Cj0KCQiAoZDJBhC0ARIsAERP-F8HnOWbk7G-BoNogxDtD83gCGiV2hUC4_2rF5Xk7QLqzrs9ZsmpCWEaAuQcEALw_wcB&tab=doc#/doc/?type=model&url=2866129) ``` python import os from openai import OpenAI # 1. 准备工作：初始化客户端 client = OpenAI( # 建议通过环境变量配置API Key，避免硬编码。 api_key=os.environ["DASHSCOPE_API_KEY"], # API Key与地域强绑定，请确保base_url与API Key的地域一致。 base_url="https://dashscope.aliyuncs.com/compatible-mode/v1", ) # 2. 发起流式请求 completion = client.chat.completions.create( model="qwen-plus", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "请介绍一下自己"} ], stream=True, stream_options={"include_usage": True} ) # 3. 处理流式响应 # 用列表暂存响应片段，最后 join 比逐次 += 字符串更高效 content_parts = [] print("AI: ", end="", flush=True) for chunk in completion: if chunk.choices: content = chunk.choices[0].delta.content or "" print(content, end="", flush=True) content_parts.append(content) elif chunk.usage: print("\n--- 请求用量 ---") print(f"输入 Tokens: {chunk.usage.prompt_tokens}") print(f"输出 Tokens: {chunk.usage.completion_tokens}") print(f"总计 Tokens: {chunk.usage.total_tokens}") full_response = "".join(content_parts) # print(f"\n--- 完整回复 ---\n{full_response}") ``` result: ```python AI: 你好！我是Qwen，是阿里巴巴集团旗下的通义实验室自主研发的超大规模语言模型。我能够回答问题、创作文字，比如写故事、写公文、写邮件、写剧本、逻辑推理、编程等等，还能表达观点，玩游戏等。我支持多种语言，包括但不限于中文、英文、德语、法语、西班牙语等。如果你有任何问题或需要帮助，欢迎随时告诉我！ --- 请求用量 --- 输入 Tokens: 26 输出 Tokens: 87 总计 Tokens: 113 ```
yindo changed title from Qwen3 `/chat` without streaming requires `enable_thinking=false` for providers? to [GH-ISSUE #3933] Qwen3 `/chat` without streaming requires `enable_thinking=false` for providers? 2026-06-05 14:46:57 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label needs info / can't replicate possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2504
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?