Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5683] Agent loop 400 errors with DeepSeek v4 thinking models — reasoning_content not preserved through tool calls #5233

New Issue
Closed
opened 2026-06-05 14:52:49 -04:00 by yindo · 1 comment
yindo commented 2026-06-05 14:52:49 -04:00
Owner
Copy Link
Copy Source

Originally created by @gurodrigues on GitHub (May 22, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5683

Originally assigned to: @timothycarambat on GitHub.

Description

The @agent loop throws 400 errors when using DeepSeek v4 thinking models (deepseek-v4-flash, deepseek-v4-pro) in multi-turn tool call flows. The root cause is that reasoning_content from previous assistant responses is not preserved through the agent pipeline — it's replaced with an empty string "".

Root Cause

DeepSeek has two model families with opposite requirements for reasoning_content:

Model Docs Rule
deepseek-reasoner Reasoning Model reasoning_content in input → 400 error (must strip it)
deepseek-v4-flash / deepseek-v4-pro Thinking Mode reasoning_content in input → required after tool calls

From the thinking_mode docs (the correct one for v4 models):

"for turns that do perform tool calls, the reasoning_content must be fully passed back to the API in all subsequent requests"

The existing reasoning_content: "" shim from #5527 passes an empty string — which fails for v4 models. The API needs the actual reasoning_content from the previous assistant response.

Where the fix needs to happen

The agent pipeline needs to capture and thread reasoning_content through:

  1. tooled.js: tooledStream() and tooledComplete() — capture reasoning_content from API responses
  2. tooled.js: formatMessagesForTools() — use actual reasoning_content from originalFunctionCall metadata, not empty string
  3. aibitat/index.js: handleAsyncExecution() / handleExecution() — store reasoning_content in originalFunctionCall on each iteration
  4. aibitat/index.js: Chat history — preserve reasoning_content across turns

Reproduction

  1. Configure AnythingLLM with deepseek-v4-pro as the agent model
  2. Run @agent with a multi-tool task (e.g., "search the web for X, then save the results to a file")
  3. On the second tool call iteration, the API returns a 400 error

Related

Note

This is not a duplicate of #5527 — that PR added model detection (isThinkingModel) but the actual reasoning_content passthrough through the agent loop remains broken. This is blocking usage of DeepSeek v4 models with the agent.

Originally created by @gurodrigues on GitHub (May 22, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5683 Originally assigned to: @timothycarambat on GitHub. ## Description The `@agent` loop throws 400 errors when using DeepSeek v4 thinking models (`deepseek-v4-flash`, `deepseek-v4-pro`) in multi-turn tool call flows. The root cause is that `reasoning_content` from previous assistant responses is not preserved through the agent pipeline — it's replaced with an empty string `""`. ## Root Cause DeepSeek has two model families with **opposite** requirements for `reasoning_content`: | Model | Docs | Rule | |---|---|---| | `deepseek-reasoner` | [Reasoning Model](https://api-docs.deepseek.com/guides/reasoning_model) | reasoning_content in input → **400 error** (must strip it) | | `deepseek-v4-flash` / `deepseek-v4-pro` | [Thinking Mode](https://api-docs.deepseek.com/guides/thinking_mode) | reasoning_content in input → **required** after tool calls | From the thinking_mode docs (the correct one for v4 models): > "for turns that do perform tool calls, the reasoning_content must be fully passed back to the API in all subsequent requests" The existing `reasoning_content: ""` shim from #5527 passes an empty string — which fails for v4 models. The API needs the **actual** reasoning_content from the previous assistant response. ## Where the fix needs to happen The agent pipeline needs to capture and thread `reasoning_content` through: 1. **`tooled.js`**: `tooledStream()` and `tooledComplete()` — capture reasoning_content from API responses 2. **`tooled.js`**: `formatMessagesForTools()` — use actual reasoning_content from `originalFunctionCall` metadata, not empty string 3. **`aibitat/index.js`**: `handleAsyncExecution()` / `handleExecution()` — store reasoning_content in `originalFunctionCall` on each iteration 4. **`aibitat/index.js`**: Chat history — preserve reasoning_content across turns ## Reproduction 1. Configure AnythingLLM with `deepseek-v4-pro` as the agent model 2. Run `@agent` with a multi-tool task (e.g., "search the web for X, then save the results to a file") 3. On the second tool call iteration, the API returns a 400 error ## Related - PR with full fix: #5579 (closed based on wrong documentation — see [comment](https://github.com/Mintplex-Labs/anything-llm/pull/5579#issuecomment-4467804126)) - Initial thinking model detection: #5527 - Related: #5570, #5509, #5525, #5523 ## Note This is **not** a duplicate of #5527 — that PR added model detection (`isThinkingModel`) but the actual `reasoning_content` passthrough through the agent loop remains broken. This is blocking usage of DeepSeek v4 models with the agent.
yindo added the needs info / can't replicateinvestigating labels 2026-06-05 14:52:49 -04:00
yindo closed this issue 2026-06-05 14:52:49 -04:00
yindo commented 2026-06-05 14:52:50 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 22, 2026):

This is not the case - on the latest codebase. At least I cannot replicate. Also deepseek-reasoner no longer even appears in the model window because it was dropped/deprecated from their API.

Here is the same query flow in agent mode for both models (see below the messages)

Image Image
<!-- gh-comment-id:4522871763 --> @timothycarambat commented on GitHub (May 22, 2026): This is not the case - on the latest codebase. At least I cannot replicate. Also `deepseek-reasoner` no longer even appears in the model window because it was dropped/deprecated from their API. Here is the same query flow in agent mode for both models (see below the messages) <img width="382" height="700" alt="Image" src="https://github.com/user-attachments/assets/84f39799-e024-4248-a7a8-9299b77dd47f" /> <img width="388" height="627" alt="Image" src="https://github.com/user-attachments/assets/d1ed2976-13a8-4f1f-868b-d2c96ec627af" />
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating needs info / can't replicate
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5233
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?