Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4987] [Bug] OpenAI-compatible mode fails with Claude Thinking models due to unfiltered thinking blocks in chat history #3127

New Issue
Closed
opened 2026-02-22 18:32:44 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:32:44 -05:00
Owner
Copy Link
Copy Source

Originally created by @mouzhi on GitHub (Feb 11, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4987

How are you running AnythingLLM?

Docker (local)

What happened?

🐞 Problem Description
When using Claude Opus 4.5/4.6 Thinking models (e.g., claude-opus-4-6-thinking) via the OpenAI-compatible API mode, AnythingLLM works on the first turn but fails on subsequent messages with the following error from Anthropic:

messages.1.content.0.thinking.signature: Field required

This happens because:
The model returns a response containing a thinking content block (e.g., { "type": "thinking", "content": "...", "signature": "..." }).
AnythingLLM preserves this entire block in the chat history and includes it in the next request.
However, when reconstructing the message for the next turn, the signature field is lost or not regenerated, violating Anthropic’s API schema.
Anthropic rejects the request with a 400 error.
🔍 Note: This issue does not occur when using the native Anthropic provider (because it uses official SDK), but the native provider does not support custom baseURL, making it incompatible with self-hosted proxies like Antigravity Manager.
✅ Expected Behavior
AnythingLLM should either:
Filter out non-text content blocks (like thinking, tool_use) when building chat history for OpenAI-compatible backends, or
Regenerate valid signature fields for thinking blocks if they must be preserved (advanced), or
Provide a configurable option to control this behavior.
Additionally, long-term, it would be ideal to:
Support custom ANTHROPIC_BASE_URL so users can use the native Anthropic provider with self-hosted proxies without relying on OpenAI compatibility.
💡 Proposed Solutions
(Short-term) In OpenAI-compatible mode, sanitize message content before sending to LLM:
// Keep only text blocks; discard thinking/tool_use/etc. const cleanContent = Array.isArray(content) ? content.filter(c => c.type === "text").map(c => c.text).join("\n") : content;
This ensures historical context never contains invalid/unverifiable thinking blocks.
(Long-term) Add environment variable support:
ANTHROPIC_BASE_URL — allow overriding the hardcoded https://api.anthropic.com
LLM_SANITIZE_HISTORY_CONTENT=true|false — toggle content filtering
🧪 Reproduction Steps
Deploy a Claude proxy that supports OpenAI-compatible API (e.g., Antigravity Manager).
Configure AnythingLLM with:
Provider: OpenAI Compatible
Base URL: your proxy /v1
Model: claude-opus-4-6-thinking
Start a chat → first message succeeds.
Send a second message → 400 error with thinking.signature: Field required.
📌 Why This Matters
Users increasingly adopt Thinking-capable models for complex reasoning.
Self-hosted proxies are essential for privacy, cost control, and routing.
Without this fix, multi-turn conversations with Opus Thinking models are broken in the only viable integration path (OpenAI-compatible mode).
Thank you for building an amazing open-source LLM workspace! Let me know if you'd like a PR for the sanitization logic.

Are there known steps to reproduce?

Image

Originally created by @mouzhi on GitHub (Feb 11, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4987 ### How are you running AnythingLLM? Docker (local) ### What happened? 🐞 Problem Description When using Claude Opus 4.5/4.6 Thinking models (e.g., claude-opus-4-6-thinking) via the OpenAI-compatible API mode, AnythingLLM works on the first turn but fails on subsequent messages with the following error from Anthropic: `messages.1.content.0.thinking.signature: Field required ` This happens because: The model returns a response containing a thinking content block (e.g., { "type": "thinking", "content": "...", "signature": "..." }). AnythingLLM preserves this entire block in the chat history and includes it in the next request. However, when reconstructing the message for the next turn, the signature field is lost or not regenerated, violating Anthropic’s API schema. Anthropic rejects the request with a 400 error. 🔍 Note: This issue does not occur when using the native Anthropic provider (because it uses official SDK), but the native provider does not support custom baseURL, making it incompatible with self-hosted proxies like Antigravity Manager. ✅ Expected Behavior AnythingLLM should either: Filter out non-text content blocks (like thinking, tool_use) when building chat history for OpenAI-compatible backends, or Regenerate valid signature fields for thinking blocks if they must be preserved (advanced), or Provide a configurable option to control this behavior. Additionally, long-term, it would be ideal to: Support custom ANTHROPIC_BASE_URL so users can use the native Anthropic provider with self-hosted proxies without relying on OpenAI compatibility. 💡 Proposed Solutions (Short-term) In OpenAI-compatible mode, sanitize message content before sending to LLM: `// Keep only text blocks; discard thinking/tool_use/etc. const cleanContent = Array.isArray(content) ? content.filter(c => c.type === "text").map(c => c.text).join("\n") : content;` This ensures historical context never contains invalid/unverifiable thinking blocks. (Long-term) Add environment variable support: ANTHROPIC_BASE_URL — allow overriding the hardcoded https://api.anthropic.com LLM_SANITIZE_HISTORY_CONTENT=true|false — toggle content filtering 🧪 Reproduction Steps Deploy a Claude proxy that supports OpenAI-compatible API (e.g., Antigravity Manager). Configure AnythingLLM with: Provider: OpenAI Compatible Base URL: your proxy /v1 Model: claude-opus-4-6-thinking Start a chat → first message succeeds. Send a second message → 400 error with thinking.signature: Field required. 📌 Why This Matters Users increasingly adopt Thinking-capable models for complex reasoning. Self-hosted proxies are essential for privacy, cost control, and routing. Without this fix, multi-turn conversations with Opus Thinking models are broken in the only viable integration path (OpenAI-compatible mode). Thank you for building an amazing open-source LLM workspace! Let me know if you'd like a PR for the sanitization logic. ### Are there known steps to reproduce? ![Image](https://github.com/user-attachments/assets/5db830d3-b1e4-4168-bf57-7fc6d0b51092)
yindo closed this issue 2026-02-22 18:32:44 -05:00
yindo commented 2026-02-22 18:32:44 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 11, 2026):

Can you clarify if you are talking about sending requests via the built-in Developer API? If so, then we can investigate this, otherwise we have a dedicated Anthropic LLM provider so handle this since its schema varies from other providers.

Can you verify if you are talking about using the developer API or where ever you get this issue as it is not currently clear.

@timothycarambat commented on GitHub (Feb 11, 2026): Can you clarify if you are talking about sending requests via the built-in Developer API? If so, then we can investigate this, otherwise we have a dedicated Anthropic LLM provider so handle this since its schema varies from other providers. Can you verify if you are talking about using the developer API or where ever you get this issue as it is not currently clear.
yindo changed title from [Bug] OpenAI-compatible mode fails with Claude Thinking models due to unfiltered thinking blocks in chat history to [GH-ISSUE #4987] [Bug] OpenAI-compatible mode fails with Claude Thinking models due to unfiltered thinking blocks in chat history 2026-06-05 14:50:28 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label core-team-only investigating needs info / can't replicate possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#3127
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?