Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4576] [BUG]: Ollama update preventing thinking models from showing thoughts (regular chat and embed widget) #2909

New Issue
Closed
opened 2026-02-22 18:31:47 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:31:47 -05:00
Owner
Copy Link
Copy Source

Originally created by @shatfield4 on GitHub (Oct 24, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4576

Originally assigned to: @angelplusultra on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

  • Ollama version 0.12.6 is having issues with displaying thoughts from thinking capable LLMs
  • Last Ollama version that worked on my machine was 0.11.4 Running MacOS 14 Apple Silicon
  • This prevents the thoughts from showing in the embed chat widget and regular chats inside AnythingLLM
  • Causes glitching in frontend UI where it looks like the message is just blank
  • Suspecting that Ollama changed the response format for LLM thoughts so we will need to parse this in a new way
  • We will need to ensure we handle this in a way that is backward compatible since not all AnythingLLM users will be on the latest version of Ollama

Relevant issues

connect #4535
connect #4561
connect #4564

Are there known steps to reproduce?

No response

Originally created by @shatfield4 on GitHub (Oct 24, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4576 Originally assigned to: @angelplusultra on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? - Ollama version 0.12.6 is having issues with displaying thoughts from thinking capable LLMs - Last Ollama version that worked on my machine was 0.11.4 Running MacOS 14 Apple Silicon - This prevents the thoughts from showing in the embed chat widget and regular chats inside AnythingLLM - Causes glitching in frontend UI where it looks like the message is just blank - Suspecting that Ollama changed the response format for LLM thoughts so we will need to parse this in a new way - We will need to ensure we handle this in a way that is backward compatible since not all AnythingLLM users will be on the latest version of Ollama ### Relevant issues connect #4535 connect #4561 connect #4564 ### Are there known steps to reproduce? _No response_
yindo added the core-team-only label 2026-02-22 18:31:47 -05:00
yindo closed this issue 2026-02-22 18:31:47 -05:00
yindo commented 2026-02-22 18:31:47 -05:00
Author
Owner
Copy Link
Copy Source

@danny0094 commented on GitHub (Oct 28, 2025):

Hey 👋 just wanted to add some context here — this issue likely isn’t caused by AnythingLLM directly, but by a format change in Ollama 0.12.x.

What’s happening
Starting with Ollama v0.12.x, the streaming JSON output seems to have changed:
Older versions ( 0.11.x) sent “thinking” content inline with think tags inside the response field.
Newer versions ( 0.12.x) appear to send thoughts or reasoning traces in a separate JSON key (e"thoughts" or inside "metadata").
AnythingLLM’s parser currently only expects think blocks in response, so the frontend sees an empty message when the “thought” key is missing from the expected place.

Suggested fix
Update the Ollama stream parser to detect both formats:

const thoughts = data.thoughts || data.metadata?.thoughts || extractThinkTags(data.response);
if (thoughts) renderThoughts(thoughts);

That keeps compatibility with both old and new Ollama versions.

Why this matters
Ensures “thinking” LLMs (DeepSeek R1, GPT-OSS, Phi-4) display their reasoning traces correctly again.
Keeps AnythingLLM backward-compatible for users still on 0.11.x.
Hope this helps narrow it down for the team
great catch from the original reporter!

@danny0094 commented on GitHub (Oct 28, 2025): Hey 👋 just wanted to add some context here — this issue likely isn’t caused by AnythingLLM directly, but by a format change in Ollama 0.12.x. What’s happening Starting with Ollama v0.12.x, the streaming JSON output seems to have changed: Older versions ( 0.11.x) sent “thinking” content inline with think tags inside the response field. Newer versions ( 0.12.x) appear to send thoughts or reasoning traces in a separate JSON key (e"thoughts" or inside "metadata"). AnythingLLM’s parser currently only expects think blocks in response, so the frontend sees an empty message when the “thought” key is missing from the expected place. Suggested fix Update the Ollama stream parser to detect both formats: ``` const thoughts = data.thoughts || data.metadata?.thoughts || extractThinkTags(data.response); if (thoughts) renderThoughts(thoughts); ``` That keeps compatibility with both old and new Ollama versions. Why this matters Ensures “thinking” LLMs (DeepSeek R1, GPT-OSS, Phi-4) display their reasoning traces correctly again. Keeps AnythingLLM backward-compatible for users still on 0.11.x. Hope this helps narrow it down for the team great catch from the original reporter!
yindo commented 2026-02-22 18:31:47 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Nov 20, 2025):

@danny0094 Additionally, older pulls of model with pre-baked chat templates like qwen3:1.7b still work fine, but newer ones are broken

@timothycarambat commented on GitHub (Nov 20, 2025): @danny0094 Additionally, older `pull`s of model with pre-baked chat templates like `qwen3:1.7b` still work fine, but newer ones are broken
yindo changed title from [BUG]: Ollama update preventing thinking models from showing thoughts (regular chat and embed widget) to [GH-ISSUE #4576] [BUG]: Ollama update preventing thinking models from showing thoughts (regular chat and embed widget) 2026-06-05 14:49:12 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label core-team-only
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2909
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?