Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #5677] fix: Various Agent Stream Reasoning Bugs & Presentation Refactor #5524

New Issue
Open
opened 2026-06-05 15:21:37 -04:00 by yindo · 0 comments
yindo commented 2026-06-05 15:21:37 -04:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/5677
Author: @angelplusultra
Created: 5/21/2026
Status: 🔄 Open

Base: feat-reasoning-control ← Head: 5676-bug-non-ollama-agent-providers-do-not-parse-and-present-reasoning-content

📝 Commits (8)

  • 7369489 parse reasoning_content in agent tool-calling helpers
  • bcfad04 hide action bar on reasoning-only agent messages
  • fa66306 fix: route agent reasoning into the collapsible status bubble instead of stacking thought panes
  • 5ef0a44 refactor ollama stream reasoning parsing and emission
  • 1853252 fix: keep agent head pulsing through all status output, not just reasoning
  • 44211e0 fix: keep agent head animation and untooled reasoning in sync with the live stream
  • b713337 simplify frontend
  • 04febb6 fix untooled reasoning stream from vanishing after final response begins streaming

📊 Changes

5 files changed (+250 additions, -43 deletions)

View changed files

📝 frontend/src/components/WorkspaceChat/ChatContainer/ChatHistory/index.jsx (+18 -12)
📝 frontend/src/utils/chat/agent.js (+51 -0)
📝 server/utils/agents/aibitat/providers/helpers/tooled.js (+45 -1)
📝 server/utils/agents/aibitat/providers/helpers/untooled.js (+88 -8)
📝 server/utils/agents/aibitat/providers/ollama.js (+48 -22)

📄 Description

Pull Request Type

  • ✨ feat (New feature)
  • 🐛 fix (Bug fix)
  • ♻️ refactor (Code refactoring without changing behavior)
  • 💄 style (UI style changes)
  • 🔨 chore (Build, CI, maintenance)
  • 📝 docs (Documentation updates)

Relevant Issues

resolves #5676

TLDR

This PR fixes a few things:

  • Surfaces thinking tokens in agent mode for both the tooledStream and untooled stream pathways (any provider emitting reasoning_content), instead of only Ollama.
  • Unifies how reasoning is presented: it now streams live into the collapsible StatusResponse bubble (same place tool activity shows) and is ephemeral — a reloaded conversation shows only tool status + the final answer. This replaces the old <think> thought-pane that stacked a separate pane per step.
  • Ollama stream refactor: Ollama does not use tooled or untooledstream pathways, instead its own bespoke implementation using </think> tags. It has been refactored to match the behavior in tooled and untooled
  • Fixes the pulsing head icon. It used to stop pulsing the moment a tool call or the answer rendered, and on untooled reasoning turns it could get stuck pulsing forever. It now pulses through the whole agent status stream (thinking + tool calls) and settles to static once the final response streams or the agent goes idle.
  • Fixes untooled streams where the thoughts would vanish the instant the final response started streaming — they now persist in the collapsible status bubble, matching tooled.

Description

In agent mode, only ollama surfaced reasoning — and it did so as a standalone <think> thought pane that was persisted into the saved message. Every other provider routing through the shared OpenAI-compatible helpers (tooled.js / untooled.js) dropped reasoning_content entirely, so users got a long pause then only the final answer.

This PR does two things:

  • Surfaces reasoning for all providers routing through the shared helpers (any model emitting reasoning_content).
  • Unifies how reasoning is presented. It now streams live into the collapsible StatusResponse bubble (the same place tool activity shows), framed with "Thinking:", and is no longer persisted — a reloaded conversation shows only tool status + the final answer. This replaces the old <think> thought-pane approach, which stacked a separate pane per step in multi-step runs (per review feedback).

This is a behavior change for Ollama too: its agent reasoning used to render as a thought pane and survive reload, and is now ephemeral.

Scope:

  • Covered: the streaming paths (tooledStream, UnTooled.stream / streamingFunctionCall) plus Ollama.
  • Out of scope (native, non-OpenAI-shaped reasoning streams, follow-up?): anthropic.js, openai.js, gemini.js, bedrock.js.
  • Regular (non-agent) chat is unchanged — it still persists reasoning as <think> panes. The agent/regular asymmetry is noted for a separate discussion.

Implementation notes:

  • Reasoning streams under its own event id (${msgUUID}:reasoning) so the answer's textResponseChunk doesn't merge into the reasoning bubble and render thoughts as visible text.
  • Frontend: the TTS/copy action bar is now gated on a visibleResponseContent() helper so reasoning-only bubbles don't get a copy button that copies an empty string.

Agent head animation (pulse) fix:

The animated agent head is meant to pulse while the agent is actively thinking or invoking a tool, and go static once the final answer is streaming or the agent is idle. It was instead deriving "is the agent working" from the chat tail — pulse while a session is open and the last message is a status bubble. In an interactive @agent session the socket stays open between turns, so any turn that ended on a status bubble (e.g. an untooled reasoning turn that finished on a thought with no trailing answer) left the head pulsing forever.

It now drives the animation off the live stream events instead of the rendered tail:

  • A statusResponse (thinking) or toolCallInvocation event starts the pulse.
  • The final answer (textResponseChunk / fullTextResponse) or the end-of-turn markers (usageMetrics / chatId, which the agent backend emits at the end of every turn even when the answer is empty) settle it to static, as does a session start/end.

This is tracked with a small useIsAgentThinking() hook (mirroring useIsAgentSessionActive) fed by AGENT_THINKING_START / AGENT_THINKING_STOP events dispatched from the websocket handler.

Untooled reasoning persistence fix:

In tooled mode the reasoning bubble persists after the answer lands (collapsible), but in untooled mode the thoughts disappeared the moment the response finished. streamingFunctionCall was streaming both the reasoning and the tentative tool-call/answer preview under the same msgUUID, then calling removeStatusResponse(msgUUID) to clear the preview once the turn resolved — which wiped the thoughts along with it. Reasoning now streams under its own ${msgUUID}:reasoning bubble so the cleanup only removes the preview, matching tooled.js and the existing second-stream path.

Visuals (if applicable)

image

Additional Information

  • Tested across tooled and untooled paths with reasoning models (LM Studio, Koboldcpp, DeepSeek, Lemonade, Moonshot) in both agent and regular chat.

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated (if applicable)
  • I have tested my code functionality
  • Docker build succeeds locally

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/5677 **Author:** [@angelplusultra](https://github.com/angelplusultra) **Created:** 5/21/2026 **Status:** 🔄 Open **Base:** `feat-reasoning-control` ← **Head:** `5676-bug-non-ollama-agent-providers-do-not-parse-and-present-reasoning-content` --- ### 📝 Commits (8) - [`7369489`](https://github.com/Mintplex-Labs/anything-llm/commit/73694891ebdf9b0afa3b9f4062b0df139747b2fd) parse reasoning_content in agent tool-calling helpers - [`bcfad04`](https://github.com/Mintplex-Labs/anything-llm/commit/bcfad04272b176e796fef8e79bfcf2f8c0e368e8) hide action bar on reasoning-only agent messages - [`fa66306`](https://github.com/Mintplex-Labs/anything-llm/commit/fa663064d97929cf94c5d32f84e09c2940154a45) fix: route agent reasoning into the collapsible status bubble instead of stacking thought panes - [`5ef0a44`](https://github.com/Mintplex-Labs/anything-llm/commit/5ef0a44a18624c159f2aa53453a5686767afbfb0) refactor ollama stream reasoning parsing and emission - [`1853252`](https://github.com/Mintplex-Labs/anything-llm/commit/1853252fd37a74aaac41e05ee75fb7eb626a58d9) fix: keep agent head pulsing through all status output, not just reasoning - [`44211e0`](https://github.com/Mintplex-Labs/anything-llm/commit/44211e0225bb2b1f7f816cadd67efd30d39a4dcb) fix: keep agent head animation and untooled reasoning in sync with the live stream - [`b713337`](https://github.com/Mintplex-Labs/anything-llm/commit/b71333776e63db1a9a2097caddc4d9b5d300be7c) simplify frontend - [`04febb6`](https://github.com/Mintplex-Labs/anything-llm/commit/04febb68ba74909d5fe9a6146a149c3e233f0d34) fix untooled reasoning stream from vanishing after final response begins streaming ### 📊 Changes **5 files changed** (+250 additions, -43 deletions) <details> <summary>View changed files</summary> 📝 `frontend/src/components/WorkspaceChat/ChatContainer/ChatHistory/index.jsx` (+18 -12) 📝 `frontend/src/utils/chat/agent.js` (+51 -0) 📝 `server/utils/agents/aibitat/providers/helpers/tooled.js` (+45 -1) 📝 `server/utils/agents/aibitat/providers/helpers/untooled.js` (+88 -8) 📝 `server/utils/agents/aibitat/providers/ollama.js` (+48 -22) </details> ### 📄 Description ### Pull Request Type - [ ] ✨ feat (New feature) - [x] 🐛 fix (Bug fix) - [ ] ♻️ refactor (Code refactoring without changing behavior) - [ ] 💄 style (UI style changes) - [ ] 🔨 chore (Build, CI, maintenance) - [ ] 📝 docs (Documentation updates) ### Relevant Issues resolves #5676 ### TLDR This PR fixes a few things: - Surfaces thinking tokens in agent mode for both the `tooledStream` and `untooled` stream pathways (any provider emitting `reasoning_content`), instead of only Ollama. - Unifies how reasoning is presented: it now streams live into the collapsible `StatusResponse` bubble (same place tool activity shows) and is ephemeral — a reloaded conversation shows only tool status + the final answer. This replaces the old `<think>` thought-pane that stacked a separate pane per step. - Ollama stream refactor: Ollama does not use `tooled` or `untooled`stream pathways, instead its own bespoke implementation using `</think>` tags. It has been refactored to match the behavior in `tooled` and `untooled` - Fixes the pulsing head icon. It used to stop pulsing the moment a tool call or the answer rendered, and on untooled reasoning turns it could get stuck pulsing forever. It now pulses through the whole agent status stream (thinking + tool calls) and settles to static once the final response streams or the agent goes idle. - Fixes untooled streams where the thoughts would vanish the instant the final response started streaming — they now persist in the collapsible status bubble, matching tooled. ### Description In agent mode, only `ollama` surfaced reasoning — and it did so as a standalone `<think>` thought pane that was persisted into the saved message. Every other provider routing through the shared OpenAI-compatible helpers (`tooled.js` / `untooled.js`) dropped `reasoning_content` entirely, so users got a long pause then only the final answer. This PR does two things: - **Surfaces reasoning for all providers** routing through the shared helpers (any model emitting `reasoning_content`). - **Unifies how reasoning is presented.** It now streams live into the collapsible `StatusResponse` bubble (the same place tool activity shows), framed with `"Thinking:"`, and is no longer persisted — a reloaded conversation shows only tool status + the final answer. This replaces the old `<think>` thought-pane approach, which stacked a separate pane per step in multi-step runs (per review feedback). This is a behavior change for Ollama too: its agent reasoning used to render as a thought pane and survive reload, and is now ephemeral. **Scope:** - Covered: the streaming paths (`tooledStream`, `UnTooled.stream` / `streamingFunctionCall`) plus Ollama. - Out of scope (native, non-OpenAI-shaped reasoning streams, follow-up?): `anthropic.js`, `openai.js`, `gemini.js`, `bedrock.js`. - Regular (non-agent) chat is unchanged — it still persists reasoning as `<think>` panes. The agent/regular asymmetry is noted for a separate discussion. **Implementation notes:** - Reasoning streams under its own event id (`${msgUUID}:reasoning`) so the answer's `textResponseChunk` doesn't merge into the reasoning bubble and render thoughts as visible text. - Frontend: the TTS/copy action bar is now gated on a `visibleResponseContent()` helper so reasoning-only bubbles don't get a copy button that copies an empty string. **Agent head animation (pulse) fix:** The animated agent head is meant to pulse while the agent is actively thinking or invoking a tool, and go static once the final answer is streaming or the agent is idle. It was instead deriving "is the agent working" from the chat tail — pulse while a session is open and the last message is a status bubble. In an interactive `@agent` session the socket stays open between turns, so any turn that ended on a status bubble (e.g. an untooled reasoning turn that finished on a thought with no trailing answer) left the head pulsing forever. It now drives the animation off the live stream events instead of the rendered tail: - A `statusResponse` (thinking) or `toolCallInvocation` event starts the pulse. - The final answer (`textResponseChunk` / `fullTextResponse`) or the end-of-turn markers (`usageMetrics` / `chatId`, which the agent backend emits at the end of every turn even when the answer is empty) settle it to static, as does a session start/end. This is tracked with a small `useIsAgentThinking()` hook (mirroring `useIsAgentSessionActive`) fed by `AGENT_THINKING_START` / `AGENT_THINKING_STOP` events dispatched from the websocket handler. **Untooled reasoning persistence fix:** In tooled mode the reasoning bubble persists after the answer lands (collapsible), but in untooled mode the thoughts disappeared the moment the response finished. `streamingFunctionCall` was streaming both the reasoning and the tentative tool-call/answer preview under the same `msgUUID`, then calling `removeStatusResponse(msgUUID)` to clear the preview once the turn resolved — which wiped the thoughts along with it. Reasoning now streams under its own `${msgUUID}:reasoning` bubble so the cleanup only removes the preview, matching `tooled.js` and the existing second-stream path. ### Visuals (if applicable) <img width="1144" height="429" alt="image" src="https://github.com/user-attachments/assets/ff9f897f-62c5-4725-b7e1-cbc122b51002" /> ### Additional Information - Tested across tooled and untooled paths with reasoning models (LM Studio, Koboldcpp, DeepSeek, Lemonade, Moonshot) in both agent and regular chat. ### Developer Validations - [x] I ran `yarn lint` from the root of the repo & committed changes - [x] Relevant documentation has been updated (if applicable) - [x] I have tested my code functionality - [x] Docker build succeeds locally --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-06-05 15:21:37 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5524
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?