Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #5584] [CLOSED] fix: parse reasoning output from LM Studio reasoning models #5489

New Issue
Closed
opened 2026-06-05 15:21:31 -04:00 by yindo · 0 comments
yindo commented 2026-06-05 15:21:31 -04:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/5584
Author: @angelplusultra
Created: 5/6/2026
Status: ❌ Closed

Base: master ← Head: 5583-bug-lm-studio-provider-does-not-present-reasoning-output

📝 Commits (3)

  • 42f5e2b implement bespoke handleStream logic with reasoning output parsing
  • 8d594af add reasoning output parsing to non-streaming path
  • 924f203 close on mid-reasoning finish | guard undefined content in non-stream parse | preserve todo comment

📊 Changes

1 file changed (+172 additions, -3 deletions)

View changed files

📝 server/utils/AiProviders/lmStudio/index.js (+172 -3)

📄 Description

Pull Request Type

  • ✨ feat (New feature)
  • 🐛 fix (Bug fix)
  • ♻️ refactor (Code refactoring without changing behavior)
  • 💄 style (UI style changes)
  • 🔨 chore (Build, CI, maintenance)
  • 📝 docs (Documentation updates)

Relevant Issues

resolves #5583

Description

The LM Studio provider was using the generic handleDefaultStreamResponseV2 stream handler, which only reads choices[0].delta.content. LM Studio's OpenAI-compatible endpoint emits reasoning tokens for reasoning models in a separate reasoning_content field (same shape as DeepSeek), so those tokens were silently dropped on both the streaming and non-streaming paths and the UI never showed the model's thinking output.

Streaming (handleStream) — replaced the default handler with a bespoke implementation modeled after the existing DeepSeek/Foundry handlers:

  • Reads both delta.content and delta.reasoning_content per chunk.
  • On the first reasoning chunk, prepends a <think> tag and streams subsequent reasoning tokens through verbatim.
  • On the first content token after a reasoning run, emits the closing </think> tag, flushes the buffered reasoning into fullText, and resumes normal content streaming.
  • Preserves the existing usage-metric handling, abort handling, and finish_reason termination logic from the default handler.

Non-streaming (getChatCompletion) — added a #parseReasoningFromResponse helper that, when message.reasoning_content is present, prepends <think>{reasoning}</think> to the response content before returning it as textResponse. Same shape the streaming path produces, so downstream rendering is consistent across both modes.

Visuals (if applicable)

Additional Information

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated (if applicable)
  • I have tested my code functionality
  • Docker build succeeds locally

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/5584 **Author:** [@angelplusultra](https://github.com/angelplusultra) **Created:** 5/6/2026 **Status:** ❌ Closed **Base:** `master` ← **Head:** `5583-bug-lm-studio-provider-does-not-present-reasoning-output` --- ### 📝 Commits (3) - [`42f5e2b`](https://github.com/Mintplex-Labs/anything-llm/commit/42f5e2b64a9a3f512a8c1708d7d3d9fdabf6c051) implement bespoke handleStream logic with reasoning output parsing - [`8d594af`](https://github.com/Mintplex-Labs/anything-llm/commit/8d594af80f2033f28dcf972350b3a1f258da6b5e) add reasoning output parsing to non-streaming path - [`924f203`](https://github.com/Mintplex-Labs/anything-llm/commit/924f2036f24898a42ad24a04caee696cb52418c3) close <think> on mid-reasoning finish | guard undefined content in non-stream parse | preserve todo comment ### 📊 Changes **1 file changed** (+172 additions, -3 deletions) <details> <summary>View changed files</summary> 📝 `server/utils/AiProviders/lmStudio/index.js` (+172 -3) </details> ### 📄 Description ### Pull Request Type <!-- For change type, change [ ] to [x]. --> - [ ] ✨ feat (New feature) - [x] 🐛 fix (Bug fix) - [ ] ♻️ refactor (Code refactoring without changing behavior) - [ ] 💄 style (UI style changes) - [ ] 🔨 chore (Build, CI, maintenance) - [ ] 📝 docs (Documentation updates) ### Relevant Issues <!-- Use "resolves #xxx" to auto resolve on merge. Otherwise, please use "connect #xxx" --> resolves #5583 ### Description The LM Studio provider was using the generic `handleDefaultStreamResponseV2` stream handler, which only reads `choices[0].delta.content`. LM Studio's OpenAI-compatible endpoint emits reasoning tokens for reasoning models in a separate `reasoning_content` field (same shape as DeepSeek), so those tokens were silently dropped on both the streaming and non-streaming paths and the UI never showed the model's thinking output. Streaming (`handleStream`) — replaced the default handler with a bespoke implementation modeled after the existing DeepSeek/Foundry handlers: - Reads both `delta.content` and `delta.reasoning_content` per chunk. - On the first reasoning chunk, prepends a `<think>` tag and streams subsequent reasoning tokens through verbatim. - On the first content token after a reasoning run, emits the closing `</think>` tag, flushes the buffered reasoning into `fullText`, and resumes normal content streaming. - Preserves the existing usage-metric handling, abort handling, and `finish_reason` termination logic from the default handler. Non-streaming (`getChatCompletion`) — added a `#parseReasoningFromResponse` helper that, when `message.reasoning_content` is present, prepends `<think>{reasoning}</think>` to the response content before returning it as `textResponse`. Same shape the streaming path produces, so downstream rendering is consistent across both modes. ### Visuals (if applicable) <!-- Add screenshots or screen recordings to demonstrate the changes, especially for UI updates. --> ### Additional Information <!-- Add any other context about the Pull Request here that was not captured above. --> ### Developer Validations <!-- All of the applicable items should be checked. --> - [x] I ran `yarn lint` from the root of the repo & committed changes - [x] Relevant documentation has been updated (if applicable) - [x] I have tested my code functionality - [x] Docker build succeeds locally --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-06-05 15:21:31 -04:00
yindo closed this issue 2026-06-05 15:21:31 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5489
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?