Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4741] [BUG]: Workspace LLM always routes to OpenAI even when LM Studio is selected (macOS, v1.9.0 and v1.8.5-r2) #2995

New Issue
Closed
opened 2026-02-22 18:32:11 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:32:11 -05:00
Owner
Copy Link
Copy Source

Originally created by @mhelms1961 on GitHub (Dec 9, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4741

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

Title:

Workspace LLM always routes to OpenAI even when LM Studio is selected (macOS, v1.9.0 and v1.8.5-r2)

Description of the Issue

On macOS, AnythingLLM is consistently routing all workspace chat requests to OpenAI instead of LM Studio, even when:

  • LM Studio is the only configured LLM provider
  • Workspace Chat LLM provider = LM Studio
  • Workspace Agent LLM provider = LM Studio
  • The LM Studio server is running at http://127.0.0.1:1234
  • The OpenAI API key is intentionally invalid (fake) to prevent fallback
  • OpenAI is not intended to be used at all

This behavior happens on:

  • AnythingLLM 1.9.0
  • AnythingLLM 1.8.5-r2 (after clean downgrade)

In all cases, the UI shows the correct LM Studio configuration, but the runtime silently sends requests to OpenAI instead of LM Studio.

Environment

  • macOS 15.x (Apple Silicon)
  • Hardware: Mac Studio M4 Max (64GB RAM)
  • LM Studio version: 0.3.33, local server enabled
  • AnythingLLM versions tested:
    • 1.9.0
    • 1.8.5-r2

Expected Behavior

When LM Studio is selected, AnythingLLM should:

  • Route ALL chat requests to http://127.0.0.1:1234/v1/chat/completions
  • Use the selected LM Studio model (e.g., meta-llama-3.1-70b-instruct)
  • Show LM Studio GPU activity
  • Never hit OpenAI unless manually chosen

Actual Behavior

AnythingLLM ALWAYS routes to OpenAI:

Example response inside the workspace:

Symptoms:

  • LM Studio receives zero requests
  • GPU remains idle
  • Requests complete in 1–3 seconds (impossible for a 70B model)
  • Even with a fake OpenAI API key, it still claims to use OpenAI models
  • LM Studio is fully ignored even though selected everywhere

Reproduction Steps

  1. Install AnythingLLM (1.9.0 or 1.8.5-r2) on macOS.
  2. Install LM Studio, enable local server → http://127.0.0.1:1234
  3. Load a GGUF model (e.g., Llama 3.1 70B) and confirm inference works.
  4. In AnythingLLM:
    • Instance Settings → LLM Provider = LM Studio
    • Workspace Settings → Chat Settings = LM Studio
    • Agent Configuration → LLM Provider = LM Studio
  5. Replace OpenAI key with a fake one such as sk-fake_key_12345
  6. Restart both apps.
  7. Ask the workspace:

Result:

AnythingLLM returns something like:
LM Studio remains completely unused.

Why This Is a Problem

  • Users cannot use local models at all
  • All chats are silently routed to OpenAI
  • Privacy expectations are broken
  • GPU acceleration on local hardware cannot be used
  • LM Studio compatibility appears effectively broken on macOS

What We Believe Might Be Happening

Potential root causes include:

  • Workspace fallback logic overriding provider selection
  • Provider priority not respecting LM Studio
  • A cached provider path being used
  • A macOS-specific initialization bug in the runtime

Requested Developer Assistance

Please advise on:

  1. Whether workspace-level LLM selection is being honored correctly.
  2. Why the router falls back to OpenAI even when disabled.
  3. How to force LM Studio usage without fallback.
  4. Whether this is a known issue in macOS builds.
  5. Any logs or debugging steps needed from my machine.

Thank you

I’m happy to provide logs, testing, or run special builds.

Are there known steps to reproduce?

No response

Originally created by @mhelms1961 on GitHub (Dec 9, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4741 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? ## Title: Workspace LLM always routes to OpenAI even when LM Studio is selected (macOS, v1.9.0 and v1.8.5-r2) ## Description of the Issue On macOS, AnythingLLM is consistently routing all workspace chat requests to **OpenAI** instead of **LM Studio**, even when: - LM Studio is the **only configured LLM provider** - Workspace Chat LLM provider = **LM Studio** - Workspace Agent LLM provider = **LM Studio** - The LM Studio server is running at `http://127.0.0.1:1234` - The OpenAI API key is intentionally invalid (fake) to prevent fallback - OpenAI is not intended to be used at all This behavior happens on: - AnythingLLM **1.9.0** - AnythingLLM **1.8.5-r2** (after clean downgrade) In all cases, the UI shows the correct LM Studio configuration, but the runtime silently sends requests to OpenAI instead of LM Studio. ## Environment - macOS 15.x (Apple Silicon) - Hardware: **Mac Studio M4 Max (64GB RAM)** - LM Studio version: **0.3.33**, local server enabled - AnythingLLM versions tested: - **1.9.0** - **1.8.5-r2** ## Expected Behavior When LM Studio is selected, AnythingLLM should: - Route ALL chat requests to `http://127.0.0.1:1234/v1/chat/completions` - Use the selected LM Studio model (e.g., `meta-llama-3.1-70b-instruct`) - Show LM Studio GPU activity - Never hit OpenAI unless manually chosen ## Actual Behavior AnythingLLM ALWAYS routes to OpenAI: Example response inside the workspace: Symptoms: - LM Studio receives **zero** requests - GPU remains idle - Requests complete in 1–3 seconds (impossible for a 70B model) - Even with a **fake OpenAI API key**, it still claims to use OpenAI models - LM Studio is fully ignored even though selected everywhere ## Reproduction Steps 1. Install AnythingLLM (1.9.0 or 1.8.5-r2) on macOS. 2. Install LM Studio, enable local server → `http://127.0.0.1:1234` 3. Load a GGUF model (e.g., Llama 3.1 70B) and confirm inference works. 4. In AnythingLLM: - Instance Settings → LLM Provider = **LM Studio** - Workspace Settings → Chat Settings = **LM Studio** - Agent Configuration → LLM Provider = **LM Studio** 5. Replace OpenAI key with a fake one such as `sk-fake_key_12345` 6. Restart both apps. 7. Ask the workspace: ### Result: AnythingLLM returns something like: LM Studio remains completely unused. ## Why This Is a Problem - Users cannot use local models at all - All chats are silently routed to OpenAI - Privacy expectations are broken - GPU acceleration on local hardware cannot be used - LM Studio compatibility appears effectively broken on macOS ## What We Believe Might Be Happening Potential root causes include: - Workspace fallback logic overriding provider selection - Provider priority not respecting LM Studio - A cached provider path being used - A macOS-specific initialization bug in the runtime ## Requested Developer Assistance Please advise on: 1. Whether workspace-level LLM selection is being honored correctly. 2. Why the router falls back to OpenAI even when disabled. 3. How to *force* LM Studio usage without fallback. 4. Whether this is a known issue in macOS builds. 5. Any logs or debugging steps needed from my machine. ## Thank you I’m happy to provide logs, testing, or run special builds. ### Are there known steps to reproduce? _No response_
yindo added the possible bugneeds info / can't replicate labels 2026-02-22 18:32:11 -05:00
yindo closed this issue 2026-02-22 18:32:11 -05:00
yindo commented 2026-02-22 18:32:11 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 9, 2025):

This is def a setup/config issue. That being said can you validate the following:

  • What does the Settings > AI Providers > LLM provider say?
  • Clicking on the "Gear" icon on a workspace. What does the LLM preference say for the "Chat Setting" Workspace provider
  • What does the LLM preference say on "Agent Configuration" Page.

The hierarchy for execution of a chat is:

  • Check for workspace-specific LLM. If @agent call, check if agent-specific LLM is set
  • If not found, look at the system LLM provider

We keep going up the hierarchy until something is set. Somewhere in that flow you have OpenAI set.

You outlined the following:

LM Studio is the only configured LLM provider
Workspace Chat LLM provider = LM Studio
Workspace Agent LLM provider = LM Studio

So if the above are 100% for sure LM Studio, then the only other place that OpenAI could be in the flow is if somehow you have it set as your embedding provider and when you run a chat, it fails to embed the query to do RAG. The default is on-device using CPU, so you would have set that intentionally (or accidentally)

I am also on a similar M4 Pro. I set OpenAI with a bad key as the system LLM. Then set LM Studio on standard host/port for workspace chat, and agent chat. I send a chat or agent chat and my request goes to LM Studio

@timothycarambat commented on GitHub (Dec 9, 2025): This is def a setup/config issue. That being said can you validate the following: - What does the Settings > AI Providers > LLM provider say? - Clicking on the "Gear" icon on a workspace. What does the LLM preference say for the "Chat Setting" Workspace provider - What does the LLM preference say on "Agent Configuration" Page. The hierarchy for execution of a chat is: - Check for workspace-specific LLM. If `@agent` call, check if agent-specific LLM is set - If not found, look at the system LLM provider We keep going up the hierarchy until something is set. Somewhere in that flow you have OpenAI set. You outlined the following: > LM Studio is the only configured LLM provider Workspace Chat LLM provider = LM Studio Workspace Agent LLM provider = LM Studio So if the above are 100% for sure LM Studio, then the only other place that OpenAI could be in the flow is if somehow you have it set as your embedding provider and when you run a chat, it fails to embed the query to do RAG. The default is on-device using CPU, so you would have set that intentionally (or accidentally) --- I am also on a similar M4 Pro. I set OpenAI with a bad key as the system LLM. Then set LM Studio on standard host/port for workspace chat, and agent chat. I send a chat or agent chat and my request goes to LM Studio
yindo changed title from [BUG]: Workspace LLM always routes to OpenAI even when LM Studio is selected (macOS, v1.9.0 and v1.8.5-r2) to [GH-ISSUE #4741] [BUG]: Workspace LLM always routes to OpenAI even when LM Studio is selected (macOS, v1.9.0 and v1.8.5-r2) 2026-06-05 14:49:43 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label needs info / can't replicate possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2995
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?