Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3454] [BUG]: LM Studio not invoked with tools list array #2232

New Issue
Closed
opened 2026-02-22 18:28:46 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:28:46 -05:00
Owner
Copy Link
Copy Source

Originally created by @mkuufnzi on GitHub (Mar 13, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3454

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

LM Studio as model provider. Using model trained for too use.
LM Studio logs:
Received request: POST to /v1/chat/completions with body {
"model": "qwen2.5-7b-instruct",
"messages": [
{
"content": "You are a helpful ai assistant who can assist the ... ...ntext available to you and your general knowledge.",
"role": "system"
},
{
"content": "@agent Summarize uploaded financial statements\nNo available document by the name "financial_statements.txt".",
"role": "user"
}
]
}

There is apparently no way to debug why the API call being made to LM Studio is missing the required tool list array. Thus, the model never behaves like an agent. Never sees uploaded docs, etc.

Are there known steps to reproduce?

Set LM Studio as LLM provider in AnythingLLM general settings, select qwen2.5-7b-instruct (Q4_K_M) agentic model. In workspace settings, select the same model. Save settings. Mention an @agent. Expect to see LM Studio logs show the call being made with tools array. Quantization level should not affect how the application makes API calls to the LLM provider.

Originally created by @mkuufnzi on GitHub (Mar 13, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3454 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? LM Studio as model provider. Using model trained for too use. LM Studio logs: Received request: POST to /v1/chat/completions with body { "model": "qwen2.5-7b-instruct", "messages": [ { "content": "You are a helpful ai assistant who can assist the ... <Truncated in logs> ...ntext available to you and your general knowledge.", "role": "system" }, { "content": "@agent Summarize uploaded financial statements\nNo available document by the name \"financial_statements.txt\".", "role": "user" } ] } There is apparently no way to debug why the API call being made to LM Studio is missing the required tool list array. Thus, the model never behaves like an agent. Never sees uploaded docs, etc. ### Are there known steps to reproduce? Set LM Studio as LLM provider in AnythingLLM general settings, select **qwen2.5-7b-instruct (Q4_K_M)** agentic model. In workspace settings, select the same model. Save settings. Mention an @agent. Expect to see LM Studio logs show the call being made with tools array. Quantization level should not affect how the application makes API calls to the LLM provider.
yindo added the possible bug label 2026-02-22 18:28:46 -05:00
yindo closed this issue 2026-02-22 18:28:46 -05:00
yindo commented 2026-02-22 18:28:46 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Mar 13, 2025):

This is just a model issue not an issue with AnythingLLM. The quantization matters because the lower the quantization, the less the model performs so a higher quantization model will perform much better for this.

@shatfield4 commented on GitHub (Mar 13, 2025): This is just a model issue not an issue with AnythingLLM. The quantization matters because the lower the quantization, the less the model performs so a higher quantization model will perform much better for this.
yindo commented 2026-02-22 18:28:46 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 13, 2025):

The tools that are available should be visible in a large system prompt above your content, it is not injected into your content block so it would be above where you showed (and likely truncated). You can open the logs for backend and when you invoke agent you will see a bunch of loaded tools - those are definitively passed to the LLM.

Now for the LLM to call those tools depends on you model quantization. But yes, for them to be included in the prompt is a deterministic process and those logs will prove that.

You can also open the Agent Skills and see which tools are "ON" - those also for sure will be in the prompt window.

@timothycarambat commented on GitHub (Mar 13, 2025): The tools that are available should be visible in a large system prompt above your content, it is not injected into your content block so it would be above where you showed (and likely truncated). You can open [the logs](https://docs.anythingllm.com/installation-desktop/debug#general-debugging) for `backend` and when you invoke agent you will see a bunch of loaded tools - those are definitively passed to the LLM. Now for the LLM to _call_ those tools depends on you model quantization. But yes, for them to be included in the prompt is a deterministic process and those logs will prove that. You can also open the Agent Skills and see which tools are "ON" - those also for sure will be in the prompt window.
yindo changed title from [BUG]: LM Studio not invoked with tools list array to [GH-ISSUE #3454] [BUG]: LM Studio not invoked with tools list array 2026-06-05 14:45:19 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2232
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?