Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5726] [BUG]: No tools available in the chat #5260

New Issue
Closed
opened 2026-06-05 14:52:59 -04:00 by yindo · 5 comments
yindo commented 2026-06-05 14:52:59 -04:00
Owner
Copy Link
Copy Source

Originally created by @jazzroutine on GitHub (May 30, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5726

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

Local LLM provider - llama.cpp (official main branch) - Qwen3.6-27B
in v1.12.1 - everything was good and local LLM endpoint could use local tool, like filesystem, in v.1.13.0 - it got broken.

Local model can no longer list and use tool calls.

Cloud provider like OpenAI have no issues with tool calls.

This is just typical answer. Tried several times, but failed to even read local workspace folder.
Image

File was created in the workspace folder without any issues.
Image

For example, in Hermes I have zero issues with this local model and it can call tool_use without any issues.

Are there known steps to reproduce?

Insrtall

LLM Provider & Model (if applicable)

llama.cpp + Qwen3.6-27B

Embedder Provider & Model (if applicable)

No response

Originally created by @jazzroutine on GitHub (May 30, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5726 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? Local LLM provider - llama.cpp (official main branch) - Qwen3.6-27B in v1.12.1 - everything was good and local LLM endpoint could use local tool, like filesystem, in v.1.13.0 - it got broken. Local model can no longer list and use tool calls. Cloud provider like OpenAI have no issues with tool calls. This is just typical answer. Tried several times, but failed to even read local workspace folder. <img width="1122" height="774" alt="Image" src="https://github.com/user-attachments/assets/99ce2029-abe1-424b-8742-1643f6d19f67" /> File was created in the workspace folder without any issues. <img width="1118" height="341" alt="Image" src="https://github.com/user-attachments/assets/b9551879-008e-469d-8a0f-f886f7d20c66" /> For example, in Hermes I have zero issues with this local model and it can call tool_use without any issues. ### Are there known steps to reproduce? Insrtall ### LLM Provider & Model (if applicable) llama.cpp + Qwen3.6-27B ### Embedder Provider & Model (if applicable) _No response_
yindo added the possible bug label 2026-06-05 14:52:59 -04:00
yindo closed this issue 2026-06-05 14:52:59 -04:00
yindo commented 2026-06-05 14:53:00 -04:00
Author
Owner
Copy Link
Copy Source

@jazzroutine commented on GitHub (May 31, 2026):

Also tried custom OpenAI-compatible provider. The picture is the same. No tools available at all.

<!-- gh-comment-id:4586818594 --> @jazzroutine commented on GitHub (May 31, 2026): Also tried custom OpenAI-compatible provider. The picture is the same. No tools available at all.
yindo commented 2026-06-05 14:53:00 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 1, 2026):

Local LLM provider - llama.cpp (official main branch) - Qwen3.6-27B

This is changing next update to be default on because this keep popping up and at this point if someone is using a model that does not support tools - they need to update to a modern day model.

This ENV setting in the storage/.env file will enable native tool calling and it would fix the issue:
https://docs.anythingllm.com/configuration#native-tool-calling-for-llm-providers

<!-- gh-comment-id:4593723193 --> @timothycarambat commented on GitHub (Jun 1, 2026): > Local LLM provider - llama.cpp (official main branch) - Qwen3.6-27B This is changing next update to be default on because this keep popping up and at this point if someone is using a model that does not support tools - they need to update to a modern day model. This ENV setting in the `storage/.env` file will enable native tool calling and it would fix the issue: https://docs.anythingllm.com/configuration#native-tool-calling-for-llm-providers
yindo commented 2026-06-05 14:53:00 -04:00
Author
Owner
Copy Link
Copy Source

@jazzroutine commented on GitHub (Jun 1, 2026):

What will change next release?

This model have the native tool support and OpenClaw or Hermes have zero issues with it.

<!-- gh-comment-id:4593928875 --> @jazzroutine commented on GitHub (Jun 1, 2026): What will change next release? This model have the native tool support and OpenClaw or Hermes have zero issues with it.
yindo commented 2026-06-05 14:53:00 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 1, 2026):

The model has native tooling, but tools like llama.cpp do not report a capability matrix so there is no way for us to pre-detect if a model can actually call tools. This is because anything could be running behind the llama.cpp endpoint.

The thing that will change is that you wont have to set that ENV to get native tool calling for a provider, we will just assume it does support tools without you having the tell AnythingLLM it does.

<!-- gh-comment-id:4593944929 --> @timothycarambat commented on GitHub (Jun 1, 2026): The **model** has native tooling, but tools like llama.cpp do not report a capability matrix so there is no way for us to pre-detect if a model can actually call tools. This is because anything could be running behind the llama.cpp endpoint. The thing that will change is that you wont have to set that ENV to get native tool calling for a provider, we will just assume it does support tools without you having the tell AnythingLLM it does.
yindo commented 2026-06-05 14:53:00 -04:00
Author
Owner
Copy Link
Copy Source

@jazzroutine commented on GitHub (Jun 1, 2026):

Ok, got it! Thank you for clarification.

I also saw some other Assistants have the ability to "probe" the model before use.

<!-- gh-comment-id:4593969846 --> @jazzroutine commented on GitHub (Jun 1, 2026): Ok, got it! Thank you for clarification. I also saw some other Assistants have the ability to "probe" the model before use.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5260
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?