Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5123] [BUG]: Custom skill agent call loops many times before finishing thinking #4932

New Issue
Closed
opened 2026-06-05 14:50:57 -04:00 by yindo · 3 comments
yindo commented 2026-06-05 14:50:57 -04:00
Owner
Copy Link
Copy Source

Originally created by @tec-tea on GitHub (Mar 3, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5123

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

I've developed a custom skill and works well, but the agent calls the tool many times before finishing thinking, even though it receives a string as a response.
This behavior makes custom skill not reliable, as it consumes too many tokens.
I've implemented many solutions and none of them works. I've also tried the ones described here: 4901

Are there known steps to reproduce?

When implementing a simple handler that just returns a string:

handler: async function ({ content, topic, version }) {
    await this.wait(10000); // simulate call wating
    return "Tool call was succesful";
}

I've also tried this one:

handler: async function ({ content, topic, version }) {
    await this.wait(10000); // simulate call wating
    return JSON.stringify({"message": "Tool call was succesful", "status": "COMPLETED", "instruction": "@agent This request is finished, reply with the previous message /exit "});
}

It also happens in other custom skills that you can download, like this one: https://hub.anythingllm.com/i/agent-skill/Y1NbU3EVxK71BPhaKwCe

Originally created by @tec-tea on GitHub (Mar 3, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5123 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? I've developed a custom skill and works well, but the agent calls the tool many times before finishing thinking, even though it receives a string as a response. This behavior makes custom skill not reliable, as it consumes too many tokens. I've implemented many solutions and none of them works. I've also tried the ones described here: [4901](https://github.com/Mintplex-Labs/anything-llm/issues/4901) ### Are there known steps to reproduce? When implementing a simple handler that just returns a string: ``` handler: async function ({ content, topic, version }) { await this.wait(10000); // simulate call wating return "Tool call was succesful"; } ``` I've also tried this one: ``` handler: async function ({ content, topic, version }) { await this.wait(10000); // simulate call wating return JSON.stringify({"message": "Tool call was succesful", "status": "COMPLETED", "instruction": "@agent This request is finished, reply with the previous message /exit "}); } ``` It also happens in other custom skills that you can download, like this one: https://hub.anythingllm.com/i/agent-skill/Y1NbU3EVxK71BPhaKwCe
yindo added the possible bug label 2026-06-05 14:50:57 -04:00
yindo closed this issue 2026-06-05 14:50:57 -04:00
yindo commented 2026-06-05 14:50:58 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 3, 2026):

This behavior makes custom skill not reliable, as it consumes too many tokens.

This is a model problem, not a skill problem. When possible we always reply with some string or confirmation. If the model continues to call it that is a model/quant issue. Small local models have this issue even when you provide data back - this is especially true if not using the native tool calling functionality merged in 1.11.1+

<!-- gh-comment-id:3993808118 --> @timothycarambat commented on GitHub (Mar 3, 2026): > This behavior makes custom skill not reliable, as it consumes too many tokens. This is a model problem, not a skill problem. When possible we always reply with some string or confirmation. If the model continues to call it that is a model/quant issue. Small local models have this issue even when you provide data back - this is **especially** true if not using the native tool calling functionality merged in 1.11.1+
yindo commented 2026-06-05 14:50:58 -04:00
Author
Owner
Copy Link
Copy Source

@tec-tea commented on GitHub (Mar 11, 2026):

I can confirm that implementing the native tool calling functionality it works fine. It avoids tool calling loops (version tested 1.11.1).

<!-- gh-comment-id:4041039178 --> @tec-tea commented on GitHub (Mar 11, 2026): I can confirm that implementing the native tool calling functionality it works fine. It avoids tool calling loops (version tested 1.11.1).
yindo commented 2026-06-05 14:50:58 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 11, 2026):

Yeah the Untooled solution was basically a bridge during the transition from the first tool-calling models when everyone was still on llama2 and models whose chat templates are just missing the function/tool calling. We still have it because lots of people still need it or some providers dont have it for specific models :/

Using native is 100% the way to go to prevent infinite tool call loops since the chat template can handle the call/responses properly

<!-- gh-comment-id:4041939819 --> @timothycarambat commented on GitHub (Mar 11, 2026): Yeah the Untooled solution was basically a bridge during the transition from the first tool-calling models when everyone was still on llama2 and models whose chat templates are just missing the function/tool calling. We still have it because lots of people still need it or some providers dont have it for specific models :/ Using native is 100% the way to go to prevent infinite tool call loops since the chat template can handle the call/responses properly
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#4932
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?