Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1657] [BUG]: Can't communicate with agents #1078

New Issue
Closed
opened 2026-02-22 18:23:01 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:23:01 -05:00
Owner
Copy Link
Copy Source

Originally created by @Ahmad4kTesting on GitHub (Jun 11, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1657

How are you running AnythingLLM?

Docker (local)

What happened?

I can't call or use any agent...

[AgentHandler] Start 09beb93d-fe8f-47be-b7b5-f9cd11e53e76::ollama:llama3:8b-instruct-q8_0
[AgentHandler] Attached websocket plugin to Agent cluster
[AgentHandler] Attached chat-history plugin to Agent cluster
[AgentHandler] Attaching user and default agent to Agent cluster.
[AgentHandler] Attached rag-memory plugin to Agent cluster
[AgentHandler] Attached document-summarizer plugin to Agent cluster
[AgentHandler] Attached web-scraping plugin to Agent cluster
[AgentHandler] Attached web-browsing plugin to Agent cluster
[AgentLLM - llama3:8b-instruct-q8_0] Invalid function tool call: Missing name or arguments in function call..
[AgentLLM - llama3:8b-instruct-q8_0] Will assume chat completion without tool call inputs.

Are there known steps to reproduce?

I'm running lancedb_revert amd64 because of my CPU architecture if that makes any difference!

Originally created by @Ahmad4kTesting on GitHub (Jun 11, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1657 ### How are you running AnythingLLM? Docker (local) ### What happened? I can't call or use any agent... > [AgentHandler] Start 09beb93d-fe8f-47be-b7b5-f9cd11e53e76::ollama:llama3:8b-instruct-q8_0 > [AgentHandler] Attached websocket plugin to Agent cluster > [AgentHandler] Attached chat-history plugin to Agent cluster > [AgentHandler] Attaching user and default agent to Agent cluster. > [AgentHandler] Attached rag-memory plugin to Agent cluster > [AgentHandler] Attached document-summarizer plugin to Agent cluster > [AgentHandler] Attached web-scraping plugin to Agent cluster > [AgentHandler] Attached web-browsing plugin to Agent cluster > [AgentLLM - llama3:8b-instruct-q8_0] Invalid function tool call: Missing name or arguments in function call.. > [AgentLLM - llama3:8b-instruct-q8_0] Will assume chat completion without tool call inputs. ### Are there known steps to reproduce? I'm running `lancedb_revert` amd64 because of my CPU architecture if that makes any difference!
yindo added the possible bug label 2026-02-22 18:23:01 -05:00
yindo closed this issue 2026-02-22 18:23:01 -05:00
yindo commented 2026-02-22 18:23:01 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 12, 2024):

[AgentLLM - llama3:8b-instruct-q8_0] Invalid function tool call: Missing name or arguments in function call..
[AgentLLM - llama3:8b-instruct-q8_0] Will assume chat completion without tool call inputs.

This means that the JSON formed by the LLM output was either raw text or was unparsable as any JSON. This is not a failure of the system and is a failure of the model to coherently follow instructions to generate a valid tool call. This is just the nature of OSS LLMs. We do a lot of work to extract any and all JSON from any response since OSS LLMs tend to put text around their JSON, but we cannot force them to create valid or correct JSON.

Some things that have helped in these situations:

  • be very explicit on which tool to call. Obviously not ideal, but will work
  • /reset the thread and then call the tool (it seems for some models the chat history impacts its ability to make a function call?)
  • Migrate to a larger param model, but use a middle-size quant since you probably cannot run a 30B Q8
@timothycarambat commented on GitHub (Jun 12, 2024): > [AgentLLM - llama3:8b-instruct-q8_0] Invalid function tool call: Missing name or arguments in function call.. > [AgentLLM - llama3:8b-instruct-q8_0] Will assume chat completion without tool call inputs. This means that the JSON formed by the LLM output was either raw text or was unparsable as any JSON. This is not a failure of the system and is a failure of the model to coherently follow instructions to generate a valid tool call. This is just the nature of OSS LLMs. We do _a lot_ of work to extract any and all JSON from any response since OSS LLMs tend to put text around their JSON, but we cannot force them to create valid or correct JSON. Some things that have helped in these situations: - be _very explicit_ on which tool to call. Obviously not ideal, but will work - `/reset` the thread and then call the tool (it seems for some models the chat history impacts its ability to make a function call?) - Migrate to a larger param model, but use a middle-size quant since you probably cannot run a 30B Q8
yindo commented 2026-02-22 18:23:02 -05:00
Author
Owner
Copy Link
Copy Source

@anorod commented on GitHub (Aug 13, 2024):

I have this case, but the response I got from the LLM seemed fine to me:

[2024-08-13 15:17:14.589] [INFO] [lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf] Accumulated 23 tokens: {"name":"web-browsing","query":"asignar rol de administrador en azure"}
[2024-08-13 15:17:14.801] [INFO] [LM STUDIO SERVER] [lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf] Generated prediction: {
"id": "*******",
"object": "chat.completion",
"created": 1723555023,
"model": "lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "{"name":"web-browsing","query":"asignar rol de administrador en azure"}"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 2044,
"completion_tokens": 23,
"total_tokens": 2067
}
}

And I also got:

[backend] info: [AgentLLM - lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF-Meta-Llama-3.1-8B-Instruct-Q8_0.gguf] Will assume chat completion without tool call inputs.
[backend] info: [AgentLLM - lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF-Meta-Llama-3.1-8B-Instruct-Q8_0.gguf] Invalid function tool call: Missing name or arguments in function call..

@anorod commented on GitHub (Aug 13, 2024): I have this case, but the response I got from the LLM seemed fine to me: > [2024-08-13 15:17:14.589] [INFO] [lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf] Accumulated 23 tokens: {"name":"web-browsing","query":"asignar rol de administrador en azure"} [2024-08-13 15:17:14.801] [INFO] [LM STUDIO SERVER] [lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf] Generated prediction: { "id": "*******", "object": "chat.completion", "created": 1723555023, "model": "lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "{\"name\":\"web-browsing\",\"query\":\"asignar rol de administrador en azure\"}" }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 2044, "completion_tokens": 23, "total_tokens": 2067 } } And I also got: > [backend] info: [AgentLLM - lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF-Meta-Llama-3.1-8B-Instruct-Q8_0.gguf] Will assume chat completion without tool call inputs. [backend] info: [AgentLLM - lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF-Meta-Llama-3.1-8B-Instruct-Q8_0.gguf] Invalid function tool call: Missing name or arguments in function call..
yindo commented 2026-02-22 18:23:02 -05:00
Author
Owner
Copy Link
Copy Source

@atomsi commented on GitHub (Jan 13, 2025):

[AgentLLM - llama3:8b-instruct-q8_0] Invalid function tool call: Missing name or arguments in function call..
[AgentLLM - llama3:8b-instruct-q8_0] Will assume chat completion without tool call inputs.

This means that the JSON formed by the LLM output was either raw text or was unparsable as any JSON. This is not a failure of the system and is a failure of the model to coherently follow instructions to generate a valid tool call. This is just the nature of OSS LLMs. We do a lot of work to extract any and all JSON from any response since OSS LLMs tend to put text around their JSON, but we cannot force them to create valid or correct JSON.

Some things that have helped in these situations:

  • be very explicit on which tool to call. Obviously not ideal, but will work
  • /reset the thread and then call the tool (it seems for some models the chat history impacts its ability to make a function call?)
  • Migrate to a larger param model, but use a middle-size quant since you probably cannot run a 30B Q8

2 more questions, could you give me a hand?

  1. which did LLM support JSON format? I tested llama3 and QWen2, they do not work
  2. how to explicit the agent tool? following is what I used:
    @agent use web search: tell me the weather in shanghai today
@atomsi commented on GitHub (Jan 13, 2025): > > [AgentLLM - llama3:8b-instruct-q8_0] Invalid function tool call: Missing name or arguments in function call.. > > [AgentLLM - llama3:8b-instruct-q8_0] Will assume chat completion without tool call inputs. > > This means that the JSON formed by the LLM output was either raw text or was unparsable as any JSON. This is not a failure of the system and is a failure of the model to coherently follow instructions to generate a valid tool call. This is just the nature of OSS LLMs. We do _a lot_ of work to extract any and all JSON from any response since OSS LLMs tend to put text around their JSON, but we cannot force them to create valid or correct JSON. > > Some things that have helped in these situations: > > * be _very explicit_ on which tool to call. Obviously not ideal, but will work > * `/reset` the thread and then call the tool (it seems for some models the chat history impacts its ability to make a function call?) > * Migrate to a larger param model, but use a middle-size quant since you probably cannot run a 30B Q8 2 more questions, could you give me a hand? 1. which did LLM support JSON format? I tested llama3 and QWen2, they do not work 2. how to explicit the agent tool? following is what I used: @agent use web search: tell me the weather in shanghai today
yindo changed title from [BUG]: Can't communicate with agents to [GH-ISSUE #1657] [BUG]: Can't communicate with agents 2026-06-05 14:38:52 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1078
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?