Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4733] [BUG]: Web search tool calling does not work #2990

New Issue
Closed
opened 2026-02-22 18:32:10 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:32:10 -05:00
Owner
Copy Link
Copy Source

Originally created by @pcgeek86 on GitHub (Dec 6, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4733

How are you running AnythingLLM?

MacOS App

What happened?

I've just started using AnythingLLM and I cannot get the Web Search tool calling to work at all. It simply refuses to invoke any tools or acknowledge that tools are available.

Hardware / Software Stack

  • MacBook Pro M4 Max 64 GB
  • MacOS Tahoe 26.1
  • Ollama
  • LM Studio
  • AnythingLLM

I've tried enabling DuckDuckGo web search at first, and that did not work at all with the following prompts:

Use web search to find the latest version of the ratatui crate for Rust
Use DuckDuckGo to find the latest version of the ratatui crate for Rust
Use DuckDuckGo web search to find the latest version of the ratatui crate for Rust
Use web search tool to find the latest version of the ratatui crate for Rust
Use DuckDuckGo web search tool to find the latest version of the ratatui crate for Rust

I tried these prompts with these models:

  • IBM Granite4
  • gpt-oss:20b
  • Mistral AI ministral-3-14b-reasoning

This happens with both Ollama and LM Studio.

I tried changing the search provider to Tavily, and provided an API key. There was no improvement in the results.

The web search (at least) seems to be completely broken.

Next I am going to try adding custom MCP servers .... we'll see if that works.

Pydantic AI works fine with tool calls

As a test, I also created a new Python project with Pydantic AI, and used their sample project. It works perfectly fine with multiple models, like granite4 and gpt-oss:20b, on my first and repeated attempts.

https://ai.pydantic.dev/tools/

import random

from pydantic_ai import Agent, RunContext
from pydantic_ai.providers.ollama import OllamaProvider
from pydantic_ai.models.openai import OpenAIChatModel


ollama_model = OpenAIChatModel(
    # model_name='gpt-oss:20b',
    model_name='granite4',
    provider=OllamaProvider(base_url='http://localhost:11434/v1'),  
)

agent = Agent(
    ollama_model,
    deps_type=str,  
    system_prompt=(
        "You're a dice game, you should roll the die and see if the number "
        "you get back matches the user's guess. If so, tell them they're a winner. "
        "Use the player's name in the response."
    ),
)


@agent.tool_plain  
def roll_dice() -> str:
    """Roll a six-sided die and return the result."""
    return str(random.randint(1, 6))


@agent.tool  
def get_player_name(ctx: RunContext[str]) -> str:
    """Get the player's name."""
    return ctx.deps


dice_result = agent.run_sync('My guess is 4', deps='Anne')  
print(dice_result.output)

And yes, I did see this document while I was working through this. It doesn't make sense to me why AnythingLLM is having trouble tool calling, but a framework like Pydantic AI works right out of the box.

https://docs.anythingllm.com/agent-not-using-tools

Originally created by @pcgeek86 on GitHub (Dec 6, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4733 ### How are you running AnythingLLM? MacOS App ### What happened? I've just started using AnythingLLM and I cannot get the Web Search tool calling to work at all. It simply refuses to invoke any tools or acknowledge that tools are available. Hardware / Software Stack * MacBook Pro M4 Max 64 GB * MacOS Tahoe 26.1 * Ollama * LM Studio * AnythingLLM I've tried enabling DuckDuckGo web search at first, and that did not work at all with the following prompts: > Use web search to find the latest version of the ratatui crate for Rust > Use DuckDuckGo to find the latest version of the ratatui crate for Rust > Use DuckDuckGo web search to find the latest version of the ratatui crate for Rust > Use web search tool to find the latest version of the ratatui crate for Rust > Use DuckDuckGo web search tool to find the latest version of the ratatui crate for Rust I tried these prompts with these models: - IBM Granite4 - gpt-oss:20b - Mistral AI ministral-3-14b-reasoning This happens with both Ollama and LM Studio. I tried changing the search provider to Tavily, and provided an API key. There was no improvement in the results. The web search (at least) seems to be completely broken. Next I am going to try adding custom MCP servers .... we'll see if that works. ### Pydantic AI works fine with tool calls As a test, I also created a new Python project with Pydantic AI, and used their sample project. It works perfectly fine with multiple models, like `granite4` and `gpt-oss:20b`, on my first and repeated attempts. https://ai.pydantic.dev/tools/ ``` import random from pydantic_ai import Agent, RunContext from pydantic_ai.providers.ollama import OllamaProvider from pydantic_ai.models.openai import OpenAIChatModel ollama_model = OpenAIChatModel( # model_name='gpt-oss:20b', model_name='granite4', provider=OllamaProvider(base_url='http://localhost:11434/v1'), ) agent = Agent( ollama_model, deps_type=str, system_prompt=( "You're a dice game, you should roll the die and see if the number " "you get back matches the user's guess. If so, tell them they're a winner. " "Use the player's name in the response." ), ) @agent.tool_plain def roll_dice() -> str: """Roll a six-sided die and return the result.""" return str(random.randint(1, 6)) @agent.tool def get_player_name(ctx: RunContext[str]) -> str: """Get the player's name.""" return ctx.deps dice_result = agent.run_sync('My guess is 4', deps='Anne') print(dice_result.output) ``` And yes, I did see this document while I was working through this. It doesn't make sense to me why AnythingLLM is having trouble tool calling, but a framework like Pydantic AI works right out of the box. https://docs.anythingllm.com/agent-not-using-tools
yindo added the possible bug label 2026-02-22 18:32:10 -05:00
yindo closed this issue 2026-02-22 18:32:10 -05:00
yindo commented 2026-02-22 18:32:10 -05:00
Author
Owner
Copy Link
Copy Source

@Sinkingdev commented on GitHub (Dec 7, 2025):

You’re right to be confused; it had me going for a spin when I first got here. Other LLM frameworks expose tools directly in the chat, so they’re always available and burning tokens. The key difference between AnythingLLM and frameworks like Pydantic AI, tools act as a separate entity working alongside your chat.

The crucial detail
Tools have to be enabled in the settings > Agent Skills
Tools in AnythingLLM are only available when you explicitly invoke the @agent prefix.
This is intentional; it keeps tools out of your normal chat to preserve tokens.

Try these prompts instead:

@agent Use web search to find the latest version of the ratatui crate for Rust
@agent Use DuckDuckGo to find the latest version of the ratatui crate for Rust
@agent Use web search tool to find the latest version of the ratatui crate for Rust

Or just simply "@agent test"

How to know it's working:

  • Your chat log will show something like this:
    Agent @agent invoked. Swapping over to agent chat. Type /exit to exit..
  • The agent will show a "thought" chain before responding (this indicates a tool was actually called)
  • You'll see Agent session completed when done

If it STILL doesn't work after adding @agent:

Then you may be hitting the model quality issue described in the docs. The models you're testing (Granite4, gpt-oss:20b, Mistral 3-14b) are smaller/quantized models that struggle with JSON generation for tool calls. Try:

  1. Reset chat history: /reset then retry
  2. Disable unused tools: In Agent Skills config, turn off tools you don't need—this reduces prompt bloat
  3. Test with a stronger model: If possible, try a higher-quantization version or a cloud model (GPT-4, Claude) to confirm tools work at all
  4. Check the docs again; you might have missed something.

The fact that Pydantic AI works fine suggests your setup is good; this is likely AnythingLLM's agent invocation pattern not being initialized or model capability - not a system issue.

@Sinkingdev commented on GitHub (Dec 7, 2025): You’re right to be confused; it had me going for a spin when I first got here. Other LLM frameworks expose tools directly in the chat, so they’re always available and burning tokens. The key difference between AnythingLLM and frameworks like Pydantic AI, tools act as a separate entity working alongside your chat. **The crucial detail** Tools have to be enabled in the settings > Agent Skills Tools in AnythingLLM are only available when you explicitly invoke the **`@agent`** prefix. This is intentional; it keeps tools out of your normal chat to preserve tokens. **Try these prompts instead:** ``` @agent Use web search to find the latest version of the ratatui crate for Rust @agent Use DuckDuckGo to find the latest version of the ratatui crate for Rust @agent Use web search tool to find the latest version of the ratatui crate for Rust Or just simply "@agent test" ``` **How to know it's working:** - Your chat log will show something like this: `Agent @agent invoked. Swapping over to agent chat. Type /exit to exit..` - The agent will show a "thought" chain before responding (this indicates a tool was actually called) - You'll see `Agent session completed` when done **If it STILL doesn't work after adding @agent:** Then you may be hitting the model quality issue described in the docs. The models you're testing (Granite4, gpt-oss:20b, Mistral 3-14b) are smaller/quantized models that struggle with JSON generation for tool calls. Try: 1. **Reset chat history**: `/reset` then retry 2. **Disable unused tools**: In Agent Skills config, turn off tools you don't need—this reduces prompt bloat 3. **Test with a stronger model**: If possible, try a higher-quantization version or a cloud model (GPT-4, Claude) to confirm tools work at all 4. **Check the docs again; you might have missed something.** The fact that Pydantic AI works fine suggests your setup is good; this is likely AnythingLLM's agent invocation pattern not being initialized or model capability - not a system issue.
yindo commented 2026-02-22 18:32:10 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Dec 8, 2025):

As @Sinkingdev said, you just need to add the @agent in front of your query in order to enter into agent mode. This will allow your chat to then have access to all the tools like web search.

@shatfield4 commented on GitHub (Dec 8, 2025): As @Sinkingdev said, you just need to add the @agent in front of your query in order to enter into agent mode. This will allow your chat to then have access to all the tools like web search.
yindo changed title from [BUG]: Web search tool calling does not work to [GH-ISSUE #4733] [BUG]: Web search tool calling does not work 2026-06-05 14:49:40 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2990
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?