Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3586] [BUG]: [Agent Flow] Function Description provided by user is not seen in the LLM Query #2317

New Issue
Closed
opened 2026-02-22 18:29:09 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:29:09 -05:00
Owner
Copy Link
Copy Source

Originally created by @rajkumar9589 on GitHub (Apr 3, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3586

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

I am trying to understand how Agent Flow works in AnythingLLM. I created a sample AgentFlow named "CheckWeather" and gave a description.

Once the agent is invoked, AnythingLLM sends a query to LLM to pick a tool among all the available tools with the query starting with "You are a program which picks the most optimal function and parameters to call".

In this, function names, description, variables are also sent. For the default tools, the description is set properly. But for the tools created using AgentFlow Builder, the description is going as "Execute agent flow: ${flow.name}" instead of "flow.description".

For example , for the weather agent I created,

Function name: flow_403c960e-6397-4129-86d5-034b90932f22\n
Function Description: Execute agent flow: CheckWeather\n
...

This is happening even though in the agent flow the user has provided a valid description. This description is very important for the LLM to select the right tool.

"name": "CheckWeather",
"description": "This tool is used to find the location of a given place. This tool can help to provide the current weather conditions. ",
"active": true,

CheckWeatherAgentFlow.json

Are there known steps to reproduce?

  1. Create a new Agent Flow using AnythingLLM Agent Flow Builder.
  2. Take a packet capture at Ollama (if you are using Ollama as your LLM provider) to check the LLM Query.
  3. The Function description should ideally contain whatever the user has provided not the default "Execute agent flow: ${flow.name}".
Originally created by @rajkumar9589 on GitHub (Apr 3, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3586 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? I am trying to understand how Agent Flow works in AnythingLLM. I created a sample AgentFlow named "CheckWeather" and gave a description. Once the agent is invoked, AnythingLLM sends a query to LLM to pick a tool among all the available tools with the query starting with "You are a program which picks the most optimal function and parameters to call". In this, function names, description, variables are also sent. For the default tools, the description is set properly. But for the tools created using AgentFlow Builder, the description is going as "Execute agent flow: ${flow.name}" instead of "flow.description". For example , for the weather agent I created, Function name: flow_403c960e-6397-4129-86d5-034b90932f22\n Function Description: **Execute agent flow: CheckWeather**\n ... This is happening even though in the agent flow the user has provided a valid description. This description is very important for the LLM to select the right tool. "name": "CheckWeather", **"description": "This tool is used to find the location of a given place. This tool can help to provide the current weather conditions. ",** "active": true, [CheckWeatherAgentFlow.json](https://github.com/user-attachments/files/19582275/CheckWeatherAgentFlow.json) ### Are there known steps to reproduce? 1. Create a new Agent Flow using AnythingLLM Agent Flow Builder. 2. Take a packet capture at Ollama (if you are using Ollama as your LLM provider) to check the LLM Query. 3. The Function description should ideally contain whatever the user has provided not the default "Execute agent flow: ${flow.name}".
yindo added the investigating label 2026-02-22 18:29:09 -05:00
yindo closed this issue 2026-02-22 18:29:09 -05:00
yindo commented 2026-02-22 18:29:09 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 3, 2025):

That is simply what we surface to the UI - we dont put the whole description in the agent call status indicator since it can be super wordy. That UI element is just to indicate to the user a specific tool was called.

Here is the exact prompt sent to Ollama, for example, with default tools enabled and a single flow that hits an API

You are a program which picks the most optimal function and parameters to call.
    DO NOT HAVE TO PICK A FUNCTION IF IT WILL NOT HELP ANSWER OR FULFILL THE USERS QUERY.
          When a function is selection, respond in JSON with no additional text.
          When there is no relevant function to call - return with a regular chat text response.
          Your task is to pick a **single** function that we will use to call, if any seem useful or relevant for the user query.
    
          All JSON responses should have two keys.
    name: this is the name of the function name to call. eg: web-scraper, rag-memory, etc..
    arguments: this is an object with the function properties to invoke the function.
          DO NOT INCLUDE ANY OTHER KEYS IN JSON RESPONSES.
    
          Here are the available tools you can use an examples of a query and response so you can understand how each one works.
          -----------
    Function name: rag-memory
    "Function Description: Search against local documents for context that is relevant to the query or store a snippet of text into memory for retrieval later. Storing information should only be done when the user specifically requests for information to be remembered or saved to long-term memory. You should use this tool before search the internet for information. Do not use this tool unless you are explicity told to remember or store information.
    Function parameters in JSON format:
    {
        "action": {
            "type": "string",
            "enum": [
                "search",
                "store"
            ],
            "description": "The action we want to take to search for existing similar context or storage of new context."
        },
        "content": {
            "type": "string",
            "description": "The plain text to search our local documents with or to store in our vector database."
        }
    }
    Query: "What is AnythingLLM?"
    JSON: {"name":"rag-memory","arguments":{"action":"search","content":"What is AnythingLLM?"}}
    `Query: "What do you know about Platos motives?"` 
    `JSON: {"name":"rag-memory","arguments":{"action":"search","content":"What are the facts about Platos motives?"}}` 
    Query: "Remember that you are a robot"
    JSON: {"name":"rag-memory","arguments":{"action":"store","content":"I am a robot, the user told me that i am."}}
    Query: "Save that to memory please."
    JSON: {"name":"rag-memory","arguments":{"action":"store","content":"<insert summary of conversation until now>"}}
    -----------
    -----------
    Function name: document-summarizer
    Function Description: Can get the list of files available to search with descriptions and can select a single file to open and summarize.
    Function parameters in JSON format:
    {
        "action": {
            "type": "string",
            "enum": [
                "list",
                "summarize"
            ],
    `        "description": "The action to take. list will return all files available with their filename and descriptions. summarize will open and summarize the file by the a document name."` 
        },
        "document_filename": {
            "type": "string",
            "x-nullable": true,
            "description": "The file name of the document you want to get the full content of."
        }
    }
    Query: "Summarize example.txt"
    JSON: {"name":"document-summarizer","arguments":{"action":"summarize","document_filename":"example.txt"}}
    Query: "What files can you see?"
    JSON: {"name":"document-summarizer","arguments":{"action":"list","document_filename":null}}
    Query: "Tell me about readme.md"
    JSON: {"name":"document-summarizer","arguments":{"action":"summarize","document_filename":"readme.md"}}
    -----------
    -----------
    Function name: web-scraping
    Function Description: Scrapes the content of a webpage or online resource from a provided URL.
    Function parameters in JSON format:
    {
        "url": {
            "type": "string",
            "format": "uri",
            "description": "A complete web address URL including protocol. Assumes https if not provided."
        }
    }
    Query: "What is anythingllm.com about?"
    JSON: {"name":"web-scraping","arguments":{"url":"https://anythingllm.com"}}
    Query: "Scrape https://example.com"
    JSON: {"name":"web-scraping","arguments":{"url":"https://example.com"}}
    -----------
    -----------
    Function name: flow_8ab9b0b2-f1fa-4622-a68d-c8c371de1086
    Function Description: Execute agent flow: Get TODOs or Invoices
    Function parameters in JSON format:
    {
        "todoID": {
            "type": "string",
            "description": "Value for variable todoID"
        },
        "class": {
            "type": "string",
            "description": "Value for variable class"
        }
    }
    -----------
    Now pick a function if there is an appropriate one to use given the last user message and the given conversation so far.

Function name: flow_8ab9b0b2-f1fa-4622-a68d-c8c371de1086
Function Description: Execute agent flow: Get TODOs or Invoices <--- my description for flow.
Function parameters in JSON format:
{
"todoID": {
"type": "string",
"description": "Value for variable todoID"
},
"class": {
"type": "string",
"description": "Value for variable class"
}
}

@timothycarambat commented on GitHub (Apr 3, 2025): That is simply what we surface to the UI - we dont put the whole description in the agent call status indicator since it can be super wordy. That UI element is just to indicate to the user a specific tool was called. Here is the exact prompt sent to Ollama, for example, with default tools enabled and a single flow that hits an API ``` You are a program which picks the most optimal function and parameters to call. DO NOT HAVE TO PICK A FUNCTION IF IT WILL NOT HELP ANSWER OR FULFILL THE USERS QUERY. When a function is selection, respond in JSON with no additional text. When there is no relevant function to call - return with a regular chat text response. Your task is to pick a **single** function that we will use to call, if any seem useful or relevant for the user query. All JSON responses should have two keys. name: this is the name of the function name to call. eg: web-scraper, rag-memory, etc.. arguments: this is an object with the function properties to invoke the function. DO NOT INCLUDE ANY OTHER KEYS IN JSON RESPONSES. Here are the available tools you can use an examples of a query and response so you can understand how each one works. ----------- Function name: rag-memory "Function Description: Search against local documents for context that is relevant to the query or store a snippet of text into memory for retrieval later. Storing information should only be done when the user specifically requests for information to be remembered or saved to long-term memory. You should use this tool before search the internet for information. Do not use this tool unless you are explicity told to remember or store information. Function parameters in JSON format: { "action": { "type": "string", "enum": [ "search", "store" ], "description": "The action we want to take to search for existing similar context or storage of new context." }, "content": { "type": "string", "description": "The plain text to search our local documents with or to store in our vector database." } } Query: "What is AnythingLLM?" JSON: {"name":"rag-memory","arguments":{"action":"search","content":"What is AnythingLLM?"}} `Query: "What do you know about Platos motives?"` `JSON: {"name":"rag-memory","arguments":{"action":"search","content":"What are the facts about Platos motives?"}}` Query: "Remember that you are a robot" JSON: {"name":"rag-memory","arguments":{"action":"store","content":"I am a robot, the user told me that i am."}} Query: "Save that to memory please." JSON: {"name":"rag-memory","arguments":{"action":"store","content":"<insert summary of conversation until now>"}} ----------- ----------- Function name: document-summarizer Function Description: Can get the list of files available to search with descriptions and can select a single file to open and summarize. Function parameters in JSON format: { "action": { "type": "string", "enum": [ "list", "summarize" ], ` "description": "The action to take. list will return all files available with their filename and descriptions. summarize will open and summarize the file by the a document name."` }, "document_filename": { "type": "string", "x-nullable": true, "description": "The file name of the document you want to get the full content of." } } Query: "Summarize example.txt" JSON: {"name":"document-summarizer","arguments":{"action":"summarize","document_filename":"example.txt"}} Query: "What files can you see?" JSON: {"name":"document-summarizer","arguments":{"action":"list","document_filename":null}} Query: "Tell me about readme.md" JSON: {"name":"document-summarizer","arguments":{"action":"summarize","document_filename":"readme.md"}} ----------- ----------- Function name: web-scraping Function Description: Scrapes the content of a webpage or online resource from a provided URL. Function parameters in JSON format: { "url": { "type": "string", "format": "uri", "description": "A complete web address URL including protocol. Assumes https if not provided." } } Query: "What is anythingllm.com about?" JSON: {"name":"web-scraping","arguments":{"url":"https://anythingllm.com"}} Query: "Scrape https://example.com" JSON: {"name":"web-scraping","arguments":{"url":"https://example.com"}} ----------- ----------- Function name: flow_8ab9b0b2-f1fa-4622-a68d-c8c371de1086 Function Description: Execute agent flow: Get TODOs or Invoices Function parameters in JSON format: { "todoID": { "type": "string", "description": "Value for variable todoID" }, "class": { "type": "string", "description": "Value for variable class" } } ----------- Now pick a function if there is an appropriate one to use given the last user message and the given conversation so far. ``` > Function name: flow_8ab9b0b2-f1fa-4622-a68d-c8c371de1086 Function Description: Execute agent flow: Get TODOs or Invoices <--- my description for flow. Function parameters in JSON format: { "todoID": { "type": "string", "description": "Value for variable todoID" }, "class": { "type": "string", "description": "Value for variable class" } }
yindo commented 2026-02-22 18:29:09 -05:00
Author
Owner
Copy Link
Copy Source

@rajkumar9589 commented on GitHub (Apr 4, 2025):

Thanks for the quick resolution !! I have verified the changes and it is working fine.

Function name: flow_403c960e-6397-4129-86d5-034b90932f22\n

Function Description: This tool is used to find the location of a given place. This tool can help to provide the current weather conditions.
....

@rajkumar9589 commented on GitHub (Apr 4, 2025): Thanks for the quick resolution !! I have verified the changes and it is working fine. Function name: flow_403c960e-6397-4129-86d5-034b90932f22\n **Function Description: This tool is used to find the location of a given place. This tool can help to provide the current weather conditions.** ....
yindo changed title from [BUG]: [Agent Flow] Function Description provided by user is not seen in the LLM Query to [GH-ISSUE #3586] [BUG]: [Agent Flow] Function Description provided by user is not seen in the LLM Query 2026-06-05 14:45:52 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2317
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?