Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4955] [BUG]: Passing information into agent flows fails #3107

New Issue
Closed
opened 2026-02-22 18:32:38 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:32:38 -05:00
Owner
Copy Link
Copy Source

Originally created by @elevatingcreativity on GitHub (Feb 1, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4955

How are you running AnythingLLM?

Docker (local)

What happened?

I am trying to pass a block of text information, or a document, into an agent flow for analysis.
I created a simple "Parameter Test" flow, with the following details:

  • Name: "Parameter Test"
  • Description: "This flow tests the passing of information. It has one required parameter: full_input. You MUST copy the entire user message word-for-word into full_input. Do not summarize. Do not extract fields. Do not omit anything."
  • Flow Variables: (one variable) full_input
  • Step 1 (the only step): "Please review both the chat, the user input, and the input in ${full_input}. Summarize it" (Direct Output is toggled on).

I was trying more complex flows, but I couldn't get any more than a few words of input passed into this flow for any kind of use or analysis. I tried it via both priming the chat with the text and then invoking the agent, and I've tried invoking it like this (and many variants thereof):

@agent parameter test with the following full_input: Here I have a .... [long text ommitted]
Response:
Agent complete

Agent @agent invoked. Swapping over to agent chat. Type /exit to exit agent execution loop early.
Assembling Tool Call: flow_f19225d7-7479-45fb-a41d-5db44b0351b6()
Executing flow: Parameter Test
Processing data with LLM instruction...
Sending request to LLM...
Successfully received LLM response
Parameter Test completed successfully
The tool call has direct output enabled! The result will be returned directly to the chat without any further processing and no further tool calls will be run.
Tool use completed.

response:

I’d be happy to help summarize the content, but I don’t see any chat history, user input, or content within angle brackets (< >) in your message.
Could you please provide:
The chat history you’d like me to review
The user input you’re referring to
The content that should be in the angle brackets
Once you share these materials, I’ll provide a clear and concise summary for you.

The agent is invoking claude Sonnet 3.5 on the backend, and my document is at most 20k tokens, claude has a token limit of 200k, and I can use the same document in a regular chat.

I have tried this many different ways. I have analyzed the situation with both claude and chatGPT to see if there's anything I missed. I followed the hacker news example, but that example does not show how to pass more extensive information in.

I tried having chatGPT analyze the source code, but I didn't get any luck with that. It did mention there is a parameter called "description" for each variable that should describe what that variable is, but there's no way I can see to set that in the GUI. And I don't know if it was hallucinating or not.

So, I am not sure this is a bug, but it seems like it may be.

It would be great if I can get this working to pass larger amounts of information into the agent flow. Thanks

Are there known steps to reproduce?

  1. Create a simple agent flow to take the text input either during agent invocation, or from the preceding chat, and then summarize it
  2. Invoke the agent with some block of text following the invocation (and/or in the preceding chat)
  3. Watch it not find any text
Originally created by @elevatingcreativity on GitHub (Feb 1, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4955 ### How are you running AnythingLLM? Docker (local) ### What happened? I am trying to pass a block of text information, or a document, into an agent flow for analysis. I created a simple "Parameter Test" flow, with the following details: - Name: "Parameter Test" - Description: "This flow tests the passing of information. It has one required parameter: full_input. You MUST copy the entire user message word-for-word into full_input. Do not summarize. Do not extract fields. Do not omit anything." - Flow Variables: (one variable) full_input - Step 1 (the only step): "Please review both the chat, the user input, and the input in ${full_input}. Summarize it" (Direct Output is toggled on). I was trying more complex flows, but I couldn't get any more than a few words of input passed into this flow for any kind of use or analysis. I tried it via both priming the chat with the text and then invoking the agent, and I've tried invoking it like this (and many variants thereof): > @agent parameter test with the following full_input: Here I have a .... [long text ommitted] Response: > Agent complete > Agent @agent invoked. Swapping over to agent chat. Type /exit to exit agent execution loop early. > Assembling Tool Call: flow_f19225d7-7479-45fb-a41d-5db44b0351b6() > Executing flow: Parameter Test > Processing data with LLM instruction... > Sending request to LLM... > Successfully received LLM response > Parameter Test completed successfully > The tool call has direct output enabled! The result will be returned directly to the chat without any further processing and no further tool calls will be run. > Tool use completed. response: > I’d be happy to help summarize the content, but I don’t see any chat history, user input, or content within angle brackets (< >) in your message. > Could you please provide: > The chat history you’d like me to review > The user input you’re referring to > The content that should be in the angle brackets > Once you share these materials, I’ll provide a clear and concise summary for you. The agent is invoking claude Sonnet 3.5 on the backend, and my document is at most 20k tokens, claude has a token limit of 200k, and I can use the same document in a regular chat. I have tried this many different ways. I have analyzed the situation with both claude and chatGPT to see if there's anything I missed. I followed the hacker news example, but that example does not show how to pass more extensive information in. I tried having chatGPT analyze the source code, but I didn't get any luck with that. It did mention there is a parameter called "description" for each variable that should describe what that variable is, but there's no way I can see to set that in the GUI. And I don't know if it was hallucinating or not. So, I am not _sure_ this is a bug, but it seems like it may be. It would be great if I can get this working to pass larger amounts of information into the agent flow. Thanks ### Are there known steps to reproduce? 1. Create a simple agent flow to take the text input either during agent invocation, or from the preceding chat, and then summarize it 2. Invoke the agent with some block of text following the invocation (and/or in the preceding chat) 3. Watch it not find any text
yindo added the possible bug label 2026-02-22 18:32:38 -05:00
yindo closed this issue 2026-02-22 18:32:38 -05:00
yindo commented 2026-02-22 18:32:39 -05:00
Author
Owner
Copy Link
Copy Source

@elevatingcreativity commented on GitHub (Feb 13, 2026):

You're right, I apologize for not listening! Here's the revised message in one block:

It seems the problem is specifically associated with the LLM's output token limit when generating tool call arguments.

I found this when I tested with shorter text (~2k characters) - the test flow worked fine. The agent correctly extracted the text into my full_input variable and summarized it. But when I used longer text (>10k characters), it failed as if I hadn't provided any input at all.

After digging into the code with help from Claude Code, we found the potential culprit: the Anthropic provider has a hard-coded 4,096 token output limit.

Here's what's happening:

  1. I invoke the agent with a long document: @agent ParameterTest <my long text>
  2. The agent's LLM tries to call the flow tool and copy my text into the parameter: flow_xyz(full_input: "Here is the long text..."
  3. But mid-way through generating this JSON, it hits the 4,096 token output limit and gets cut off
  4. The system receives broken JSON like {"full_input": "Here is the long te (truncated)
  5. The JSON parser tries to fix it but ends up with an empty or incomplete value
  6. My flow runs with basically no input, so it responds "I don't see any content"

However, the behavior is inconsistent. Even with very large documents (80k+ characters), flows sometimes succeed and sometimes fail - both with the original 4,096 token limit AND with my modified 8,192 token limit.

I believe this inconsistency occurs because the LLM sometimes attempts to copy the full text verbatim (which gets truncated), and sometimes intelligently summarizes or condenses it to fit within the token limit. This is non-deterministic behavior based on the LLM's internal decision-making.

It also explains why I've noticed Claude output is sometimes truncated prematurely - this appears to be a separate but related bug where regular Claude responses get cut off mid-response due to the same 4,096 token output limit.

Fix Attempted

I increased the limit in my local branch from 4,096 to 8,192 tokens in the Anthropic provider code. Claude 3.5 Sonnet supports up to 8,192 output tokens, and Opus 4 goes up to 16,384.

Setting it to 8,192 seems to increase reliability somewhat, but the inconsistency remains - even 80k character documents sometimes work and sometimes fail.

Proposal

I'm thinking of testing an option in the agent flows that, when turned ON, would bypass the LLM tool-calling mechanism entirely and pass the entire user input (and optionally the chat context) directly to the flow. This mode would be most useful for single-variable inputs focused on document analysis, while the current extraction mode remains appropriate for multi-variable structured data analysis.

Since many use cases of AI involve processing of long text documents, this would make agent flows more suitable for document analysis use cases.

I may implement this in my local fork for testing.

Peripheral Issues I Found:

While investigating, I also noticed there's no UI field to add descriptions for flow variables, even though the backend code checks for them.

Files Being Modified for the token limit fix:

  • server/utils/agents/aibitat/providers/anthropic.js - bump max_tokens to 8192
  • server/utils/agentFlows/index.js - add required field to parameters

I'll test these changes more thoroughly and submit a PR. Happy to discuss alternative approaches if anyone has concerns!

Thanks

NOTE: I accidentally marked this CLOSED because I apparently forgot how to use Github (it's been a while). I'm not sure whether it should be marked closed quite yet.

@elevatingcreativity commented on GitHub (Feb 13, 2026): You're right, I apologize for not listening! Here's the revised message in one block: --- It seems the problem is specifically associated with **the LLM's output token limit when generating tool call arguments**. I found this when I tested with shorter text (~2k characters) - the test flow worked fine. The agent correctly extracted the text into my `full_input` variable and summarized it. But when I used longer text (>10k characters), it failed as if I hadn't provided any input at all. After digging into the code with help from Claude Code, we found the potential culprit: **the Anthropic provider has a hard-coded 4,096 token output limit**. Here's what's happening: 1. I invoke the agent with a long document: `@agent ParameterTest <my long text>` 2. The agent's LLM tries to call the flow tool and copy my text into the parameter: `flow_xyz(full_input: "Here is the long text..."` 3. But mid-way through generating this JSON, it hits the 4,096 token **output limit** and gets cut off 4. The system receives broken JSON like `{"full_input": "Here is the long te` (truncated) 5. The JSON parser tries to fix it but ends up with an empty or incomplete value 6. My flow runs with basically no input, so it responds "I don't see any content" **However, the behavior is inconsistent.** Even with very large documents (80k+ characters), flows sometimes succeed and sometimes fail - both with the original 4,096 token limit AND with my modified 8,192 token limit. I believe this inconsistency occurs because **the LLM sometimes attempts to copy the full text verbatim (which gets truncated), and sometimes intelligently summarizes or condenses it to fit within the token limit.** This is non-deterministic behavior based on the LLM's internal decision-making. It also explains why I've noticed Claude output is sometimes truncated prematurely - this appears to be a separate but related bug where regular Claude responses get cut off mid-response due to the same 4,096 token output limit. ### Fix Attempted I increased the limit in my local branch from 4,096 to 8,192 tokens in the Anthropic provider code. Claude 3.5 Sonnet supports up to 8,192 output tokens, and Opus 4 goes up to 16,384. **Setting it to 8,192 seems to increase reliability somewhat, but the inconsistency remains** - even 80k character documents sometimes work and sometimes fail. ### Proposal I'm thinking of testing an option in the agent flows that, when turned ON, would bypass the LLM tool-calling mechanism entirely and pass the entire user input (and optionally the chat context) directly to the flow. **This mode would be most useful for single-variable inputs focused on document analysis, while the current extraction mode remains appropriate for multi-variable structured data analysis.** Since many use cases of AI involve processing of long text documents, this would make agent flows more suitable for document analysis use cases. I may implement this in my local fork for testing. **Peripheral Issues I Found:** While investigating, I also noticed there's no UI field to add descriptions for flow variables, even though the backend code checks for them. **Files Being Modified for the token limit fix:** - `server/utils/agents/aibitat/providers/anthropic.js` - bump max_tokens to 8192 - `server/utils/agentFlows/index.js` - add required field to parameters I'll test these changes more thoroughly and submit a PR. Happy to discuss alternative approaches if anyone has concerns! Thanks NOTE: I accidentally marked this CLOSED because I apparently forgot how to use Github (it's been a while). I'm not sure whether it should be marked closed quite yet.
yindo changed title from [BUG]: Passing information into agent flows fails to [GH-ISSUE #4955] [BUG]: Passing information into agent flows fails 2026-06-05 14:50:20 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#3107
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?