langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 03:55:25 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

bug: FC agent runner silently loses data when LLM calls the same tool multiple times #22258

New Issue
Open
opened 2026-02-21 20:16:20 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:16:20 -05:00
Owner
Copy Link
Copy Source

Originally created by @trongtrandp on GitHub (Feb 20, 2026).

Self Checks

  • This is only a bug report, not a feature request or other type of issue.
  • I have searched for existing issues to avoid creating duplicates.

Dify version

1.13.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

  1. Create an Agent app with Function Calling mode
  2. Add a tool (e.g., search) that the LLM might call multiple times
  3. Send a prompt that causes the LLM to call the same tool 2+ times in a single thought (e.g., "Search for 'Python tutorial' and also search for 'JavaScript tutorial'")
  4. Check the agent thought logs

✅ Expected Behavior

All tool calls should be preserved with their individual inputs and outputs. For example:

  • search("Python tutorial") → result A
  • search("JavaScript tutorial") → result B

❌ Actual Behavior

Only the last tool call's data survives. Previous calls are silently overwritten.

In the example above, only search("JavaScript tutorial") → result B is stored. The first search call is completely lost.

Root Cause

The FC agent runner uses dict comprehensions keyed by tool name to store tool data. When the same tool is called multiple times, later entries overwrite earlier ones:

# api/core/agent/fc_agent_runner.py

# Line ~128 (streaming path) and ~155 (blocking path):
tool_call_inputs = json.dumps(
    {tool_call[1]: tool_call[2] for tool_call in tool_calls},  # ← dict key collision!
    ensure_ascii=False,
)

# Line ~287-292 (observation and meta):
tool_invoke_meta={
    tool_response["tool_call_name"]: tool_response["tool_invoke_meta"]
    for tool_response in tool_responses  # ← same collision
},
observation={
    tool_response["tool_call_name"]: tool_response["tool_response"]
    for tool_response in tool_responses  # ← same collision
},

4 collision sites in total — affecting tool_input, observation, and tool_invoke_meta storage.

Impact

  • Data loss: Tool call inputs, outputs, and metadata are silently discarded
  • Misleading logs: Agent thought logs show only one tool call when multiple were actually executed
  • Affects all FC agent apps where the LLM decides to call the same tool more than once in a single thought

Degraded agent reasoning in multi-turn conversations

This bug doesn't just affect log display — it directly degrades agent quality in subsequent turns.

When building conversation history for the next LLM call (base_agent_runner.py → organize_agent_history), the system:

  1. Reads agent_thought.tool_input from DB → creates AssistantPromptMessage.ToolCall list
  2. Reads agent_thought.observation from DB → creates ToolPromptMessage list
  3. Sends this as conversation history to the LLM

Since the data is already lost at write time, the reconstructed history is incomplete. The LLM only sees the last tool call instead of all calls that were actually executed. This causes:

  • Redundant tool calls: The LLM thinks a previous search was never performed → calls the same tool again → wastes tokens and increases latency
  • Incorrect reasoning: The LLM makes decisions based on partial information, missing results from earlier tool calls
  • Potential infinite loops: In edge cases, the agent may repeatedly re-invoke the "missing" tool call, never converging to a final answer

Example: LLM calls search("Python") then search("JavaScript") in thought 1. In thought 2, history only contains search("JavaScript"). The LLM believes it never searched for Python → calls search("Python") again → wasted round-trip, and the agent's multi-step reasoning is compromised.

Proposed Fix

PR #32442 switches the storage format from dict to array:

# Before (loses data):
{tool_call[1]: tool_call[2] for tool_call in tool_calls}

# After (preserves all):
[{"name": tool_call[1], "arguments": tool_call[2]} for tool_call in tool_calls]

With backward-compatible property accessors that handle both old and new formats — no database migration required.

Originally created by @trongtrandp on GitHub (Feb 20, 2026). ## Self Checks - [x] This is only a bug report, not a feature request or other type of issue. - [x] I have searched for existing issues to avoid creating duplicates. ## Dify version 1.13.0 ## Cloud or Self Hosted Self Hosted (Docker) ## Steps to reproduce 1. Create an Agent app with Function Calling mode 2. Add a tool (e.g., `search`) that the LLM might call multiple times 3. Send a prompt that causes the LLM to call the same tool 2+ times in a single thought (e.g., "Search for 'Python tutorial' and also search for 'JavaScript tutorial'") 4. Check the agent thought logs ## ✅ Expected Behavior All tool calls should be preserved with their individual inputs and outputs. For example: - `search("Python tutorial")` → result A - `search("JavaScript tutorial")` → result B ## ❌ Actual Behavior Only the **last** tool call's data survives. Previous calls are silently overwritten. In the example above, only `search("JavaScript tutorial")` → result B is stored. The first search call is completely lost. ## Root Cause The FC agent runner uses **dict comprehensions keyed by tool name** to store tool data. When the same tool is called multiple times, later entries overwrite earlier ones: ```python # api/core/agent/fc_agent_runner.py # Line ~128 (streaming path) and ~155 (blocking path): tool_call_inputs = json.dumps( {tool_call[1]: tool_call[2] for tool_call in tool_calls}, # ← dict key collision! ensure_ascii=False, ) # Line ~287-292 (observation and meta): tool_invoke_meta={ tool_response["tool_call_name"]: tool_response["tool_invoke_meta"] for tool_response in tool_responses # ← same collision }, observation={ tool_response["tool_call_name"]: tool_response["tool_response"] for tool_response in tool_responses # ← same collision }, ``` **4 collision sites** in total — affecting `tool_input`, `observation`, and `tool_invoke_meta` storage. ## Impact - **Data loss**: Tool call inputs, outputs, and metadata are silently discarded - **Misleading logs**: Agent thought logs show only one tool call when multiple were actually executed - **Affects all FC agent apps** where the LLM decides to call the same tool more than once in a single thought ### Degraded agent reasoning in multi-turn conversations This bug doesn't just affect log display — it **directly degrades agent quality** in subsequent turns. When building conversation history for the next LLM call (`base_agent_runner.py` → `organize_agent_history`), the system: 1. Reads `agent_thought.tool_input` from DB → creates `AssistantPromptMessage.ToolCall` list 2. Reads `agent_thought.observation` from DB → creates `ToolPromptMessage` list 3. Sends this as conversation history to the LLM Since the data is already lost at write time, the reconstructed history is **incomplete**. The LLM only sees the last tool call instead of all calls that were actually executed. This causes: - **Redundant tool calls**: The LLM thinks a previous search was never performed → calls the same tool again → wastes tokens and increases latency - **Incorrect reasoning**: The LLM makes decisions based on partial information, missing results from earlier tool calls - **Potential infinite loops**: In edge cases, the agent may repeatedly re-invoke the "missing" tool call, never converging to a final answer **Example**: LLM calls `search("Python")` then `search("JavaScript")` in thought 1. In thought 2, history only contains `search("JavaScript")`. The LLM believes it never searched for Python → calls `search("Python")` again → wasted round-trip, and the agent's multi-step reasoning is compromised. ## Proposed Fix PR #32442 switches the storage format from dict to array: ```python # Before (loses data): {tool_call[1]: tool_call[2] for tool_call in tool_calls} # After (preserves all): [{"name": tool_call[1], "arguments": tool_call[2]} for tool_call in tool_calls] ``` With backward-compatible property accessors that handle both old and new formats — no database migration required.
yindo added the 🐞 bug label 2026-02-21 20:16:20 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#22258
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?