langchain-ai/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-07-15 12:35:43 -04:00
Code Issues 314 Packages Projects Releases 17 Wiki Activity

ReAct Agent doesn't recognize ALL the tool responses made after a single AI (aka assistant) message contained several tool calls #333

New Issue
Closed
opened 2026-02-20 17:36:54 -05:00 by yindo · 5 comments
yindo commented 2026-02-20 17:36:54 -05:00
Owner
Copy Link
Copy Source

Originally created by @daqo98 on GitHub (Dec 2, 2024).

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangGraph/LangChain documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangGraph/LangChain rather than my code.
  • I am sure this is better as an issue rather than a GitHub discussion, since this is a LangGraph bug and not a design question.

Example Code

import os
import logging
from langchain_core.messages import SystemMessage, HumanMessage
from langgraph.prebuilt import create_react_agent
from langchain_openai import ChatOpenAI
from .toolkit import CustomSQLDatabaseToolkit
from .prompts import PROMPTS

class Agent():
    """
    Agent: 
    Usage example: 

    """
    def __init__(self, db, user_prompt):
        self.db = db
        self.user_prompt = user_prompt

        self.llm = ChatOpenAI(model=GPT_MODEL,
                        temperature=0,
                        max_retries=MAX_RETRIES,
                        request_timeout=TIMEOUT_IN_SECONDS
                    )
        
        # Initialize the toolkit
        toolkit = CustomSQLDatabaseToolkit(db=self.db, llm=self.llm)
        # Get the tools from the toolkit
        self.tools = toolkit.get_tools()

        # Create the Agent
        self.sys_msg_dataquery = SystemMessage(content="<some prompt>")
        self.agent_executor = create_react_agent(self.llm, self.tools, state_modifier=self.sys_msg_dataquery)

    def gen_response_dataquery_table(self):
        messages = []
        inputs = {"messages": [HumanMessage(content=self.user_prompt)]}

        for s in self.agent_executor.stream(inputs, stream_mode="values"):

            message = s["messages"][-1]
            if isinstance(message, tuple):
                logger.info(message)
            else:
                message.pretty_print()

            messages.append(message)

Error Message and Stack Trace (if applicable)

[2024-11-27 21:43:22,239: WARNING/ForkPoolWorker-1] ================================ Human Message =================================

Show me the data of Italy
[2024-11-27 21:43:23,253: INFO/ForkPoolWorker-1] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
[2024-11-27 21:43:31,124: WARNING/ForkPoolWorker-1] ================================== Ai Message ==================================
Tool Calls:
  sql_db_query_checker (call_RGzcL13f3gicJYM8PV9mRFtM)
 Call ID: call_RGzcL13f3gicJYM8PV9mRFtM
  Args:
    query: SELECT * FROM country_codes WHERE Country_name = 'Italy'
  sql_db_query_checker (call_2tHWQsf8JzFRTwUcwQmhDUGX)
 Call ID: call_2tHWQsf8JzFRTwUcwQmhDUGX
  Args:
    query: SELECT * FROM geo_data WHERE Country = 'ITA'
  sql_db_query_checker (call_0mZ8gcfp8Ax7sntdwNcDu9AM)
 Call ID: call_0mZ8gcfp8Ax7sntdwNcDu9AM
  Args:
    query: SELECT * FROM italian_cities WHERE Regione = 'Lazio' OR Regione = 'Lombardia' OR Regione = 'Campania' OR Regione = 'Sicilia' OR Regione = 'Veneto' OR Regione = 'Emilia-Romagna' OR Regione = 'Piemonte' OR Regione = 'Toscana' OR Regione = 'Liguria' OR Regione = 'Marche' OR Regione = 'Abruzzo' OR Regione = 'Puglia' OR Regione = 'Calabria' OR Regione = 'Sardegna' OR Regione = 'Friuli-Venezia Giulia' OR Regione = 'Umbria' OR Regione = 'Basilicata' OR Regione = 'Molise' OR Regione = 'Aosta Valley' OR Regione = 'Trentino-Alto Adige' OR Regione = 'Apulia' OR Regione = 'Lazio' OR Regione = 'Lombardia' OR Regione = 'Campania' OR Regione = 'Sicilia' OR Regione = 'Veneto' OR Regione = 'Emilia-Romagna' OR Regione = 'Piemonte' OR Regione = 'Toscana' OR Regione = 'Liguria' OR Regione = 'Marche' OR Regione = 'Abruzzo' OR Regione = 'Puglia' OR Regione = 'Calabria' OR Regione = 'Sardegna' OR Regione = 'Friuli-Venezia Giulia' OR Regione = 'Umbria' OR Regione = 'Basilicata' OR Regione = 'Molise' OR Regione = 'Aosta Valley' OR Regione = 'Trentino-Alto Adige'
  sql_db_query_checker (call_3nQeSZ7vJCr5GAoCI6vVPfx7)
 Call ID: call_3nQeSZ7vJCr5GAoCI6vVPfx7
  Args:
    query: SELECT * FROM italian_cities
[2024-11-27 21:43:31,878: INFO/ForkPoolWorker-1] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
[2024-11-27 21:43:31,932: INFO/ForkPoolWorker-1] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
[2024-11-27 21:43:33,373: INFO/ForkPoolWorker-1] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
[2024-11-27 21:43:35,026: INFO/ForkPoolWorker-1] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
[2024-11-27 21:43:35,033: WARNING/ForkPoolWorker-1] ================================= Tool Message =================================
Name: sql_db_query_checker

sql
SELECT * FROM italian_cities

[2024-11-27 21:43:46,796: WARNING/ForkPoolWorker-1] ================================== Ai Message ==================================
sql
SELECT * FROM country_codes WHERE Country_name = 'Italy';

[2024-11-27 21:43:47,205: INFO/ForkPoolWorker-1] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 400 Bad Request"
[2024-11-27 21:43:47,214: INFO/ForkPoolWorker-1] Error code: 400 - {'error': {'message': "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_RGzcL13f3gicJYM8PV9mRFtM, call_2tHWQsf8JzFRTwUcwQmhDUGX, call_0mZ8gcfp8Ax7sntdwNcDu9AM", 'type': 'invalid_request_error', 'param': 'messages.[7].role', 'code': None}}

Description

Hi everyone, I am working on a Django app that uses Celery for processing tasks in the background. I am calling a wrapper function of the ReAct agent (create_react_agent) and when the agent performs multiple tool calls inside a single assistant/AI message, it only detects the last tool call and breaks the app with a 400 Error code at the end of the agent streaming. How to solve this? i.e. getting the agent to recognize all the tool calls.

When I run it in a Jupyter notebook the cell is executed without any problem while when I run it in the Django app, it breaks the app. Do you have any leads to at least avoid breaking the app?

Thanks in advance

System Info

System Information

OS: Linux
OS Version: #1 SMP Tue Nov 5 00:21:55 UTC 2024
Python Version: 3.11.9 (main, May 13 2024, 11:06:43) [GCC 9.4.0]

Package Information

langchain_core: 0.2.43
langchain: 0.2.16
langchain_community: 0.2.17
langsmith: 0.1.143
langchain_anthropic: 0.1.23
langchain_openai: 0.1.10
langchain_text_splitters: 0.2.4
langgraph: 0.2.50

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.11.6
anthropic: 0.39.0
async-timeout: Installed. No version info available.
dataclasses-json: 0.6.7
defusedxml: 0.7.1
httpx: 0.27.2
jsonpatch: 1.33
langgraph-checkpoint: 2.0.5
langgraph-sdk: 0.1.36
numpy: 1.26.4
openai: 1.28.1
orjson: 3.10.11
packaging: 24.2
pydantic: 2.9.2
PyYAML: 6.0.2
requests: 2.32.3
requests-toolbelt: 1.0.0
SQLAlchemy: 2.0.36
tenacity: 8.5.0
tiktoken: 0.8.0
typing-extensions: 4.12.2

Originally created by @daqo98 on GitHub (Dec 2, 2024). ### Checked other resources - [X] I added a very descriptive title to this issue. - [X] I searched the [LangGraph](https://langchain-ai.github.io/langgraph/)/LangChain documentation with the integrated search. - [X] I used the GitHub search to find a similar question and didn't find it. - [X] I am sure that this is a bug in LangGraph/LangChain rather than my code. - [X] I am sure this is better as an issue [rather than a GitHub discussion](https://github.com/langchain-ai/langgraph/discussions/new/choose), since this is a LangGraph bug and not a design question. ### Example Code ```python import os import logging from langchain_core.messages import SystemMessage, HumanMessage from langgraph.prebuilt import create_react_agent from langchain_openai import ChatOpenAI from .toolkit import CustomSQLDatabaseToolkit from .prompts import PROMPTS class Agent(): """ Agent: Usage example: """ def __init__(self, db, user_prompt): self.db = db self.user_prompt = user_prompt self.llm = ChatOpenAI(model=GPT_MODEL, temperature=0, max_retries=MAX_RETRIES, request_timeout=TIMEOUT_IN_SECONDS ) # Initialize the toolkit toolkit = CustomSQLDatabaseToolkit(db=self.db, llm=self.llm) # Get the tools from the toolkit self.tools = toolkit.get_tools() # Create the Agent self.sys_msg_dataquery = SystemMessage(content="<some prompt>") self.agent_executor = create_react_agent(self.llm, self.tools, state_modifier=self.sys_msg_dataquery) def gen_response_dataquery_table(self): messages = [] inputs = {"messages": [HumanMessage(content=self.user_prompt)]} for s in self.agent_executor.stream(inputs, stream_mode="values"): message = s["messages"][-1] if isinstance(message, tuple): logger.info(message) else: message.pretty_print() messages.append(message) ``` ### Error Message and Stack Trace (if applicable) ```shell [2024-11-27 21:43:22,239: WARNING/ForkPoolWorker-1] ================================ Human Message ================================= Show me the data of Italy [2024-11-27 21:43:23,253: INFO/ForkPoolWorker-1] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" [2024-11-27 21:43:31,124: WARNING/ForkPoolWorker-1] ================================== Ai Message ================================== Tool Calls: sql_db_query_checker (call_RGzcL13f3gicJYM8PV9mRFtM) Call ID: call_RGzcL13f3gicJYM8PV9mRFtM Args: query: SELECT * FROM country_codes WHERE Country_name = 'Italy' sql_db_query_checker (call_2tHWQsf8JzFRTwUcwQmhDUGX) Call ID: call_2tHWQsf8JzFRTwUcwQmhDUGX Args: query: SELECT * FROM geo_data WHERE Country = 'ITA' sql_db_query_checker (call_0mZ8gcfp8Ax7sntdwNcDu9AM) Call ID: call_0mZ8gcfp8Ax7sntdwNcDu9AM Args: query: SELECT * FROM italian_cities WHERE Regione = 'Lazio' OR Regione = 'Lombardia' OR Regione = 'Campania' OR Regione = 'Sicilia' OR Regione = 'Veneto' OR Regione = 'Emilia-Romagna' OR Regione = 'Piemonte' OR Regione = 'Toscana' OR Regione = 'Liguria' OR Regione = 'Marche' OR Regione = 'Abruzzo' OR Regione = 'Puglia' OR Regione = 'Calabria' OR Regione = 'Sardegna' OR Regione = 'Friuli-Venezia Giulia' OR Regione = 'Umbria' OR Regione = 'Basilicata' OR Regione = 'Molise' OR Regione = 'Aosta Valley' OR Regione = 'Trentino-Alto Adige' OR Regione = 'Apulia' OR Regione = 'Lazio' OR Regione = 'Lombardia' OR Regione = 'Campania' OR Regione = 'Sicilia' OR Regione = 'Veneto' OR Regione = 'Emilia-Romagna' OR Regione = 'Piemonte' OR Regione = 'Toscana' OR Regione = 'Liguria' OR Regione = 'Marche' OR Regione = 'Abruzzo' OR Regione = 'Puglia' OR Regione = 'Calabria' OR Regione = 'Sardegna' OR Regione = 'Friuli-Venezia Giulia' OR Regione = 'Umbria' OR Regione = 'Basilicata' OR Regione = 'Molise' OR Regione = 'Aosta Valley' OR Regione = 'Trentino-Alto Adige' sql_db_query_checker (call_3nQeSZ7vJCr5GAoCI6vVPfx7) Call ID: call_3nQeSZ7vJCr5GAoCI6vVPfx7 Args: query: SELECT * FROM italian_cities [2024-11-27 21:43:31,878: INFO/ForkPoolWorker-1] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" [2024-11-27 21:43:31,932: INFO/ForkPoolWorker-1] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" [2024-11-27 21:43:33,373: INFO/ForkPoolWorker-1] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" [2024-11-27 21:43:35,026: INFO/ForkPoolWorker-1] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" [2024-11-27 21:43:35,033: WARNING/ForkPoolWorker-1] ================================= Tool Message ================================= Name: sql_db_query_checker sql SELECT * FROM italian_cities [2024-11-27 21:43:46,796: WARNING/ForkPoolWorker-1] ================================== Ai Message ================================== sql SELECT * FROM country_codes WHERE Country_name = 'Italy'; [2024-11-27 21:43:47,205: INFO/ForkPoolWorker-1] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 400 Bad Request" [2024-11-27 21:43:47,214: INFO/ForkPoolWorker-1] Error code: 400 - {'error': {'message': "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_RGzcL13f3gicJYM8PV9mRFtM, call_2tHWQsf8JzFRTwUcwQmhDUGX, call_0mZ8gcfp8Ax7sntdwNcDu9AM", 'type': 'invalid_request_error', 'param': 'messages.[7].role', 'code': None}} ``` ### Description Hi everyone, I am working on a Django app that uses Celery for processing tasks in the background. I am calling a wrapper function of the ReAct agent (create_react_agent) and when the agent performs multiple tool calls inside a single assistant/AI message, it only detects the last tool call and breaks the app with a 400 Error code at the end of the agent streaming. How to solve this? i.e. getting the agent to recognize all the tool calls. When I run it in a Jupyter notebook the cell is executed without any problem while when I run it in the Django app, it breaks the app. Do you have any leads to at least avoid breaking the app? Thanks in advance ### System Info System Information ------------------ > OS: Linux > OS Version: #1 SMP Tue Nov 5 00:21:55 UTC 2024 > Python Version: 3.11.9 (main, May 13 2024, 11:06:43) [GCC 9.4.0] Package Information ------------------- > langchain_core: 0.2.43 > langchain: 0.2.16 > langchain_community: 0.2.17 > langsmith: 0.1.143 > langchain_anthropic: 0.1.23 > langchain_openai: 0.1.10 > langchain_text_splitters: 0.2.4 > langgraph: 0.2.50 Optional packages not installed ------------------------------- > langserve Other Dependencies ------------------ > aiohttp: 3.11.6 > anthropic: 0.39.0 > async-timeout: Installed. No version info available. > dataclasses-json: 0.6.7 > defusedxml: 0.7.1 > httpx: 0.27.2 > jsonpatch: 1.33 > langgraph-checkpoint: 2.0.5 > langgraph-sdk: 0.1.36 > numpy: 1.26.4 > openai: 1.28.1 > orjson: 3.10.11 > packaging: 24.2 > pydantic: 2.9.2 > PyYAML: 6.0.2 > requests: 2.32.3 > requests-toolbelt: 1.0.0 > SQLAlchemy: 2.0.36 > tenacity: 8.5.0 > tiktoken: 0.8.0 > typing-extensions: 4.12.2
yindo closed this issue 2026-02-20 17:36:55 -05:00
yindo commented 2026-02-20 17:36:57 -05:00
Author
Owner
Copy Link
Copy Source

@yyosifov commented on GitHub (Jan 10, 2025):

@daqo98 I just hit the same issue - I see you closed this, do you remember what was the issue and how did you solve it?

@yyosifov commented on GitHub (Jan 10, 2025): @daqo98 I just hit the same issue - I see you closed this, do you remember what was the issue and how did you solve it?
yindo commented 2026-02-20 17:36:59 -05:00
Author
Owner
Copy Link
Copy Source

@daqo98 commented on GitHub (Jan 10, 2025):

Hi @yyosifov, the issue was that I was not keeping track correctly of all the tool messages generated, so when sending the chat history (list of messages) to other interaction with the LLM, the flow crashed. The cause was that the tool responses are generated in parallel, so it is not enough to only append the "last" message, you will miss the other Tool messages.

How did I solve it?
Keeping the index of the last message read and appending one message at a time. Here the code fragment:

messages = []
inputs = {"messages": [HumanMessage(content=self.user_prompt)]}

last_read_idx = -1

for s in self.agent_executor.stream(inputs, stream_mode="values"):
    for message in s["messages"][last_read_idx+1:]:
         
        if isinstance(message, tuple):
            logger.info(message)
        else:
            message.pretty_print()

        messages.append(message)
        last_read_idx += 1

Hope it helps!

@daqo98 commented on GitHub (Jan 10, 2025): Hi @yyosifov, the issue was that I was not keeping track correctly of all the tool messages generated, so when sending the chat history (list of messages) to other interaction with the LLM, the flow crashed. The cause was that the tool responses are generated in parallel, so it is not enough to only append the "last" message, you will miss the other Tool messages. How did I solve it? Keeping the index of the last message read and appending one message at a time. Here the code fragment: ``` messages = [] inputs = {"messages": [HumanMessage(content=self.user_prompt)]} last_read_idx = -1 for s in self.agent_executor.stream(inputs, stream_mode="values"): for message in s["messages"][last_read_idx+1:]: if isinstance(message, tuple): logger.info(message) else: message.pretty_print() messages.append(message) last_read_idx += 1 ``` Hope it helps!
yindo commented 2026-02-20 17:37:01 -05:00
Author
Owner
Copy Link
Copy Source

@vbarda commented on GitHub (Jan 10, 2025):

out of curiosity, why not just use messages = agent_executor.invoke(inputs)["messages"]?

@vbarda commented on GitHub (Jan 10, 2025): out of curiosity, why not just use `messages = agent_executor.invoke(inputs)["messages"]`?
yindo commented 2026-02-20 17:37:04 -05:00
Author
Owner
Copy Link
Copy Source

@daqo98 commented on GitHub (Jan 10, 2025):

Yes, maybe that can also work.
In my case, I wanted to print each message as it was being generated (that's why it is in streaming mode), and while doing that, I appended them to a message list.

Pd. It's curious that when I had the problem nobody replied, and now after I posted my solution (because someone asked for it), just 15 minutes later, someone try to give another solution... Internet :)

@daqo98 commented on GitHub (Jan 10, 2025): Yes, maybe that can also work. In my case, I wanted to print each message as it was being generated (that's why it is in streaming mode), and while doing that, I appended them to a message list. Pd. It's curious that when I had the problem nobody replied, and now after I posted my solution (because someone asked for it), just 15 minutes later, someone try to give another solution... Internet :)
yindo commented 2026-02-20 17:37:06 -05:00
Author
Owner
Copy Link
Copy Source

@yyosifov commented on GitHub (Jan 11, 2025):

@daqo98 Thanks for the information. In my case I have decided to strip the tools messages to reduce the "tokens" sent to openai, but I guess I didn't clean them correctly and the result was the same error as the one you encountered.

@yyosifov commented on GitHub (Jan 11, 2025): @daqo98 Thanks for the information. In my case I have decided to strip the tools messages to reduce the "tokens" sent to openai, but I guess I didn't clean them correctly and the result was the same error as the one you encountered.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agents
awaiting author revision
bug
change
chat models
checkpointer
ci
code quality
code quality
dependencies
documentation
drawing
duplicate
enhancement
good first issue
help wanted
invalid
maintainer
meta
missing information
not maintained by langchain
open-swe-auto
open-swe-max
open-swe-max-auto
pending
performance
pull-request
Mirrored from GitHub Pull Request
 question
requires design
sdk
server
stale
studio
tests
type checking
usage question
v0.5
v1.1
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraph#333
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?