langchain-ai/langgraphjs
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraphjs.git synced 2026-07-22 09:05:28 -04:00
Code Issues 108 Packages Projects Releases 20 Wiki Activity

Bedrock: Passing the "responseFormat" into "createReactAgent" function sometimes leads to the "invoke()" error #210

New Issue
Closed
opened 2026-02-15 17:17:21 -05:00 by yindo · 12 comments
yindo commented 2026-02-15 17:17:21 -05:00
Owner
Copy Link
Copy Source

Originally created by @vlenko-eq on GitHub (Mar 21, 2025).

Passing the responseFormat parameter into createReactAgent function sometimes leads to the invoke() error:

⨯ Error: No tool calls found in the response.
    at RunnableLambda.eval [as func] (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/language_models/chat_models.js:561:23)
    at eval (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:1714:45)
    at MockAsyncLocalStorage.run (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/singletons/async_local_storage/index.js:18:16)
    at AsyncLocalStorageProvider.runWithConfig (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/singletons/async_local_storage/index.js:64:24)
    at eval (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:1712:104)
    at new Promise (<anonymous>)
    at RunnableLambda._invoke (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:1707:16)
    at RunnableLambda._callWithConfig (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:253:34)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async RunnableSequence.invoke (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:1317:27)
    at async RunnableCallable.generateStructuredResponse [as func] (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/prebuilt/react_agent_executor.js:242:26)
    at async RunnableCallable.invoke (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/utils.js:92:27)
    at async RunnableSequence.invoke (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:1311:33)
    at async _runWithRetry (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/pregel/retry.js:80:22)
    at async PregelRunner._executeTasksWithRetry (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/pregel/runner.js:228:33)
    at async PregelRunner.tick (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/pregel/runner.js:53:40)
    at async CompiledStateGraph._runLoop (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/pregel/index.js:1310:17)
    at async createAndRunLoop (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/pregel/index.js:1209:17) {
  pregelTaskId: '3e34fdfd-e930-55ca-a4e8-57294177e95b'
}

Agent configuration:

...
const llm = new BedrockChat({ model: "anthropic.claude-3-7-sonnet-20250219-v1:0", temperature: 0 });
const responseFormat = z.object({ a: z.string().describe("Answer"), b: z.string().describe("Explanation") });
const tools = new SqlToolkit(db, llm).getTools();
const agent = createReactAgent({ llm, tools, prompt, responseFormat });
const agentState = await agent.invoke({ messages: [new HumanMessage(query)] });

Response from ChatModel that leads to the error:

AIMessage {
  "content": "Some content....",
  "additional_kwargs": {
    "id": "msg_bdrk_....."
  },
  "response_metadata": {
    "type": "message",
    "role": "assistant",
    "model": "claude-3-7-sonnet-20250219",
    "stop_reason": "end_turn",
    "stop_sequence": null,
    "usage": {
      "input_tokens": 1257,
      "cache_creation_input_tokens": 0,
      "cache_read_input_tokens": 0,
      "output_tokens": 107
    }
  },
  "tool_calls": [],
  "invalid_tool_calls": []
}
Originally created by @vlenko-eq on GitHub (Mar 21, 2025). Passing the `responseFormat` parameter into `createReactAgent` function sometimes leads to the `invoke()` error: ``` ⨯ Error: No tool calls found in the response. at RunnableLambda.eval [as func] (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/language_models/chat_models.js:561:23) at eval (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:1714:45) at MockAsyncLocalStorage.run (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/singletons/async_local_storage/index.js:18:16) at AsyncLocalStorageProvider.runWithConfig (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/singletons/async_local_storage/index.js:64:24) at eval (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:1712:104) at new Promise (<anonymous>) at RunnableLambda._invoke (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:1707:16) at RunnableLambda._callWithConfig (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:253:34) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) at async RunnableSequence.invoke (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:1317:27) at async RunnableCallable.generateStructuredResponse [as func] (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/prebuilt/react_agent_executor.js:242:26) at async RunnableCallable.invoke (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/utils.js:92:27) at async RunnableSequence.invoke (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:1311:33) at async _runWithRetry (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/pregel/retry.js:80:22) at async PregelRunner._executeTasksWithRetry (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/pregel/runner.js:228:33) at async PregelRunner.tick (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/pregel/runner.js:53:40) at async CompiledStateGraph._runLoop (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/pregel/index.js:1310:17) at async createAndRunLoop (webpack-internal:///(rsc)/./node_modules/@langchain/langgraph/dist/pregel/index.js:1209:17) { pregelTaskId: '3e34fdfd-e930-55ca-a4e8-57294177e95b' } ``` Agent configuration: ``` ... const llm = new BedrockChat({ model: "anthropic.claude-3-7-sonnet-20250219-v1:0", temperature: 0 }); const responseFormat = z.object({ a: z.string().describe("Answer"), b: z.string().describe("Explanation") }); const tools = new SqlToolkit(db, llm).getTools(); const agent = createReactAgent({ llm, tools, prompt, responseFormat }); const agentState = await agent.invoke({ messages: [new HumanMessage(query)] }); ``` Response from ChatModel that leads to the error: ``` AIMessage { "content": "Some content....", "additional_kwargs": { "id": "msg_bdrk_....." }, "response_metadata": { "type": "message", "role": "assistant", "model": "claude-3-7-sonnet-20250219", "stop_reason": "end_turn", "stop_sequence": null, "usage": { "input_tokens": 1257, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0, "output_tokens": 107 } }, "tool_calls": [], "invalid_tool_calls": [] } ```
yindo added the prebuilt label 2026-02-15 17:17:21 -05:00
yindo closed this issue 2026-02-15 17:17:21 -05:00
yindo commented 2026-02-15 17:17:22 -05:00
Author
Owner
Copy Link
Copy Source

@vbarda commented on GitHub (Mar 21, 2025):

does that happen with any other model providers or only with Bedrock? this looks like a Bedrock integration issue to me

@vbarda commented on GitHub (Mar 21, 2025): does that happen with any other model providers or only with Bedrock? this looks like a Bedrock integration issue to me
yindo commented 2026-02-15 17:17:22 -05:00
Author
Owner
Copy Link
Copy Source

@vlenko-eq commented on GitHub (Mar 21, 2025):

The error does not occur on "gpt-4o-mini". Looks like a Bedrock integration issue.

@vlenko-eq commented on GitHub (Mar 21, 2025): The error does not occur on "gpt-4o-mini". Looks like a Bedrock integration issue.
yindo commented 2026-02-15 17:17:22 -05:00
Author
Owner
Copy Link
Copy Source

@cwoolum commented on GitHub (Apr 21, 2025):

I think I'm running into a similar issue. For some reason, with a basic ReACT agent, I'm no longer able to successfully return a response from invoke.

const agent = createReactAgent({
    llm,
    tools: [...toolDefs],
    name: 'party_planning_expert',
    responseFormat: partyPlanningResponseSchema,
    prompt: PARTY_SYSTEM_PROMPT_TEMPLATE,
  });

await agent.invoke({
    messages: partyPlanningPromptTemplate,
}

Looking at the streaming response, The tools are invoked successfully but when the agent goes to return its response, it fails with Your API request included an assistantmessage in the final position, which would pre-fill theassistantresponse. When using tools, pre-filling theassistant response is not supported..

This used to work so I wonder if there was a recent change that broke this?

EDIT: Did some more testing and it seems to fail with anthropic.claude-3-5-sonnet-20241022-v2:0 but not anthropic.claude-3-7-sonnet-20250219-v1:0. There's no reason we should need Claude 3.7 to handle passing back responses though..

@cwoolum commented on GitHub (Apr 21, 2025): I think I'm running into a similar issue. For some reason, with a basic ReACT agent, I'm no longer able to successfully return a response from invoke. ``` const agent = createReactAgent({ llm, tools: [...toolDefs], name: 'party_planning_expert', responseFormat: partyPlanningResponseSchema, prompt: PARTY_SYSTEM_PROMPT_TEMPLATE, }); await agent.invoke({ messages: partyPlanningPromptTemplate, } ``` Looking at the streaming response, The tools are invoked successfully but when the agent goes to return its response, it fails with `Your API request included an `assistant` message in the final position, which would pre-fill the `assistant` response. When using tools, pre-filling the `assistant` response is not supported.`. This used to work so I wonder if there was a recent change that broke this? EDIT: Did some more testing and it seems to fail with `anthropic.claude-3-5-sonnet-20241022-v2:0` but not `anthropic.claude-3-7-sonnet-20250219-v1:0`. There's no reason we should need Claude 3.7 to handle passing back responses though..
yindo commented 2026-02-15 17:17:22 -05:00
Author
Owner
Copy Link
Copy Source

@cwoolum commented on GitHub (Apr 21, 2025):

Removing responseFormat does fix the issue so it does seem this is some issue going on with using responseFormat and ChatBedrockConverse together.

@cwoolum commented on GitHub (Apr 21, 2025): Removing `responseFormat` does fix the issue so it does seem this is some issue going on with using `responseFormat ` and `ChatBedrockConverse` together.
yindo commented 2026-02-15 17:17:22 -05:00
Author
Owner
Copy Link
Copy Source

@cwoolum commented on GitHub (Apr 21, 2025):

It looks like https://github.com/langchain-ai/langgraphjs/pull/1050 introduced this issue. When reverting const messages = state.messages.slice(0, -1); to const messages = [...state.messages];, the issue no longer occurs.

@cwoolum commented on GitHub (Apr 21, 2025): It looks like https://github.com/langchain-ai/langgraphjs/pull/1050 introduced this issue. When reverting `const messages = state.messages.slice(0, -1);` to `const messages = [...state.messages];`, the issue no longer occurs.
yindo commented 2026-02-15 17:17:23 -05:00
Author
Owner
Copy Link
Copy Source

@vbarda commented on GitHub (Apr 21, 2025):

@cwoolum do you mean reverting to const messages = state.messages.slice(0, -1) fixes the issue? it might be bedrock-specific issue, since it doesn't happen with anthropic API

@vbarda commented on GitHub (Apr 21, 2025): @cwoolum do you mean reverting to `const messages = state.messages.slice(0, -1)` fixes the issue? it might be bedrock-specific issue, since it doesn't happen with anthropic API
yindo commented 2026-02-15 17:17:23 -05:00
Author
Owner
Copy Link
Copy Source

@cwoolum commented on GitHub (Apr 21, 2025):

Yes, before that PR, the message order looked like this

['[object SystemMessage]', '[object HumanMessage]', '[object AIMessageChunk]', '[object ToolMessage]', '[object AIMessageChunk]', '[object ToolMessage]']

Now, since it doesn't remove the last item, it looks like this

['[object SystemMessage]', '[object HumanMessage]', '[object AIMessageChunk]', '[object ToolMessage]', '[object AIMessageChunk]', '[object ToolMessage]', '[object AIMessageChunk]']

When the LLM is invoked, the last message is an assistant message. This shouldn't be possible with the direct Anthropic either. Shouldn't the last message should always be some sort of user message?

Found the docs. Putting an assistant message at the end if the equivalent of putting words in Claude's mouth on the last message. It appears this functionality doesn't translate to Bedrock. Do Gemini and OpenAPI support this pattern? I can't find any docs on it.

I'll put in a ticket with the Bedrock team. On the LangGraph side, is there a more universal solution that could be used? Maybe appending another user message asking to use the structured output?

@cwoolum commented on GitHub (Apr 21, 2025): Yes, before that PR, the message order looked like this ``` ['[object SystemMessage]', '[object HumanMessage]', '[object AIMessageChunk]', '[object ToolMessage]', '[object AIMessageChunk]', '[object ToolMessage]'] ``` Now, since it doesn't remove the last item, it looks like this ``` ['[object SystemMessage]', '[object HumanMessage]', '[object AIMessageChunk]', '[object ToolMessage]', '[object AIMessageChunk]', '[object ToolMessage]', '[object AIMessageChunk]'] ``` When the LLM is invoked, the last message is an assistant message. ~This shouldn't be possible with the direct Anthropic either. Shouldn't the last message should always be some sort of user message?~ Found the [docs](https://docs.anthropic.com/en/api/migrating-from-text-completions-to-messages#putting-words-in-claudes-mouth). Putting an assistant message at the end if the equivalent of putting words in Claude's mouth on the last message. It appears this functionality doesn't translate to Bedrock. Do Gemini and OpenAPI support this pattern? I can't find any docs on it. I'll put in a ticket with the Bedrock team. On the LangGraph side, is there a more universal solution that could be used? Maybe appending another user message asking to use the structured output?
yindo commented 2026-02-15 17:17:23 -05:00
Author
Owner
Copy Link
Copy Source

@vbarda commented on GitHub (Apr 21, 2025):

This works with all major providers - OAI, Gemini, Anthropic - but i don't believe that OAI/Gemini do any prefilling. I don't think this actually has the prefilling effect for Anthropic either in this case - it matters more when you supply a single incomplete AI message, or (user message, incomplete AI message), not the full history like we do in react agent.

We don't have any other ready solutions at the moment besides making a custom react agent graph from scratch (you could just copy the implementation and make necessary modifications)

We are planning to add another structured output mode to createReactAgent that relies on tool calling rather than a separate structured output, but don't have a timeline for that yet

@vbarda commented on GitHub (Apr 21, 2025): This works with all major providers - OAI, Gemini, Anthropic - but i don't believe that OAI/Gemini do any prefilling. I don't think this actually has the prefilling effect for Anthropic either in this case - it matters more when you supply a single incomplete AI message, or (user message, incomplete AI message), not the full history like we do in react agent. We don't have any other ready solutions at the moment besides making a custom react agent graph from scratch (you could just copy the implementation and make necessary modifications) We are planning to add another structured output mode to `createReactAgent` that relies on tool calling rather than a separate structured output, but don't have a timeline for that yet
yindo commented 2026-02-15 17:17:23 -05:00
Author
Owner
Copy Link
Copy Source

@vbarda commented on GitHub (Apr 21, 2025):

Going to close in the meantime, since the original issue is not a LangGraph issue.

@vbarda commented on GitHub (Apr 21, 2025): Going to close in the meantime, since the original issue is not a LangGraph issue.
yindo commented 2026-02-15 17:17:23 -05:00
Author
Owner
Copy Link
Copy Source

@cwoolum commented on GitHub (Apr 22, 2025):

Thanks for the clarification. I'll follow-up on the Bedrock side and let you know what comes of it.

@cwoolum commented on GitHub (Apr 22, 2025): Thanks for the clarification. I'll follow-up on the Bedrock side and let you know what comes of it.
yindo commented 2026-02-15 17:17:23 -05:00
Author
Owner
Copy Link
Copy Source

@MrRoiz commented on GitHub (Dec 12, 2025):

Hey @cwoolum have you got any news from Bedrock? having the same issue here.

@MrRoiz commented on GitHub (Dec 12, 2025): Hey @cwoolum have you got any news from Bedrock? having the same issue here.
yindo commented 2026-02-15 17:17:24 -05:00
Author
Owner
Copy Link
Copy Source

@cwoolum commented on GitHub (Dec 12, 2025):

Sorry, I'm no longer with AWS

@cwoolum commented on GitHub (Dec 12, 2025): Sorry, I'm no longer with AWS
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
checkpointer
docs-needed
documentation
enhancement
good first issue
good writeup
help wanted
invalid
langgraph-supervisor
open-swe-auto
platform
prebuilt
pull-request
Mirrored from GitHub Pull Request
 question
No Label prebuilt
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraphjs#210
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?