langchain-ai/langgraphjs
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraphjs.git synced 2026-07-22 00:55:26 -04:00
Code Issues 108 Packages Projects Releases 20 Wiki Activity

New ToolNode and ChatGroq doesn't work for Multi-calling tools #84

New Issue
Closed
opened 2026-02-15 17:15:33 -05:00 by yindo · 7 comments
yindo commented 2026-02-15 17:15:33 -05:00
Owner
Copy Link
Copy Source

Originally created by @hgoona on GitHub (Sep 7, 2024).

Hey @bracesproul @hinthornw @jacoblee93 - I noted you guys have already reviewed some existing issues on github regarding the new ToolNode - have you experienced the following with ChatGroq? 👇🏾👇🏾👇🏾
(Deno code and LangSmith trace below - would greatly appreciate your help🙏🏾)

I've previously been using ToolExecutor (deprecated) with my LangGraph using ChatGroq - this has been working.

I'm now trying to implement the new ToolNode to replace the ToolExecutor, however, this gives errors/fails to complete when triggering the use of multiple tools, or the same tool used multiple times in an LLM query.

Here is a test I've run in Deno:

import { ChatGroq } from "npm:@langchain/groq";
const modelGroq = new ChatGroq({
	temperature: 0,
	apiKey: GROQ_API_KEY,
	model: "llama-3.1-8b-instant",
	streaming: true,
	// streaming: false,
});

import { ChatOpenAI } from "npm:@langchain/openai";
const modelOAI = new ChatOpenAI({
	temperature: 0,
	model: "gpt-3.5-turbo-0125",
	apiKey: OPENAI_API_KEY
	// streaming: true,
});

import {
  StateGraph,
  MessagesAnnotation,
} from "npm:@langchain/langgraph";
import { ToolNode } from "npm:@langchain/langgraph/prebuilt"; // ❗ SEE NOTE BELOW👇🏾👇🏾👇🏾
import { tool } from "npm:@langchain/core/tools";
import { z } from "npm:zod";
import { Calculator } from "npm:@langchain/community/tools/calculator";

const tools = [
  new Calculator() 
];

const modelWithTools = modelGroq.bindTools(tools); //❌fails multi-call
// const modelWithTools = modelOAI.bindTools(tools); //✅works withmulti-call using 

const toolNodeForGraph = new ToolNode(tools)

const shouldContinue = (state: typeof MessagesAnnotation.State) => {
  const { messages } = state;
  const lastMessage = messages[messages.length - 1];
  if ("tool_calls" in lastMessage && Array.isArray(lastMessage.tool_calls) && lastMessage.tool_calls?.length) {
    return "tools";
  }
  return "__end__";
}

const callModel = async (state: typeof MessagesAnnotation.State) => {
  const { messages } = state;
  const response = await modelWithTools.invoke(messages);
  return { messages: response };
}

const graph = new StateGraph(MessagesAnnotation)
  .addNode("agent", callModel)
  .addNode("tools", toolNodeForGraph)
  .addEdge("__start__", "agent")
  .addConditionalEdges("agent", shouldContinue)
  .addEdge("tools", "agent")
  .compile();

const inputs = {
  // messages: [{ role: "user", content: "what is 2 plus 5? Use a tool to calculate this" }],
  messages: [{ role: "user", content: "what is 2 plus 5, and also, 18 into half? Use a tool to calculate this" }],
};

const stream = await graph.stream(inputs, {
  streamMode: "values",
});

for await (const { messages } of stream) {
  console.log(messages);
}```
Originally created by @hgoona on GitHub (Sep 7, 2024). Hey @bracesproul @hinthornw @jacoblee93 - I noted you guys have already reviewed some existing issues on github regarding the new `ToolNode` - have you experienced the following with `ChatGroq`? 👇🏾👇🏾👇🏾 (Deno code and LangSmith trace below - would greatly appreciate your help🙏🏾) I've previously been using `ToolExecutor` (deprecated) with my LangGraph using ChatGroq - this has been working. I'm now trying to implement the new `ToolNode` to replace the `ToolExecutor`, however, this gives errors/fails to complete when triggering the use of multiple tools, or the same tool used multiple times in an LLM query. Here is a test I've run in Deno: ```ts import { ChatGroq } from "npm:@langchain/groq"; const modelGroq = new ChatGroq({ temperature: 0, apiKey: GROQ_API_KEY, model: "llama-3.1-8b-instant", streaming: true, // streaming: false, }); import { ChatOpenAI } from "npm:@langchain/openai"; const modelOAI = new ChatOpenAI({ temperature: 0, model: "gpt-3.5-turbo-0125", apiKey: OPENAI_API_KEY // streaming: true, }); ``` ```ts import { StateGraph, MessagesAnnotation, } from "npm:@langchain/langgraph"; import { ToolNode } from "npm:@langchain/langgraph/prebuilt"; // ❗ SEE NOTE BELOW👇🏾👇🏾👇🏾 import { tool } from "npm:@langchain/core/tools"; import { z } from "npm:zod"; import { Calculator } from "npm:@langchain/community/tools/calculator"; const tools = [ new Calculator() ]; const modelWithTools = modelGroq.bindTools(tools); //❌fails multi-call // const modelWithTools = modelOAI.bindTools(tools); //✅works withmulti-call using const toolNodeForGraph = new ToolNode(tools) const shouldContinue = (state: typeof MessagesAnnotation.State) => { const { messages } = state; const lastMessage = messages[messages.length - 1]; if ("tool_calls" in lastMessage && Array.isArray(lastMessage.tool_calls) && lastMessage.tool_calls?.length) { return "tools"; } return "__end__"; } const callModel = async (state: typeof MessagesAnnotation.State) => { const { messages } = state; const response = await modelWithTools.invoke(messages); return { messages: response }; } const graph = new StateGraph(MessagesAnnotation) .addNode("agent", callModel) .addNode("tools", toolNodeForGraph) .addEdge("__start__", "agent") .addConditionalEdges("agent", shouldContinue) .addEdge("tools", "agent") .compile(); const inputs = { // messages: [{ role: "user", content: "what is 2 plus 5? Use a tool to calculate this" }], messages: [{ role: "user", content: "what is 2 plus 5, and also, 18 into half? Use a tool to calculate this" }], }; const stream = await graph.stream(inputs, { streamMode: "values", }); for await (const { messages } of stream) { console.log(messages); }```
yindo added the enhancementgood first issue labels 2026-02-15 17:15:33 -05:00
yindo closed this issue 2026-02-15 17:15:33 -05:00
yindo commented 2026-02-15 17:15:34 -05:00
Author
Owner
Copy Link
Copy Source

@hgoona commented on GitHub (Sep 7, 2024):

LangSmith traces:
❌ChatGroq + ToolNode fails with Multi-tool call: https://smith.langchain.com/public/baf963bb-89c4-4f50-8729-6aed4f6448fe/r
✅ChatGroq + ToolNode works with Single-tool call:
https://smith.langchain.com/public/cd4c7ea2-5fae-468f-919f-e1582e9b8a20/r

@hgoona commented on GitHub (Sep 7, 2024): LangSmith traces: ❌ChatGroq + ToolNode fails with **Multi-tool** call: https://smith.langchain.com/public/baf963bb-89c4-4f50-8729-6aed4f6448fe/r ✅ChatGroq + ToolNode works with **Single-tool** call: https://smith.langchain.com/public/cd4c7ea2-5fae-468f-919f-e1582e9b8a20/r
yindo commented 2026-02-15 17:15:34 -05:00
Author
Owner
Copy Link
Copy Source

@hinthornw commented on GitHub (Sep 8, 2024):

Hm maybe I'm reading that incorrectly but it looks from that trace like the error is constrained to the ChatGroq class (the generation fails) and isn't related to ToolNode or LangGraph?

@hinthornw commented on GitHub (Sep 8, 2024): Hm maybe I'm reading that incorrectly but it looks from that trace like the error is constrained to the ChatGroq class (the generation fails) and isn't related to ToolNode or LangGraph?
yindo commented 2026-02-15 17:15:34 -05:00
Author
Owner
Copy Link
Copy Source

@hgoona commented on GitHub (Sep 8, 2024):

Hm maybe I'm reading that incorrectly but it looks from that trace like the error is constrained to the ChatGroq class (the generation fails) and isn't related to ToolNode or LangGraph?

@hinthornw If I used the ChatOpenAI in the test it seems to work, so yes - I think the issue might be originating from the ChatGroq class, HOWEVER I did note this message when logging state in my application👇🏾👇🏾
ChatGroq does
✅correctly generate mutli-tool calls inside kwargs.additional_kwargs.tool_calls <<<why is this correct? but it
❌incorrectly generates kwargs.tool_calls as an empty [ ]
and tool_call_chunks is an invalid tool call inside the same state update?
The log of state, below:

"messages": [
    {
      "lc": 1,
      "type": "constructor",
      "id": [
        "langchain_core",
        "messages",
        "AIMessageChunk"
      ],
      "kwargs": {
        "content": "",
        "additional_kwargs": {
          "tool_calls": [
            {
              "id": "call_jd3k",
              "type": "function",
              "function": {
                "name": "calculator",
                "arguments": "{\"input\": \"3*6+2\"}"
              },
              "index": 0
            },
            {
              "id": "call_gsfa",
              "type": "function",
              "function": {
                "name": "calculator",
                "arguments": "{\"input\": \"346/2\"}"
              },
              "index": 1
            }
          ]
        },
        "response_metadata": {
          "id": "chatcmpl-20fb28ba-9901-4540-97ed-30c59241f3c5",
          "object": "chat.completion.chunk",
          "created": 1725777315,
          "model": "llama-3.1-8b-instant",
          "system_fingerprint": "fp_f66ccb39ec",
          "x_groq": {
            "id": "req_01j785g1yye4wa6ezj13szn28y",
            "usage": {
              "queue_time": 0.0025225389999999903,
              "prompt_tokens": 461,
              "prompt_time": 0.094258511,
              "completion_tokens": 33,
              "completion_time": 0.044,
              "total_tokens": 494,
              "total_time": 0.138258511
            }
          }
        },
        "tool_call_chunks": [
          {
            "id": "call_jd3kcall_gsfa",
            "name": "calculatorcalculator",
            "index": 0,
            "type": "tool_call_chunk",
            "args": "{\"input\": \"3*6+2\"}{\"input\": \"346/2\"}"
          }
        ],
        "id": "run-ff55a37e-daa7-41d1-92fd-ac949f99b230",
        "usage_metadata": {
          "input_tokens": 461,
          "output_tokens": 33,
          "total_tokens": 494
        },
        "tool_calls": [],
        "invalid_tool_calls": [
          {
            "name": "calculatorcalculator",
            "args": "{\"input\": \"3*6+2\"}{\"input\": \"346/2\"}",
            "id": "call_jd3kcall_gsfa",
            "error": "Malformed args.",
            "type": "invalid_tool_call"
          }
        ]
      }
    }
  ]
@hgoona commented on GitHub (Sep 8, 2024): > Hm maybe I'm reading that incorrectly but it looks from that trace like the error is constrained to the ChatGroq class (the generation fails) and isn't related to ToolNode or LangGraph? @hinthornw If I used the `ChatOpenAI` in the test it seems to work, so yes - I think the issue might be originating from the `ChatGroq` class, HOWEVER I did note this `message` when logging state in my application👇🏾👇🏾 ChatGroq does ✅correctly generate mutli-tool calls inside `kwargs.additional_kwargs.tool_calls` <<<why is this correct? but it ❌incorrectly generates `kwargs.tool_calls` as an empty `[ ]` and `tool_call_chunks` is an `invalid` tool call inside the same state update? The log of state, below: ```json "messages": [ { "lc": 1, "type": "constructor", "id": [ "langchain_core", "messages", "AIMessageChunk" ], "kwargs": { "content": "", "additional_kwargs": { "tool_calls": [ { "id": "call_jd3k", "type": "function", "function": { "name": "calculator", "arguments": "{\"input\": \"3*6+2\"}" }, "index": 0 }, { "id": "call_gsfa", "type": "function", "function": { "name": "calculator", "arguments": "{\"input\": \"346/2\"}" }, "index": 1 } ] }, "response_metadata": { "id": "chatcmpl-20fb28ba-9901-4540-97ed-30c59241f3c5", "object": "chat.completion.chunk", "created": 1725777315, "model": "llama-3.1-8b-instant", "system_fingerprint": "fp_f66ccb39ec", "x_groq": { "id": "req_01j785g1yye4wa6ezj13szn28y", "usage": { "queue_time": 0.0025225389999999903, "prompt_tokens": 461, "prompt_time": 0.094258511, "completion_tokens": 33, "completion_time": 0.044, "total_tokens": 494, "total_time": 0.138258511 } } }, "tool_call_chunks": [ { "id": "call_jd3kcall_gsfa", "name": "calculatorcalculator", "index": 0, "type": "tool_call_chunk", "args": "{\"input\": \"3*6+2\"}{\"input\": \"346/2\"}" } ], "id": "run-ff55a37e-daa7-41d1-92fd-ac949f99b230", "usage_metadata": { "input_tokens": 461, "output_tokens": 33, "total_tokens": 494 }, "tool_calls": [], "invalid_tool_calls": [ { "name": "calculatorcalculator", "args": "{\"input\": \"3*6+2\"}{\"input\": \"346/2\"}", "id": "call_jd3kcall_gsfa", "error": "Malformed args.", "type": "invalid_tool_call" } ] } } ] ```
yindo commented 2026-02-15 17:15:34 -05:00
Author
Owner
Copy Link
Copy Source

@hgoona commented on GitHub (Sep 8, 2024):

Btw @hinthornw Is this the right place to post/solve this or should I repost it on LangChain/Groq? 😬 If so, any idea who I can direct this to? - would greatly appreciate any help!

@hgoona commented on GitHub (Sep 8, 2024): Btw @hinthornw Is this the right place to post/solve this or should I repost it on LangChain/Groq? 😬 If so, any idea who I can direct this to? - would greatly appreciate any help!
yindo commented 2026-02-15 17:15:35 -05:00
Author
Owner
Copy Link
Copy Source

@jacoblee93 commented on GitHub (Sep 8, 2024):

This is a good place since it looks ToolNode related, will dig in.

@jacoblee93 commented on GitHub (Sep 8, 2024): This is a good place since it looks ToolNode related, will dig in.
yindo commented 2026-02-15 17:15:35 -05:00
Author
Owner
Copy Link
Copy Source

@hgoona commented on GitHub (Sep 9, 2024):

This is a good place since it looks ToolNode related, will dig in.

Any luck with this @jacoblee93 ? I'm also noticing the token usage is (still) stored in a odd/different manner to ChatOpenAI..

@hgoona commented on GitHub (Sep 9, 2024): > This is a good place since it looks ToolNode related, will dig in. Any luck with this @jacoblee93 ? I'm also noticing the token usage is (still) stored in a odd/different manner to ChatOpenAI..
yindo commented 2026-02-15 17:15:35 -05:00
Author
Owner
Copy Link
Copy Source

@dqbd commented on GitHub (Sep 11, 2025):

Hello! Investigating further, I think the issue is not coming from ToolNode, rather than llama-3.1-8b-instant does not handle parallel tool calls well. Swapping the model to openai/gpt-oss-20b makes parallel tool calling work properly.

Closing this issue, let me know if the issue needs to be reopened again

@dqbd commented on GitHub (Sep 11, 2025): Hello! Investigating further, I think the issue is not coming from ToolNode, rather than `llama-3.1-8b-instant` does not handle parallel tool calls well. Swapping the model to `openai/gpt-oss-20b` makes parallel tool calling work properly. Closing this issue, let me know if the issue needs to be reopened again
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
checkpointer
docs-needed
documentation
enhancement
good first issue
good writeup
help wanted
invalid
langgraph-supervisor
open-swe-auto
platform
prebuilt
pull-request
Mirrored from GitHub Pull Request
 question
No Label enhancement good first issue
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraphjs#84
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?