langchain-ai/langgraphjs
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraphjs.git synced 2026-07-22 17:15:25 -04:00
Code Issues 108 Packages Projects Releases 20 Wiki Activity

When generating tool call parameters, on_chat_model_stream is erroneously sent #58

New Issue
Closed
opened 2026-02-15 17:15:11 -05:00 by yindo · 4 comments
yindo commented 2026-02-15 17:15:11 -05:00
Owner
Copy Link
Copy Source

Originally created by @renxinyan on GitHub (Aug 12, 2024).

I would like to obtain the generated token in real-time through the streamEvent(), but when using anthropic and ToolNode, the workflow sends the on_chat_model_stream event when generating tool call parameters, and there is no additional information to distinguish whether it is generating ordinary text or tool call parameters.

Code:

import { AIMessage, BaseMessage, HumanMessage } from "@langchain/core/messages";
import { tool } from "@langchain/core/tools";
import { ChatAnthropic } from "@langchain/anthropic";
import { StateGraph, StateGraphArgs } from "@langchain/langgraph";
import { ToolNode } from "@langchain/langgraph/prebuilt";

const weatherTool = tool(
  (_) => "The weather in San Francisco is 70 degrees and sunny.",
  {
    name: "get_current_weather",
    schema: {
      type: 'object',
      properties: {
        location: {
          type: 'string',
        },
      },
      required: ['location'],
    },
    description: "Get the current weather for a location.",
  }
);

const tools = [weatherTool];

const model = new ChatAnthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
  model: 'claude-3-haiku-20240307',
}).bindTools(tools);

interface AgentState {
  messages: BaseMessage[];
}

const graphState: StateGraphArgs<AgentState>["channels"] = {
  messages: {
    reducer: (x: BaseMessage[], y: BaseMessage[]) => x.concat(y),
  },
};

const toolNode = new ToolNode<AgentState>(tools);

function shouldContinue(state: AgentState) {
  const messages = state.messages;
  const lastMessage = messages[messages.length - 1] as AIMessage;

  if (lastMessage.tool_calls?.length) {
    return "tools";
  }
  return "__end__";
}

async function callModel(state: AgentState) {
  const messages = state.messages;
  const response = await model.invoke(messages);

  return { messages: [response] };
}

const workflow = new StateGraph<AgentState>({ channels: graphState })
  .addNode("agent", callModel)
  .addNode("tools", toolNode)
  .addEdge("__start__", "agent")
  .addConditionalEdges("agent", shouldContinue)
  .addEdge("tools", "agent");

const app = workflow.compile();

async function run() {
  const response = await app.streamEvents(
    { messages: [new HumanMessage("what is the weather in sf")] },
    {
      version: 'v2',
    }
  );

  for await (const event of response) {
    console.log(event);
  }
}

run();

Output:

....
{
  "event": "on_chat_model_stream",
  "data": {
    "chunk": { 
      "lc": 1,
      "type": "constructor",
      "id": [
        "langchain_core",
        "messages",
        "AIMessageChunk"
      ],
      "kwargs": {
        "content": "{\"input",
        "tool_calls": [],
        "invalid_tool_calls": [],
        "tool_call_chunks": [],
        "additional_kwargs": {},
        "response_metadata": {}
      }
    }
  },
  "run_id": "a69b3dab-66c5-48a5-9086-5c2197074a66",
  "name": "ChatAnthropic",
  "tags": [],
  "metadata": {
    "langgraph_step": 1,
    "langgraph_node": "agent",
    "langgraph_triggers": [
      "__pregel_tasks"
    ],
    "langgraph_task_idx": 0,
    "ls_provider": "anthropic",
    "ls_model_name": "claude-3-haiku-20240307",
    "ls_model_type": "chat",
    "ls_temperature": 1,
    "ls_max_tokens": 2048
  }
}
....

You can see that the content of data.chunk.kwargs.content is "{"input", which is part of the tool call parameters. I hope that when generating tool call parameters, the on_chat_model_stream event is not sent, or there is a way to determine whether it is generating ordinary text or tool call parameters.
If using the OpenAI model, it meets my expectations.

Originally created by @renxinyan on GitHub (Aug 12, 2024). I would like to obtain the generated token in real-time through the `streamEvent()`, but when using anthropic and ToolNode, the workflow sends the `on_chat_model_stream` event when generating tool call parameters, and there is no additional information to distinguish whether it is generating ordinary text or tool call parameters. Code: ```ts import { AIMessage, BaseMessage, HumanMessage } from "@langchain/core/messages"; import { tool } from "@langchain/core/tools"; import { ChatAnthropic } from "@langchain/anthropic"; import { StateGraph, StateGraphArgs } from "@langchain/langgraph"; import { ToolNode } from "@langchain/langgraph/prebuilt"; const weatherTool = tool( (_) => "The weather in San Francisco is 70 degrees and sunny.", { name: "get_current_weather", schema: { type: 'object', properties: { location: { type: 'string', }, }, required: ['location'], }, description: "Get the current weather for a location.", } ); const tools = [weatherTool]; const model = new ChatAnthropic({ apiKey: process.env.ANTHROPIC_API_KEY, model: 'claude-3-haiku-20240307', }).bindTools(tools); interface AgentState { messages: BaseMessage[]; } const graphState: StateGraphArgs<AgentState>["channels"] = { messages: { reducer: (x: BaseMessage[], y: BaseMessage[]) => x.concat(y), }, }; const toolNode = new ToolNode<AgentState>(tools); function shouldContinue(state: AgentState) { const messages = state.messages; const lastMessage = messages[messages.length - 1] as AIMessage; if (lastMessage.tool_calls?.length) { return "tools"; } return "__end__"; } async function callModel(state: AgentState) { const messages = state.messages; const response = await model.invoke(messages); return { messages: [response] }; } const workflow = new StateGraph<AgentState>({ channels: graphState }) .addNode("agent", callModel) .addNode("tools", toolNode) .addEdge("__start__", "agent") .addConditionalEdges("agent", shouldContinue) .addEdge("tools", "agent"); const app = workflow.compile(); async function run() { const response = await app.streamEvents( { messages: [new HumanMessage("what is the weather in sf")] }, { version: 'v2', } ); for await (const event of response) { console.log(event); } } run(); ``` Output: ``` .... { "event": "on_chat_model_stream", "data": { "chunk": { "lc": 1, "type": "constructor", "id": [ "langchain_core", "messages", "AIMessageChunk" ], "kwargs": { "content": "{\"input", "tool_calls": [], "invalid_tool_calls": [], "tool_call_chunks": [], "additional_kwargs": {}, "response_metadata": {} } } }, "run_id": "a69b3dab-66c5-48a5-9086-5c2197074a66", "name": "ChatAnthropic", "tags": [], "metadata": { "langgraph_step": 1, "langgraph_node": "agent", "langgraph_triggers": [ "__pregel_tasks" ], "langgraph_task_idx": 0, "ls_provider": "anthropic", "ls_model_name": "claude-3-haiku-20240307", "ls_model_type": "chat", "ls_temperature": 1, "ls_max_tokens": 2048 } } .... ``` You can see that the content of `data.chunk.kwargs.content` is `"{"input"`, which is part of the tool call parameters. I hope that when generating tool call parameters, the `on_chat_model_stream` event is not sent, or there is a way to determine whether it is generating ordinary text or tool call parameters. If using the OpenAI model, it meets my expectations.
yindo closed this issue 2026-02-15 17:15:11 -05:00
yindo commented 2026-02-15 17:15:12 -05:00
Author
Owner
Copy Link
Copy Source

@jacoblee93 commented on GitHub (Aug 12, 2024):

This is how Anthropic returns generated tool params vs. OpenAI - however there ought to be some other way to distinguish between a generated tool response and ordinary text. Will look into it.

For now, does something like just checking whether the first chunk of a chat model run starts with a curly brace { work?

@jacoblee93 commented on GitHub (Aug 12, 2024): This is how Anthropic returns generated tool params vs. OpenAI - however there ought to be some other way to distinguish between a generated tool response and ordinary text. Will look into it. For now, does something like just checking whether the first chunk of a chat model run starts with a curly brace `{` work?
yindo commented 2026-02-15 17:15:12 -05:00
Author
Owner
Copy Link
Copy Source

@jacoblee93 commented on GitHub (Aug 12, 2024):

I'll open this on the LangChain.js repo and link the issue here.

@jacoblee93 commented on GitHub (Aug 12, 2024): I'll open this on the LangChain.js repo and link the issue here.
yindo commented 2026-02-15 17:15:12 -05:00
Author
Owner
Copy Link
Copy Source

@renxinyan commented on GitHub (Aug 12, 2024):

For now, does something like just checking whether the first chunk of a chat model run starts with a curly brace { work?

I think this method is not reliable, as the Anthropic model may generate some "thoughts" before generating the tool call parameters.

@renxinyan commented on GitHub (Aug 12, 2024): > For now, does something like just checking whether the first chunk of a chat model run starts with a curly brace { work? I think this method is not reliable, as the Anthropic model may generate some "thoughts" before generating the tool call parameters.
yindo commented 2026-02-15 17:15:13 -05:00
Author
Owner
Copy Link
Copy Source

@jacoblee93 commented on GitHub (Aug 12, 2024):

Ah right. Yeah we'll need something else then...

@jacoblee93 commented on GitHub (Aug 12, 2024): Ah right. Yeah we'll need something else then...
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
checkpointer
docs-needed
documentation
enhancement
good first issue
good writeup
help wanted
invalid
langgraph-supervisor
open-swe-auto
platform
prebuilt
pull-request
Mirrored from GitHub Pull Request
 question
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraphjs#58
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?