Add tool details to tool call stream event metadata #57

Closed
opened 2026-02-15 17:15:09 -05:00 by yindo · 3 comments
Owner

Originally created by @airhorns on GitHub (Aug 7, 2024).

The on_tool_call_start event and end event don't tell you which tool is actually being invoked. It'd be very helpful to be able to get that metadata in there somehow if you want to show the user what you're doing under the hood. ChatGPT does this for example when it searches the web -- you get a structured little widget in the UI that shows that that is happening under the hood, and if we want to use ToolNode, you can't actually tell which tool is doing its thing under the hood right now. Can this metadata be exposed on the stream events so that we can tell?

Thanks!

Originally created by @airhorns on GitHub (Aug 7, 2024). The `on_tool_call_start` event and end event don't tell you which tool is actually being invoked. It'd be very helpful to be able to get that metadata in there somehow if you want to show the user what you're doing under the hood. ChatGPT does this for example when it searches the web -- you get a structured little widget in the UI that shows that that is happening under the hood, and if we want to use `ToolNode`, you can't actually tell which tool is doing its thing under the hood right now. Can this metadata be exposed on the stream events so that we can tell? Thanks!
yindo closed this issue 2026-02-15 17:15:09 -05:00
Author
Owner

@hinthornw commented on GitHub (Aug 7, 2024):

(oops was responding with python 1 sec)

@hinthornw commented on GitHub (Aug 7, 2024): (oops was responding with python 1 sec)
Author
Owner

@hinthornw commented on GitHub (Aug 7, 2024):

Running this code:

import { TavilySearchResults } from "@langchain/community/tools/tavily_search";
import { ChatOpenAI } from "@langchain/openai";
import { MemorySaver } from "@langchain/langgraph";
import { HumanMessage } from "@langchain/core/messages";
import { createReactAgent } from "@langchain/langgraph/prebuilt";

// Define the tools for the agent to use
const agentTools = [new TavilySearchResults({ maxResults: 3 })];
const agentModel = new ChatOpenAI({ temperature: 0 });

// Initialize memory to persist state between graph runs
const agentCheckpointer = new MemorySaver();
const agent = createReactAgent({
  llm: agentModel,
  tools: agentTools,
  checkpointSaver: agentCheckpointer,
});

// Now it's time to use!
const agentFinalState = await agent.invoke(
  { messages: [new HumanMessage("what is the current weather in sf")] },
  { configurable: { thread_id: "42" } },
);

console.log(
  agentFinalState.messages[agentFinalState.messages.length - 1].content,
);

let events = agent.streamEvents(
  { messages: [new HumanMessage("what about ny")] },
  { configurable: { thread_id: "42" }, version: "v2" },
);
for await (const event of events) {
  if (event?.event == 'on_tool_start')
  {
    console.log(event);
  
  }

I get:

{
event: 'on_tool_start',
data: { input: { input: 'current weather in New York' } },
name: 'TavilySearchResults',
run_id: 'a5d4d5c7-c150-4616-879c-5f610049f56c',
tags: [ 'seq:step:1' ],
metadata: { thread_id: '42' }
}

The on_tool_start gets the name of the tool and the tags and metadata provided to it, as well as the data inputs.

Are you seeing something different?

@hinthornw commented on GitHub (Aug 7, 2024): Running this code: ``` import { TavilySearchResults } from "@langchain/community/tools/tavily_search"; import { ChatOpenAI } from "@langchain/openai"; import { MemorySaver } from "@langchain/langgraph"; import { HumanMessage } from "@langchain/core/messages"; import { createReactAgent } from "@langchain/langgraph/prebuilt"; // Define the tools for the agent to use const agentTools = [new TavilySearchResults({ maxResults: 3 })]; const agentModel = new ChatOpenAI({ temperature: 0 }); // Initialize memory to persist state between graph runs const agentCheckpointer = new MemorySaver(); const agent = createReactAgent({ llm: agentModel, tools: agentTools, checkpointSaver: agentCheckpointer, }); // Now it's time to use! const agentFinalState = await agent.invoke( { messages: [new HumanMessage("what is the current weather in sf")] }, { configurable: { thread_id: "42" } }, ); console.log( agentFinalState.messages[agentFinalState.messages.length - 1].content, ); let events = agent.streamEvents( { messages: [new HumanMessage("what about ny")] }, { configurable: { thread_id: "42" }, version: "v2" }, ); for await (const event of events) { if (event?.event == 'on_tool_start') { console.log(event); } ``` I get: ``` { event: 'on_tool_start', data: { input: { input: 'current weather in New York' } }, name: 'TavilySearchResults', run_id: 'a5d4d5c7-c150-4616-879c-5f610049f56c', tags: [ 'seq:step:1' ], metadata: { thread_id: '42' } } ``` The `on_tool_start` gets the name of the tool and the tags and metadata provided to it, as well as the data inputs. Are you seeing something different?
Author
Owner

@airhorns commented on GitHub (Aug 7, 2024):

Ah frig 🤦 My logger overrides the name attribute and was hiding it, my apologies! Thanks!

@airhorns commented on GitHub (Aug 7, 2024): Ah frig 🤦 My logger overrides the `name` attribute and was hiding it, my apologies! Thanks!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraphjs#57