langchain-ai/docs
1
0
Fork 0
mirror of https://github.com/langchain-ai/docs.git synced 2026-07-22 20:35:34 -04:00
Code Issues 490 Packages Projects Releases Wiki Activity

[GH-ISSUE #1539] How can a template access the streaming tokens generated during its execution? #208

New Issue
Open
opened 2026-02-17 17:19:24 -05:00 by yindo · 0 comments
yindo commented 2026-02-17 17:19:24 -05:00
Owner
Copy Link
Copy Source

Originally created by @xunfeng2zkj on GitHub (Nov 20, 2025).
Original GitHub issue: https://github.com/langchain-ai/docs/issues/1539

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
import time
import json


def create_fundamentals_analyst(llm):
    async def fundamentals_analyst_node(state):
        current_date = state["trade_date"]

        tools = [
            get_fundamentals,
        ]

        system_message = (
            "....",
        )

        prompt = ChatPromptTemplate.from_messages(
            [
                (
                    "system",
                    "You are a helpful AI assistant, collaborating with other assistants."
                    " You have access to the following tools: {tool_names}.\n{system_message}"
                    "For your reference, the current date is {current_date}. The company we want to look at is {ticker}",
                ),
                MessagesPlaceholder(variable_name="messages"),
            ]
        )

        prompt = prompt.partial(system_message=system_message)
        prompt = prompt.partial(tool_names=", ".join([tool.name for tool in tools]))
        prompt = prompt.partial(current_date=current_date)
        prompt = prompt.partial(ticker=ticker)

        chain = prompt | llm.bind_tools(tools)

        result = await chain.ainvoke(state["messages"])

        report = ""

        if len(result.tool_calls) == 0:
            report = result.content

        return {
            "messages": [result],
            "fundamentals_report": report,
        }

    return fundamentals_analyst_node

by:

            async for chunk in self.graph.astream(init_agent_state, **args):
                if len(chunk["messages"]) == 0:
                    pass
                else:
                    if chunk['messages'][-1].type == 'tool':
                        continue
                    #chunk["messages"][-1].pretty_print()
                    trace.append(chunk)
                    yield chunk["messages"][-1].content

Currently, I'm only able to stream the aggregated output of each LangGraph node, but not the raw token-level stream produced during the model invocation. How can I stream individual tokens from the underlying LLM call during node execution?

Originally created by @xunfeng2zkj on GitHub (Nov 20, 2025). Original GitHub issue: https://github.com/langchain-ai/docs/issues/1539 ``` from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder import time import json def create_fundamentals_analyst(llm): async def fundamentals_analyst_node(state): current_date = state["trade_date"] tools = [ get_fundamentals, ] system_message = ( "....", ) prompt = ChatPromptTemplate.from_messages( [ ( "system", "You are a helpful AI assistant, collaborating with other assistants." " You have access to the following tools: {tool_names}.\n{system_message}" "For your reference, the current date is {current_date}. The company we want to look at is {ticker}", ), MessagesPlaceholder(variable_name="messages"), ] ) prompt = prompt.partial(system_message=system_message) prompt = prompt.partial(tool_names=", ".join([tool.name for tool in tools])) prompt = prompt.partial(current_date=current_date) prompt = prompt.partial(ticker=ticker) chain = prompt | llm.bind_tools(tools) result = await chain.ainvoke(state["messages"]) report = "" if len(result.tool_calls) == 0: report = result.content return { "messages": [result], "fundamentals_report": report, } return fundamentals_analyst_node ``` by: ``` async for chunk in self.graph.astream(init_agent_state, **args): if len(chunk["messages"]) == 0: pass else: if chunk['messages'][-1].type == 'tool': continue #chunk["messages"][-1].pretty_print() trace.append(chunk) yield chunk["messages"][-1].content ``` Currently, I'm only able to stream the aggregated output of each LangGraph node, but not the raw token-level stream produced during the model invocation. How can I stream individual tokens from the underlying LLM call during node execution?
yindo added the external label 2026-02-17 17:19:24 -05:00
yindo changed title from How can a template access the streaming tokens generated during its execution? to [GH-ISSUE #1539] How can a template access the streaming tokens generated during its execution? 2026-06-05 17:25:40 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
deepagents
docs-infra
external
external
help wanted
integration
internal
internal
javascript
js/ts
langchain
langgraph
langsmith
needs-title
pull-request
Mirrored from GitHub Pull Request
 python
reference
No Label external
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/docs#208
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?