langchain-ai/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-07-15 12:35:43 -04:00
Code Issues 314 Packages Projects Releases 17 Wiki Activity

Inconsistency .invoke and .stream for graph's last callback and output_keys affecting streamed values #289

New Issue
Closed
opened 2026-02-20 17:35:30 -05:00 by yindo · 4 comments
yindo commented 2026-02-20 17:35:30 -05:00
Owner
Copy Link
Copy Source

Originally created by @kaiwend on GitHub (Oct 24, 2024).

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangGraph/LangChain documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangGraph/LangChain rather than my code.
  • I am sure this is better as an issue rather than a GitHub discussion, since this is a LangGraph bug and not a design question.

Example Code

from typing import TypedDict

from langchain_core.callbacks.base import BaseCallbackHandler
from langchain_core.runnables.config import RunnableConfig
from langgraph.graph import END, StateGraph
from langgraph.graph.graph import START


class CustomCallbackHandler(BaseCallbackHandler):
    def on_chain_end(self, _outputs, **kwargs):
        print("on_chain_end", _outputs)


class OutputType(TypedDict):
    a: int


class State(TypedDict):
    a: int
    b: int


graph = StateGraph(State, output=OutputType)
graph.add_node("node_a", lambda state: {"a": state["a"] + 1})
graph.add_node("node_b", lambda state: {"b": state["b"] + 1})

graph.add_edge(START, "node_a")
graph.add_edge("node_a", "node_b")
graph.add_edge("node_b", END)

input = {"a": 0, "b": 0}

config = RunnableConfig(callbacks=[CustomCallbackHandler()])

compiled_graph = graph.compile()

invoke_result = compiled_graph.invoke(input, config)
print("invoke result: ", invoke_result)

print("**************")
print("graph without output keys defined")
print("**************")
graph_without_output_keys_results = []
for stream_output in compiled_graph.stream(input, config):
    graph_without_output_keys_results.append(stream_output)

print("*** stream outputs ***")
for stream_output in graph_without_output_keys_results:
    print(stream_output)

print("**************")
print("graph with output keys (a) defined")
print("**************")
graph_with_output_keys_results = []
for stream_output in compiled_graph.stream(input, config, output_keys=["a"]):
    graph_with_output_keys_results.append(stream_output)

print("*** stream outputs ***")
for stream_output in graph_with_output_keys_results:
    print(stream_output)

Error Message and Stack Trace (if applicable)

[Annotated result]

on_chain_end {'a': 0, 'b': 0}
on_chain_end {'a': 1}
on_chain_end {'a': 1}
on_chain_end {'b': 1}
on_chain_end {'b': 1}
on_chain_end {'a': 1} <--- This is the graph's final callback that I am tracing
invoke result:  {'a': 1}
**************
graph without output keys defined
**************
on_chain_end {'a': 0, 'b': 0}
on_chain_end {'a': 1}
on_chain_end {'a': 1}
on_chain_end {'b': 1}
on_chain_end {'b': 1}
on_chain_end {'a': 1, 'b': 1} <-- This is the graph's final callback result when using stream
*** stream outputs ***
{'node_a': {'a': 1}}
{'node_b': {'b': 1}}
**************
graph with output keys (a) defined
**************
on_chain_end {'a': 0, 'b': 0}
on_chain_end {'a': 1}
on_chain_end {'a': 1}
on_chain_end {'b': 1}
on_chain_end {'b': 1}
on_chain_end {'a': 1} <-- This is the graph's final callback result when using stream with output_keys
*** stream outputs ***
{'node_a': {'a': 1}}
{'node_b': None} <-- I expect this not to be None, as I want to stream content, that is not necessarily returned from the graph in the end

Description

Here is a minimal repo to show the encountered inconsistency: https://github.com/kaiwend/langgraph-streaming-inconsistency

I have been working with langgraph for a while now, mostly invoking my graph using .invoke. Now after some time, I wanted to introduce streaming into my app. The overall goal is to get some internals from the graph execution out earlier to the user and not just after the whole invocation is done, e.g. logging and streaming the output to the user. While doing that, I noticed some inconsistencies on how the streaming behaves in combination with callbacks. I am using tracing that uses the callbacks to collect traces. I am working with quite large amounts of data, so limiting what is coming in and out of nodes is crucial for the task at hand. However when using streaming, I noticed the following difference to the non-streamed version:

  • I have an output TypedDict, which I assign to my graph StateGraph(State, output=OutputType). When using .invoke, the output type is used and I only get what's defined in OutputType. This also leads to the last callback also being triggered with OutputType.
  • When using streaming on the other hand, the last callback from the graph get my whole state, which is a problem in regards to the sheer size of my state's contents
  • I tried adding output_keys= (keys from my OutputType) to my graph.stream call. This lead to the last callback being triggered correctly, with only returning OutputType. However this also had the side effect of the streamed values becoming limited to those output_keys as well.

To summarize, I think there are 2 potential issues at hand:

  • When streaming the graph, the final callback is called with the whole state instead of being limited to the type that is defined in output=
  • When setting output_keys= on graph.stream, the streamed values in between are also filtered by those keys and not only the final output keys, which is the opposite of what I would have expected, but might be intended?

System Info

langgraph==0.2.39
langgraph-checkpoint==2.0.1
langgraph-sdk==0.1.33

mac os

Python 3.11.6

Originally created by @kaiwend on GitHub (Oct 24, 2024). ### Checked other resources - [X] I added a very descriptive title to this issue. - [X] I searched the [LangGraph](https://langchain-ai.github.io/langgraph/)/LangChain documentation with the integrated search. - [X] I used the GitHub search to find a similar question and didn't find it. - [X] I am sure that this is a bug in LangGraph/LangChain rather than my code. - [X] I am sure this is better as an issue [rather than a GitHub discussion](https://github.com/langchain-ai/langgraph/discussions/new/choose), since this is a LangGraph bug and not a design question. ### Example Code ```python from typing import TypedDict from langchain_core.callbacks.base import BaseCallbackHandler from langchain_core.runnables.config import RunnableConfig from langgraph.graph import END, StateGraph from langgraph.graph.graph import START class CustomCallbackHandler(BaseCallbackHandler): def on_chain_end(self, _outputs, **kwargs): print("on_chain_end", _outputs) class OutputType(TypedDict): a: int class State(TypedDict): a: int b: int graph = StateGraph(State, output=OutputType) graph.add_node("node_a", lambda state: {"a": state["a"] + 1}) graph.add_node("node_b", lambda state: {"b": state["b"] + 1}) graph.add_edge(START, "node_a") graph.add_edge("node_a", "node_b") graph.add_edge("node_b", END) input = {"a": 0, "b": 0} config = RunnableConfig(callbacks=[CustomCallbackHandler()]) compiled_graph = graph.compile() invoke_result = compiled_graph.invoke(input, config) print("invoke result: ", invoke_result) print("**************") print("graph without output keys defined") print("**************") graph_without_output_keys_results = [] for stream_output in compiled_graph.stream(input, config): graph_without_output_keys_results.append(stream_output) print("*** stream outputs ***") for stream_output in graph_without_output_keys_results: print(stream_output) print("**************") print("graph with output keys (a) defined") print("**************") graph_with_output_keys_results = [] for stream_output in compiled_graph.stream(input, config, output_keys=["a"]): graph_with_output_keys_results.append(stream_output) print("*** stream outputs ***") for stream_output in graph_with_output_keys_results: print(stream_output) ``` ### Error Message and Stack Trace (if applicable) ```shell [Annotated result] on_chain_end {'a': 0, 'b': 0} on_chain_end {'a': 1} on_chain_end {'a': 1} on_chain_end {'b': 1} on_chain_end {'b': 1} on_chain_end {'a': 1} <--- This is the graph's final callback that I am tracing invoke result: {'a': 1} ************** graph without output keys defined ************** on_chain_end {'a': 0, 'b': 0} on_chain_end {'a': 1} on_chain_end {'a': 1} on_chain_end {'b': 1} on_chain_end {'b': 1} on_chain_end {'a': 1, 'b': 1} <-- This is the graph's final callback result when using stream *** stream outputs *** {'node_a': {'a': 1}} {'node_b': {'b': 1}} ************** graph with output keys (a) defined ************** on_chain_end {'a': 0, 'b': 0} on_chain_end {'a': 1} on_chain_end {'a': 1} on_chain_end {'b': 1} on_chain_end {'b': 1} on_chain_end {'a': 1} <-- This is the graph's final callback result when using stream with output_keys *** stream outputs *** {'node_a': {'a': 1}} {'node_b': None} <-- I expect this not to be None, as I want to stream content, that is not necessarily returned from the graph in the end ``` ### Description Here is a minimal repo to show the encountered inconsistency: https://github.com/kaiwend/langgraph-streaming-inconsistency I have been working with langgraph for a while now, mostly invoking my graph using `.invoke`. Now after some time, I wanted to introduce streaming into my app. The overall goal is to get some internals from the graph execution out earlier to the user and not just after the whole invocation is done, e.g. logging and streaming the output to the user. While doing that, I noticed some inconsistencies on how the streaming behaves in combination with callbacks. I am using tracing that uses the callbacks to collect traces. I am working with quite large amounts of data, so limiting what is coming in and out of nodes is crucial for the task at hand. However when using streaming, I noticed the following difference to the non-streamed version: - I have an output TypedDict, which I assign to my graph `StateGraph(State, output=OutputType)`. When using `.invoke`, the output type is used and I only get what's defined in `OutputType`. This also leads to the last callback also being triggered with OutputType. - When using streaming on the other hand, the last callback from the graph get my whole state, which is a problem in regards to the sheer size of my state's contents - I tried adding `output_keys=` (keys from my OutputType) to my `graph.stream` call. This lead to the last callback being triggered correctly, with only returning OutputType. However this also had the side effect of the streamed values becoming limited to those `output_keys` as well. To summarize, I think there are 2 potential issues at hand: - When streaming the graph, the final callback is called with the whole state instead of being limited to the type that is defined in `output=` - When setting `output_keys=` on `graph.stream`, the streamed values in between are also filtered by those keys and not only the final output keys, which is the opposite of what I would have expected, but might be intended? ### System Info langgraph==0.2.39 langgraph-checkpoint==2.0.1 langgraph-sdk==0.1.33 mac os Python 3.11.6
yindo closed this issue 2026-02-20 17:35:30 -05:00
yindo commented 2026-02-20 17:35:32 -05:00
Author
Owner
Copy Link
Copy Source

@highdealist commented on GitHub (Jan 10, 2025):

Have you really not gotten any answer on this in almost 3 months?

@highdealist commented on GitHub (Jan 10, 2025): Have you really not gotten any answer on this in almost 3 months?
yindo commented 2026-02-20 17:35:33 -05:00
Author
Owner
Copy Link
Copy Source

@kaiwend commented on GitHub (Jan 14, 2025):

No, not yet

@kaiwend commented on GitHub (Jan 14, 2025): No, not yet
yindo commented 2026-02-20 17:35:36 -05:00
Author
Owner
Copy Link
Copy Source

@javieribanez17 commented on GitHub (Apr 14, 2025):

What happen with LangGraph support?

@javieribanez17 commented on GitHub (Apr 14, 2025): What happen with LangGraph support?
yindo commented 2026-02-20 17:35:36 -05:00
Author
Owner
Copy Link
Copy Source

@sydney-runkle commented on GitHub (Jun 10, 2025):

Hi! Sorry for the delay here! I believe this is the expected result - based on the output specified, you're seeing the corresponding keys in the output results.

Going to close for now - feel free to follow up if you have any more specific questions!

@sydney-runkle commented on GitHub (Jun 10, 2025): Hi! Sorry for the delay here! I believe this is the expected result - based on the `output` specified, you're seeing the corresponding keys in the output results. Going to close for now - feel free to follow up if you have any more specific questions!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agents
awaiting author revision
bug
change
chat models
checkpointer
ci
code quality
code quality
dependencies
documentation
drawing
duplicate
enhancement
good first issue
help wanted
invalid
maintainer
meta
missing information
not maintained by langchain
open-swe-auto
open-swe-max
open-swe-max-auto
pending
performance
pull-request
Mirrored from GitHub Pull Request
 question
requires design
sdk
server
stale
studio
tests
type checking
usage question
v0.5
v1.1
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraph#289
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?