langchain-ai/langgraphjs
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraphjs.git synced 2026-07-22 17:15:25 -04:00
Code Issues 108 Packages Projects Releases 20 Wiki Activity

LLM response format structuring while streaming? #50

New Issue
Closed
opened 2026-02-15 17:15:04 -05:00 by yindo · 4 comments
yindo commented 2026-02-15 17:15:04 -05:00
Owner
Copy Link
Copy Source

Originally created by @epleaner on GitHub (Jul 17, 2024).

Hello,

With LangChain, I am able to quite easily stream LLM responses in a particular format (say, JSON), via JsonOutputFunctionsParser (see this doc, with { diff: false}), and get structured JSON every step of the stream.

With LangGraph, I am following this guide to ensure the agent responds in JSON format. However, this essentially breaks streaming: the LLM's structured output is not streamed token-by-token, as it is in LangChain – it comes out all at once. If I use the streamEvents API in combination with the response tool described in the previous guide to stream LLM output token-by-token, the structure enforcement breaks and I get no output.

Has anyone succeeded in this in LangGraph?

Originally created by @epleaner on GitHub (Jul 17, 2024). Hello, With LangChain, I am able to quite easily stream LLM responses in a particular format (say, JSON), via JsonOutputFunctionsParser (see [this doc](https://js.langchain.com/v0.1/docs/modules/model_io/output_parsers/types/json_functions/#streaming), with { diff: false}), and get structured JSON every step of the stream. With LangGraph, I am following [this guide](https://langchain-ai.github.io/langgraphjs/how-tos/respond-in-format/) to ensure the agent responds in JSON format. However, this essentially breaks streaming: the LLM's structured output is not streamed token-by-token, as it is in LangChain – it comes out all at once. If I use [the streamEvents API](https://langchain-ai.github.io/langgraphjs/how-tos/stream-tokens/) in combination with the response tool described in the previous guide to stream LLM output token-by-token, the structure enforcement breaks and I get no output. Has anyone succeeded in this in LangGraph?
yindo closed this issue 2026-02-15 17:15:04 -05:00
yindo commented 2026-02-15 17:15:05 -05:00
Author
Owner
Copy Link
Copy Source

@jacoblee93 commented on GitHub (Jul 19, 2024):

Hey @epleaner, thanks for raising this! I've added an example here:

https://langchain-ai.github.io/langgraphjs/how-tos/respond-in-format/#partially-streaming-json

Let me know if you have any other questions

@jacoblee93 commented on GitHub (Jul 19, 2024): Hey @epleaner, thanks for raising this! I've added an example here: https://langchain-ai.github.io/langgraphjs/how-tos/respond-in-format/#partially-streaming-json Let me know if you have any other questions
yindo commented 2026-02-15 17:15:05 -05:00
Author
Owner
Copy Link
Copy Source

@epleaner commented on GitHub (Jul 22, 2024):

Hey @jacoblee93 , thanks for the response. However, the logs shown in the LangGraph example you've linked do not show structured JSON at every step of the stream. The LangChain example I've linked in my original post does log structured JSON at every step of the stream.

@epleaner commented on GitHub (Jul 22, 2024): Hey @jacoblee93 , thanks for the response. However, the logs shown in the LangGraph example you've linked do not show structured JSON at every step of the stream. The LangChain example I've linked in my original post does log structured JSON at every step of the stream.
yindo commented 2026-02-15 17:15:05 -05:00
Author
Owner
Copy Link
Copy Source

@jacoblee93 commented on GitHub (Jul 22, 2024):

do not show structured JSON at every step of the stream

Ah I see what you're saying, yeah will update later today.

@jacoblee93 commented on GitHub (Jul 22, 2024): > do not show structured JSON at every step of the stream Ah I see what you're saying, yeah will update later today.
yindo commented 2026-02-15 17:15:05 -05:00
Author
Owner
Copy Link
Copy Source

@jacoblee93 commented on GitHub (Jul 22, 2024):

Actually, I added it here:

      // You can also log the raw args instead:
      // console.log(chunk.tool_call_chunks);

Can you try that and LMK if it works for you? Keep in mind you may need to do some special tracking of the last valid args for chunks where we can't aggregate fully valid JSON (you could end up in a state like { "temperature": 67, "other_not }.

@jacoblee93 commented on GitHub (Jul 22, 2024): Actually, I added it here: ```ts // You can also log the raw args instead: // console.log(chunk.tool_call_chunks); ``` Can you try that and LMK if it works for you? Keep in mind you may need to do some special tracking of the last valid args for chunks where we can't aggregate fully valid JSON (you could end up in a state like `{ "temperature": 67, "other_not }`.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
checkpointer
docs-needed
documentation
enhancement
good first issue
good writeup
help wanted
invalid
langgraph-supervisor
open-swe-auto
platform
prebuilt
pull-request
Mirrored from GitHub Pull Request
 question
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraphjs#50
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?