langchain-ai/langgraphjs
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraphjs.git synced 2026-07-22 09:05:28 -04:00
Code Issues 108 Packages Projects Releases 20 Wiki Activity

Need help in streaming the graph's agent response from backend to frontend #132

New Issue
Closed
opened 2026-02-15 17:16:04 -05:00 by yindo · 6 comments
yindo commented 2026-02-15 17:16:04 -05:00
Owner
Copy Link
Copy Source

Originally created by @lokeswaran-aj on GitHub (Nov 10, 2024).

Hi, I'm learning langgraph right now. I'm trying to build a full-stack app with Nextjs. I'm not able to stream the response from the backend to the frontend. I can invoke the graph and send the completed response, but I don't know how to send the stream response. Right now, I'm not using Langgraph Studio or Cloud. Can you please share code snippets for NextJs or ExpressJs that invoke a graph and stream the final response?

In the frontend, I'm trying to use Vercel's AI package to stream the chat response. It would be great if you could share a code that works with this package.

Thanks in advance.

Originally created by @lokeswaran-aj on GitHub (Nov 10, 2024). Hi, I'm learning langgraph right now. I'm trying to build a full-stack app with Nextjs. I'm not able to stream the response from the backend to the frontend. I can invoke the graph and send the completed response, but I don't know how to send the stream response. Right now, I'm not using Langgraph Studio or Cloud. Can you please share code snippets for NextJs or ExpressJs that invoke a graph and stream the final response? In the frontend, I'm trying to use [Vercel's AI](https://www.npmjs.com/package/ai) package to stream the chat response. It would be great if you could share a code that works with this package. Thanks in advance.
yindo added the enhancement label 2026-02-15 17:16:04 -05:00
yindo closed this issue 2026-02-15 17:16:04 -05:00
yindo commented 2026-02-15 17:16:05 -05:00
Author
Owner
Copy Link
Copy Source

@stevemandl commented on GitHub (Nov 12, 2024):

I'm trying something similar with expressjs and having trouble getting it to work. I can stream the response, but there is no benefit to streaming because I can't seem to pass the streamed chunks coming from the model to the client. The only way I've gotten it to work is invokng the model and getting a message string as a response. Here is a working example that I'm hoping someone can change to actually use the streamed output of the llm:

import express from 'express';

import { z } from "zod";
import { tool } from "@langchain/core/tools";
import { AIMessage, BaseMessage, SystemMessage } from '@langchain/core/messages';
import { ChatBedrockConverse } from "@langchain/aws";

import { StateGraph, START, END, MessagesAnnotation } from "@langchain/langgraph";
import { ToolNode } from "@langchain/langgraph/prebuilt";

import { getCredentials } from './lib/credentials';

const router = express.Router();
const lastMessageContent = (messages: BaseMessage[]) => {
  if (messages.length > 0) {
    const lastMessage = messages[messages.length - 1];
    if (lastMessage instanceof AIMessage && lastMessage.response_metadata.stopReason === "end_turn")
      return lastMessage.content;
  }
  return "";
}

const weatherTool = tool(async ({ query }) => {
  if (query.toLowerCase().includes("sf")) {
    return "It's 60 degrees and foggy."
  }
  return "It's 90 degrees and sunny."
}, {
  name: "weather",
  description: "Call to get the current weather for a location.",
  schema: z.object({ query: z.string().describe("The query to use in your search.") })
}
);
const tools = [weatherTool];
const toolNode = new ToolNode(tools);
const model = new ChatBedrockConverse({
  model: process.env.MODEL_ID || "anthropic.claude-3-sonnet-20240229-v1:0",
  region: process.env.AWS_REGION,
  credentials: getCredentials()
}).bindTools(tools);
const prompt = `You are a Redbaird, a snarky pirate.`;
const systemMessage = new SystemMessage(prompt);
async function shouldContinue(state: typeof MessagesAnnotation.State) {
  const messages = state.messages;
  const lastMessage = messages[messages.length - 1] as AIMessage;
  if (lastMessage?.tool_calls?.length) {
    return "tools";
  }
  return END;
}
async function callModel(state: typeof MessagesAnnotation.State) {
  const messages: BaseMessage[] = [systemMessage].concat(state.messages);
  const message = await model.invoke(messages);
  return { messages: [message] };
}

const workflow = new StateGraph(MessagesAnnotation)
  .addNode("llm", callModel)
  .addNode("tools", toolNode)
  .addEdge(START, "llm")
  .addConditionalEdges("llm", shouldContinue)
  .addEdge("tools", "llm");

const app = workflow.compile();

router.get('/', function (_req, res) {
  res.send('API server')
})

router.post('/chat', async (req, res) => {
  try {
    const messages: BaseMessage[] = req.body.messages;
    const stream = await app.stream({ messages }, { streamMode: ["values"] });
    for await (const [_type, chunk] of stream) {
      if (chunk.messages) {
        res.write(lastMessageContent(chunk.messages));
      }
    }
    res.end();

  } catch (e: any) {
    console.error("POST api/chat error:", e, "at", e.stack?.split("\n  at p"));
    res.status(e.status ?? 500).json({ error: e.message });
  }

})
export default router
@stevemandl commented on GitHub (Nov 12, 2024): I'm trying something similar with expressjs and having trouble getting it to work. I can stream the response, but there is no benefit to streaming because I can't seem to pass the streamed chunks coming from the model to the client. The only way I've gotten it to work is invokng the model and getting a message string as a response. Here is a working example that I'm hoping someone can change to actually use the streamed output of the llm: ```ts import express from 'express'; import { z } from "zod"; import { tool } from "@langchain/core/tools"; import { AIMessage, BaseMessage, SystemMessage } from '@langchain/core/messages'; import { ChatBedrockConverse } from "@langchain/aws"; import { StateGraph, START, END, MessagesAnnotation } from "@langchain/langgraph"; import { ToolNode } from "@langchain/langgraph/prebuilt"; import { getCredentials } from './lib/credentials'; const router = express.Router(); const lastMessageContent = (messages: BaseMessage[]) => { if (messages.length > 0) { const lastMessage = messages[messages.length - 1]; if (lastMessage instanceof AIMessage && lastMessage.response_metadata.stopReason === "end_turn") return lastMessage.content; } return ""; } const weatherTool = tool(async ({ query }) => { if (query.toLowerCase().includes("sf")) { return "It's 60 degrees and foggy." } return "It's 90 degrees and sunny." }, { name: "weather", description: "Call to get the current weather for a location.", schema: z.object({ query: z.string().describe("The query to use in your search.") }) } ); const tools = [weatherTool]; const toolNode = new ToolNode(tools); const model = new ChatBedrockConverse({ model: process.env.MODEL_ID || "anthropic.claude-3-sonnet-20240229-v1:0", region: process.env.AWS_REGION, credentials: getCredentials() }).bindTools(tools); const prompt = `You are a Redbaird, a snarky pirate.`; const systemMessage = new SystemMessage(prompt); async function shouldContinue(state: typeof MessagesAnnotation.State) { const messages = state.messages; const lastMessage = messages[messages.length - 1] as AIMessage; if (lastMessage?.tool_calls?.length) { return "tools"; } return END; } async function callModel(state: typeof MessagesAnnotation.State) { const messages: BaseMessage[] = [systemMessage].concat(state.messages); const message = await model.invoke(messages); return { messages: [message] }; } const workflow = new StateGraph(MessagesAnnotation) .addNode("llm", callModel) .addNode("tools", toolNode) .addEdge(START, "llm") .addConditionalEdges("llm", shouldContinue) .addEdge("tools", "llm"); const app = workflow.compile(); router.get('/', function (_req, res) { res.send('API server') }) router.post('/chat', async (req, res) => { try { const messages: BaseMessage[] = req.body.messages; const stream = await app.stream({ messages }, { streamMode: ["values"] }); for await (const [_type, chunk] of stream) { if (chunk.messages) { res.write(lastMessageContent(chunk.messages)); } } res.end(); } catch (e: any) { console.error("POST api/chat error:", e, "at", e.stack?.split("\n at p")); res.status(e.status ?? 500).json({ error: e.message }); } }) export default router ```
yindo commented 2026-02-15 17:16:05 -05:00
Author
Owner
Copy Link
Copy Source

@lokeswaran-aj commented on GitHub (Nov 15, 2024):

For anyone looking for a Nextjs + Langgraphjs + Vercel AI SDK + streaming, here is my example app, and you can find the code here. Currently it has a simple chat completion and a chat with history features. I'm currently working on tool calling and generative UI.

@lokeswaran-aj commented on GitHub (Nov 15, 2024): For anyone looking for a Nextjs + Langgraphjs + Vercel AI SDK + streaming, here is my [example app](https://next-langgraph-example.vercel.app), and you can find the code [here](https://github.com/lokeswaran-aj/next-langgraph-example). Currently it has a simple chat completion and a chat with history features. I'm currently working on tool calling and generative UI.
yindo commented 2026-02-15 17:16:05 -05:00
Author
Owner
Copy Link
Copy Source

@vuletuanbt commented on GitHub (Jan 17, 2025):

For anyone looking for a Nextjs + Langgraphjs + Vercel AI SDK + streaming, here is my example app, and you can find the code here. Currently it has a simple chat completion and a chat with history features. I'm currently working on tool calling and generative UI.

You saved my day.

@vuletuanbt commented on GitHub (Jan 17, 2025): > For anyone looking for a Nextjs + Langgraphjs + Vercel AI SDK + streaming, here is my [example app](https://next-langgraph-example.vercel.app), and you can find the code [here](https://github.com/lokeswaran-aj/next-langgraph-example). Currently it has a simple chat completion and a chat with history features. I'm currently working on tool calling and generative UI. You saved my day.
yindo commented 2026-02-15 17:16:06 -05:00
Author
Owner
Copy Link
Copy Source

@hgoona commented on GitHub (Mar 6, 2025):

@lokeswaran-aj @vuletuanbt when using LangGraph with Vercel AI sdk are you guys able to see the token usage onFinish?

useChat({
	api: "/api/chat1",
		onFinish(message, { usage, finishReason }) {
			console.log("Usage", usage);
			console.log("FinishReason", finishReason);
			stopStreaming();
		}
	});

For me it keeps logging Usage as Nan

@hgoona commented on GitHub (Mar 6, 2025): @lokeswaran-aj @vuletuanbt when using LangGraph with Vercel AI sdk are you guys able to see the token usage onFinish? ``` useChat({ api: "/api/chat1", onFinish(message, { usage, finishReason }) { console.log("Usage", usage); console.log("FinishReason", finishReason); stopStreaming(); } }); ``` For me it keeps logging Usage as `Nan`
yindo commented 2026-02-15 17:16:06 -05:00
Author
Owner
Copy Link
Copy Source

@vuletuanbt commented on GitHub (Mar 6, 2025):

@lokeswaran-aj @vuletuanbt when using LangGraph with Vercel AI sdk are you guys able to see the token usage onFinish?

useChat({
			api: "/api/chat1",
			onFinish(message, { usage, finishReason }) {
				console.log("Usage", usage);
				console.log("FinishReason", finishReason);
				stopStreaming();
			}
		});

For me it keeps logging result as Nan

I haven't implemented this yet, so I'm unable to answer your question.

@vuletuanbt commented on GitHub (Mar 6, 2025): > [@lokeswaran-aj](https://github.com/lokeswaran-aj) [@vuletuanbt](https://github.com/vuletuanbt) when using LangGraph with Vercel AI sdk are you guys able to see the token usage onFinish? > > ``` > useChat({ > api: "/api/chat1", > onFinish(message, { usage, finishReason }) { > console.log("Usage", usage); > console.log("FinishReason", finishReason); > stopStreaming(); > } > }); > ``` > > For me it keeps logging result as `Nan` I haven't implemented this yet, so I'm unable to answer your question.
yindo commented 2026-02-15 17:16:06 -05:00
Author
Owner
Copy Link
Copy Source

@dqbd commented on GitHub (Sep 11, 2025):

Hello! Please use useStream() for first-class streaming handling from LangGraph to your FE of choice
https://docs.langchain.com/langgraph-platform/use-stream-react#customizing-your-ui

@dqbd commented on GitHub (Sep 11, 2025): Hello! Please use `useStream()` for first-class streaming handling from LangGraph to your FE of choice https://docs.langchain.com/langgraph-platform/use-stream-react#customizing-your-ui
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
checkpointer
docs-needed
documentation
enhancement
good first issue
good writeup
help wanted
invalid
langgraph-supervisor
open-swe-auto
platform
prebuilt
pull-request
Mirrored from GitHub Pull Request
 question
No Label enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraphjs#132
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?