langchain-ai/langgraphjs
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraphjs.git synced 2026-07-22 00:55:26 -04:00
Code Issues 108 Packages Projects Releases 20 Wiki Activity

langgraph-supervisor broken with openai 0.6.x #316

New Issue
Closed
opened 2026-02-15 18:15:51 -05:00 by yindo · 2 comments
yindo commented 2026-02-15 18:15:51 -05:00
Owner
Copy Link
Copy Source

Originally created by @ddewaele on GitHub (Jul 29, 2025).

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangGraph.js documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangGraph.js rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangGraph (or the specific integration package).

Example Code

You can use the sample code from the supervisor agent

import { AzureChatOpenAI, ChatOpenAI } from "@langchain/openai";
import { createSupervisor } from "@langchain/langgraph-supervisor";
import { createReactAgent } from "@langchain/langgraph/prebuilt";
import { tool } from "@langchain/core/tools";
import { z } from "zod";
import { MessagesAnnotation } from "@langchain/langgraph";

const model = new AzureChatOpenAI({
    azureOpenAIApiKey: process.env.AZURE_OPENAI_API_KEY,
    azureOpenAIApiVersion: process.env.AZURE_OPENAI_API_VERSION || "2024-02-01",
    azureOpenAIEndpoint: process.env.AZURE_OPENAI_API_URL,
    azureOpenAIApiDeploymentName: process.env.AZURE_OPENAI_DEPLOYMENT_NAME || 'gpt-4o-mini',
    model: 'gpt-4o-mini'
});

// const model = new ChatOpenAI({ model: "gpt-4o-mini" });


// Create specialized agents
const add = tool(
    async (args) => args.a + args.b,
    {
        name: "add",
        description: "Add two numbers.",
        schema: z.object({
            a: z.number(),
            b: z.number()
        })
    }
);

const multiply = tool(
    async (args) => args.a * args.b,
    {
        name: "multiply",
        description: "Multiply two numbers.",
        schema: z.object({
            a: z.number(),
            b: z.number()
        })
    }
);

const webSearch = tool(
    async (args) => {
        return (
            "Here are the headcounts for each of the FAANG companies in 2024:\n" +
            "1. **Facebook (Meta)**: 67,317 employees.\n" +
            "2. **Apple**: 164,000 employees.\n" +
            "3. **Amazon**: 1,551,000 employees.\n" +
            "4. **Netflix**: 14,000 employees.\n" +
            "5. **Google (Alphabet)**: 181,269 employees."
        );
    },
    {
        name: "web_search",
        description: "Search the web for information.",
        schema: z.object({
            query: z.string()
        })
    }
);

const mathAgent = createReactAgent({
    llm: model,
    tools: [add, multiply],
    name: "math_expert",
    prompt: "You are a math expert. Always use one tool at a time."
});

const researchAgent = createReactAgent({
    llm: model,
    tools: [webSearch],
    name: "research_expert",
    prompt: "You are a world class researcher with access to web search. Do not do any math."
});

// Create supervisor workflow
const workflow = createSupervisor({
    agents: [researchAgent, mathAgent],
    llm: model,
    prompt:
        "You are a team supervisor managing a research expert and a math expert. " +
        "For current events, use research_agent. " +
        "For math problems, use math_agent.",
    stateSchema: MessagesAnnotation,

});

// Compile and run
export const app = workflow.compile();
// const result = await app.invoke({
//   messages: [
//     {
//       role: "user",
//       content: "what's the combined headcount of the FAANG companies in 2024??"
//     }
//   ]
// });

Description

The issue is that with OpenAI 0.6.x the supervisor agent is unable to process a second prompt. The supervisor agent nevers reaches out to other agents anymore but attempts to handle the output itself, calling tools that don't exist on the supervisor level

With Open AI 0.5.x the supervisor agent is able to process second prompts

  • Prompt 1 : What is the sum of 234 and 5467? (ensure that it calls the math agent)
  • Prompt 2 : what's the combined headcount of the FAANG companies in 2024?? (should call the research agent and math agent)

Potential Root Cause ? :

This bug might have been introduced with the withConfig support in langchain openai.
It seems that the underlying LLM is constantly shared and through the tool binding we are not calling the withConfig function that will update the default options of the underlying LLM .

    withConfig(config) {
        this.defaultOptions = { ...this.defaultOptions, ...config };
        return this;
    }

What do you think @hntrl ? During my second prompt (when the supervisor has forwarded a msg to the math react agents the defaultOptions on the LLM will be set to the add and multiply tools). Next time the supervisor needs to do something he will only see these 2 tools. The LLM will hallucinate and try to use these tools but the supervisor agent cannot invoke these tools as they are not attached to the supervisor.

If this makes sense I can log an issue in langchainjs

Error Message and Stack Trace (if applicable)

Trace with OpenAI 0.6

  "dependencies": {
    "@langchain/community": "^0.3.49",
    "@langchain/core": "^0.3.66",
    "@langchain/langgraph": "^0.3.11",
    "@langchain/langgraph-supervisor": "^0.0.16",
    "@langchain/openai": "^0.6.3"
  }
Image

Trace with OpenAI 0.5

(better, although it should also call the math agent for the second prompt

  "dependencies": {
    "@langchain/community": "^0.3.49",
    "@langchain/core": "^0.3.66",
    "@langchain/langgraph": "^0.3.11",
    "@langchain/langgraph-supervisor": "^0.0.16",
    "@langchain/openai": "^0.5.18"
  }
Image

System Info

base ❯ npm ls
├── @langchain/community@0.3.49
├── @langchain/core@0.3.66
├── @langchain/langgraph-cli@0.0.51
├── @langchain/langgraph-supervisor@0.0.16
├── @langchain/langgraph@0.3.12
├── @langchain/openai@0.6.3
├── @types/node@24.1.0
├── ts-node-dev@2.0.0
└── typescript@5.8.3


base ❯ npm -v
10.9.0

base ❯ node -v
v20.17.0

base ❯ uname
Darwin
Originally created by @ddewaele on GitHub (Jul 29, 2025). ### Checked other resources - [x] I added a very descriptive title to this issue. - [x] I searched the LangGraph.js documentation with the integrated search. - [x] I used the GitHub search to find a similar question and didn't find it. - [x] I am sure that this is a bug in LangGraph.js rather than my code. - [x] The bug is not resolved by updating to the latest stable version of LangGraph (or the specific integration package). ### Example Code You can use the sample code from the supervisor agent ``` import { AzureChatOpenAI, ChatOpenAI } from "@langchain/openai"; import { createSupervisor } from "@langchain/langgraph-supervisor"; import { createReactAgent } from "@langchain/langgraph/prebuilt"; import { tool } from "@langchain/core/tools"; import { z } from "zod"; import { MessagesAnnotation } from "@langchain/langgraph"; const model = new AzureChatOpenAI({ azureOpenAIApiKey: process.env.AZURE_OPENAI_API_KEY, azureOpenAIApiVersion: process.env.AZURE_OPENAI_API_VERSION || "2024-02-01", azureOpenAIEndpoint: process.env.AZURE_OPENAI_API_URL, azureOpenAIApiDeploymentName: process.env.AZURE_OPENAI_DEPLOYMENT_NAME || 'gpt-4o-mini', model: 'gpt-4o-mini' }); // const model = new ChatOpenAI({ model: "gpt-4o-mini" }); // Create specialized agents const add = tool( async (args) => args.a + args.b, { name: "add", description: "Add two numbers.", schema: z.object({ a: z.number(), b: z.number() }) } ); const multiply = tool( async (args) => args.a * args.b, { name: "multiply", description: "Multiply two numbers.", schema: z.object({ a: z.number(), b: z.number() }) } ); const webSearch = tool( async (args) => { return ( "Here are the headcounts for each of the FAANG companies in 2024:\n" + "1. **Facebook (Meta)**: 67,317 employees.\n" + "2. **Apple**: 164,000 employees.\n" + "3. **Amazon**: 1,551,000 employees.\n" + "4. **Netflix**: 14,000 employees.\n" + "5. **Google (Alphabet)**: 181,269 employees." ); }, { name: "web_search", description: "Search the web for information.", schema: z.object({ query: z.string() }) } ); const mathAgent = createReactAgent({ llm: model, tools: [add, multiply], name: "math_expert", prompt: "You are a math expert. Always use one tool at a time." }); const researchAgent = createReactAgent({ llm: model, tools: [webSearch], name: "research_expert", prompt: "You are a world class researcher with access to web search. Do not do any math." }); // Create supervisor workflow const workflow = createSupervisor({ agents: [researchAgent, mathAgent], llm: model, prompt: "You are a team supervisor managing a research expert and a math expert. " + "For current events, use research_agent. " + "For math problems, use math_agent.", stateSchema: MessagesAnnotation, }); // Compile and run export const app = workflow.compile(); // const result = await app.invoke({ // messages: [ // { // role: "user", // content: "what's the combined headcount of the FAANG companies in 2024??" // } // ] // }); ``` ### Description The issue is that with OpenAI 0.6.x the supervisor agent is unable to process a second prompt. The supervisor agent nevers reaches out to other agents anymore but attempts to handle the output itself, calling tools that don't exist on the supervisor level With Open AI 0.5.x the supervisor agent is able to process second prompts - Prompt 1 : What is the sum of 234 and 5467? (ensure that it calls the math agent) - Prompt 2 : what's the combined headcount of the FAANG companies in 2024?? (should call the research agent and math agent) **Potential Root Cause ? :** This bug might have been introduced with the `withConfig` support in langchain openai. It seems that the underlying LLM is constantly shared and through the tool binding we are not calling the `withConfig` function that will update the default options of the underlying LLM . ``` withConfig(config) { this.defaultOptions = { ...this.defaultOptions, ...config }; return this; } ``` What do you think @hntrl ? During my second prompt (when the supervisor has forwarded a msg to the math react agents the defaultOptions on the LLM will be set to the add and multiply tools). Next time the supervisor needs to do something he will only see these 2 tools. The LLM will hallucinate and try to use these tools but the supervisor agent cannot invoke these tools as they are not attached to the supervisor. If this makes sense I can log an issue in langchainjs ### Error Message and Stack Trace (if applicable) ## Trace with OpenAI 0.6 ``` "dependencies": { "@langchain/community": "^0.3.49", "@langchain/core": "^0.3.66", "@langchain/langgraph": "^0.3.11", "@langchain/langgraph-supervisor": "^0.0.16", "@langchain/openai": "^0.6.3" } ``` - prompt 1 : https://smith.langchain.com/public/6c1a1ff7-c292-4a30-951b-ffb03ef11f51/r - prompt 2 : https://smith.langchain.com/public/9fceacd4-f4f8-4378-a21e-46d48a85de31/r <img width="1383" height="1407" alt="Image" src="https://github.com/user-attachments/assets/059d231b-3dd8-4bd5-9101-41e0beec91bb" /> ## Trace with OpenAI 0.5 (better, although it should also call the math agent for the second prompt ``` "dependencies": { "@langchain/community": "^0.3.49", "@langchain/core": "^0.3.66", "@langchain/langgraph": "^0.3.11", "@langchain/langgraph-supervisor": "^0.0.16", "@langchain/openai": "^0.5.18" } ``` - prompt 1 : https://smith.langchain.com/public/be13d0c1-e066-4701-81aa-0962dd377497/r - prompt 2 : https://smith.langchain.com/public/1effe6e0-ef9b-44f2-9f74-12bd6aca2098/r <img width="988" height="1467" alt="Image" src="https://github.com/user-attachments/assets/0cb02a1e-3748-428c-8da7-7d8f6ba17075" /> ### System Info ``` base ❯ npm ls ├── @langchain/community@0.3.49 ├── @langchain/core@0.3.66 ├── @langchain/langgraph-cli@0.0.51 ├── @langchain/langgraph-supervisor@0.0.16 ├── @langchain/langgraph@0.3.12 ├── @langchain/openai@0.6.3 ├── @types/node@24.1.0 ├── ts-node-dev@2.0.0 └── typescript@5.8.3 base ❯ npm -v 10.9.0 base ❯ node -v v20.17.0 base ❯ uname Darwin ```
yindo closed this issue 2026-02-15 18:15:51 -05:00
yindo commented 2026-02-15 18:15:52 -05:00
Author
Owner
Copy Link
Copy Source

@hntrl commented on GitHub (Jul 30, 2025):

@ddewaele thanks for flagging and for the detailed write up! This is a downstream regression inside of @langchain/openai. I'll keep this issue open for visibility until a fix lands.

@hntrl commented on GitHub (Jul 30, 2025): @ddewaele thanks for flagging and for the detailed write up! This is a downstream regression inside of `@langchain/openai`. I'll keep this issue open for visibility until a fix lands.
yindo commented 2026-02-15 18:15:52 -05:00
Author
Owner
Copy Link
Copy Source

@hntrl commented on GitHub (Aug 19, 2025):

This was fixed with a recent release of @langchain/openai

@hntrl commented on GitHub (Aug 19, 2025): This was fixed with a recent release of `@langchain/openai`
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
checkpointer
docs-needed
documentation
enhancement
good first issue
good writeup
help wanted
invalid
langgraph-supervisor
open-swe-auto
platform
prebuilt
pull-request
Mirrored from GitHub Pull Request
 question
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraphjs#316
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?