langchain-ai/deepagentsjs
1
0
Fork 0
mirror of https://github.com/langchain-ai/deepagentsjs.git synced 2026-07-19 16:33:38 -04:00
Code Issues 42 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #467] SummarizationMiddleware never triggers with non-OpenAI models #263

New Issue
Closed
opened 2026-06-05 17:21:20 -04:00 by yindo · 2 comments
yindo commented 2026-06-05 17:21:20 -04:00
Owner
Copy Link
Copy Source

Originally created by @xinzine on GitHub (Apr 15, 2026).
Original GitHub issue: https://github.com/langchain-ai/deepagentsjs/issues/467

Bug Description

SummarizationMiddleware never triggers when using non-OpenAI models (e.g. qwen3.6-plus), even after 2 million tokens of conversation.

Root Cause Analysis

1. maxInputTokens always undefined for non-OpenAI models

createDeepAgent internally calls createSummarizationMiddleware({ model, backend }) without passing trigger or keep parameters (line ~6567 in dist/index.cjs):

createSummarizationMiddleware({
  model,
  backend
})

Inside createSummarizationMiddleware, maxInputTokens is derived from model.profile.maxInputTokens. However, @langchain/openai's PROFILES table only contains OpenAI models - qwen models are not listed. This causes getMaxInputTokens(resolvedModel) to return undefined.

2. FALLBACK_TRIGGER is hardcoded to 17e4 (170,000 tokens)

Even when no model profile exists, computeSummarizationDefaults() falls back to {type: "tokens", value: 170000}. This is far too high for most use cases and essentially makes the middleware never fire.

3. No public API to configure trigger/keep

There is no way to pass custom trigger or keep options to createSummarizationMiddleware through createDeepAgent's public interface. Users cannot override:

  • The trigger threshold (e.g. 4000 tokens instead of 170000)
  • The keep amount (e.g. last 20 messages)
  • The summarization model (separate from the main agent model)

Expected Behavior

Users should be able to configure:

createDeepAgent({
  summarization: {
    trigger: { tokens: 4000 },
    keep: { messages: 20 },
    model: someModel,
  }
})

Proposed Solutions

  1. Accept trigger/keep in createDeepAgent options and forward them to createSummarizationMiddleware
  2. Document FALLBACK_TRIGGER value and expose a way to override it
  3. Add a maxInputTokens override option so non-OpenAI models can specify their context window
  4. Provide a separate summarization model option (e.g. cheaper/faster model for summarization)
  5. Lower the FALLBACK_TRIGGER value from 170000 to something more reasonable (e.g. 16000)

Environment

  • deepagents: 1.9.0
  • @langchain/openai: (latest)
  • Model: qwen3.6-plus (via custom baseURL)
  • Conversation size: ~2M tokens with no summarization triggered
Originally created by @xinzine on GitHub (Apr 15, 2026). Original GitHub issue: https://github.com/langchain-ai/deepagentsjs/issues/467 ## Bug Description `SummarizationMiddleware` never triggers when using non-OpenAI models (e.g. `qwen3.6-plus`), even after 2 million tokens of conversation. ## Root Cause Analysis ### 1. `maxInputTokens` always `undefined` for non-OpenAI models `createDeepAgent` internally calls `createSummarizationMiddleware({ model, backend })` without passing `trigger` or `keep` parameters (line ~6567 in `dist/index.cjs`): ```js createSummarizationMiddleware({ model, backend }) ``` Inside `createSummarizationMiddleware`, `maxInputTokens` is derived from `model.profile.maxInputTokens`. However, `@langchain/openai`'s `PROFILES` table only contains OpenAI models - qwen models are not listed. This causes `getMaxInputTokens(resolvedModel)` to return `undefined`. ### 2. `FALLBACK_TRIGGER` is hardcoded to `17e4` (170,000 tokens) Even when no model profile exists, `computeSummarizationDefaults()` falls back to `{type: "tokens", value: 170000}`. This is far too high for most use cases and essentially makes the middleware never fire. ### 3. No public API to configure `trigger`/`keep` There is no way to pass custom `trigger` or `keep` options to `createSummarizationMiddleware` through `createDeepAgent`'s public interface. Users cannot override: - The trigger threshold (e.g. `4000` tokens instead of `170000`) - The keep amount (e.g. last 20 messages) - The summarization model (separate from the main agent model) ## Expected Behavior Users should be able to configure: ```ts createDeepAgent({ summarization: { trigger: { tokens: 4000 }, keep: { messages: 20 }, model: someModel, } }) ``` ## Proposed Solutions 1. Accept `trigger`/`keep` in `createDeepAgent` options and forward them to `createSummarizationMiddleware` 2. Document `FALLBACK_TRIGGER` value and expose a way to override it 3. Add a `maxInputTokens` override option so non-OpenAI models can specify their context window 4. Provide a separate summarization model option (e.g. cheaper/faster model for summarization) 5. Lower the `FALLBACK_TRIGGER` value from `170000` to something more reasonable (e.g. `16000`) ## Environment - deepagents: 1.9.0 - @langchain/openai: (latest) - Model: qwen3.6-plus (via custom baseURL) - Conversation size: ~2M tokens with no summarization triggered
yindo closed this issue 2026-06-05 17:21:20 -04:00
yindo commented 2026-06-05 17:21:20 -04:00
Author
Owner
Copy Link
Copy Source

@Oscar-Umana commented on GitHub (Apr 23, 2026):

Hey @xinzine , looking at the code you shared I think the issue is how you are configuring the middleware, I ran into a similar issue where the number of messages I was setting to keep exceeded the amount of tokens set in the trigger, for that reason the summarization never happens, try instead using a different configuration:

createDeepAgent({
  summarization: {
    trigger: { tokens: 4000 },
    keep: { tokens: 2000 },
    model: someModel,
  }
})

If your keep setting is “last 20 messages”, and those last 20 messages by themselves already exceed 4000 tokens, then:

  • There is no cutoff that can reduce the context below the trigger threshold while still keeping 20 messages.

determineCutoffIndex() computes the cutoff as messages.length - keep.value when keep.type === "messages".
That means summarization only ever removes messages older than the last 20; it never touches the last 20.

<!-- gh-comment-id:4307592259 --> @Oscar-Umana commented on GitHub (Apr 23, 2026): Hey @xinzine , looking at the code you shared I think the issue is how you are configuring the middleware, I ran into a similar issue where the number of messages I was setting to `keep` exceeded the amount of tokens set in the trigger, for that reason the summarization never happens, try instead using a different configuration: ```typescript createDeepAgent({ summarization: { trigger: { tokens: 4000 }, keep: { tokens: 2000 }, model: someModel, } }) ``` If your `keep` setting is “last 20 messages”, and those last 20 messages by themselves already exceed 4000 tokens, then: - There is no cutoff that can reduce the context below the trigger threshold while still keeping 20 messages. `determineCutoffIndex()` computes the cutoff as `messages.length - keep.value` when `keep.type === "messages"`. That means summarization only ever removes messages older than the last 20; it never touches the last 20.
yindo commented 2026-06-05 17:21:20 -04:00
Author
Owner
Copy Link
Copy Source

@hntrl commented on GitHub (Jun 2, 2026):

Hey all! There's a couple of different levers for customizing compaction. We generally take the stance that the "when" to compact is a harness level detail, and giving people the option to configure this works against the pre built prompting that we build around.

That being said, there's an issue in how we determine how compaction should work if the model is missing the model profile:

const model = new ChatOpenAI("qwen3.6-plus");

You can remediate this by detailing what the max tokens look like for a moel

// deepagents will pick up on this and use this value to calculate when to compact
// (currently when tokens > 85% of maxInputTokens)
model.profile = {
  maxInputTokens: 50_000
};

We do offer a mechanism for overriding the summarization middleware that comes with deepagents on a per model basis in case this truly isn't enough:

  • Profiles: use this when you want more explicit configuration over summarization
<!-- gh-comment-id:4606592743 --> @hntrl commented on GitHub (Jun 2, 2026): Hey all! There's a couple of different levers for customizing compaction. We generally take the stance that the "when" to compact is a harness level detail, and giving people the option to configure this works against the pre built prompting that we build around. That being said, there's an issue in how we determine how compaction should work if the model is missing the **model profile**: ```ts const model = new ChatOpenAI("qwen3.6-plus"); ``` You can remediate this by detailing what the max tokens look like for a moel ```ts // deepagents will pick up on this and use this value to calculate when to compact // (currently when tokens > 85% of maxInputTokens) model.profile = { maxInputTokens: 50_000 }; ``` We do offer a mechanism for overriding the summarization middleware that comes with deepagents on a per model basis in case this truly isn't enough: * [Profiles](https://docs.langchain.com/oss/javascript/deepagents/profiles): use this when you want more explicit configuration over summarization
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
enhancement
help wanted
needs-response
pull-request
Mirrored from GitHub Pull Request
 stale
triage: high-impact
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/deepagentsjs#263
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?