langchain-ai/deepagentsjs
1
0
Fork 0
mirror of https://github.com/langchain-ai/deepagentsjs.git synced 2026-07-18 16:04:44 -04:00
Code Issues 42 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #550] cache_control leaks to non-Anthropic providers on modelFallbackMiddleware swap #276

New Issue
Closed
opened 2026-06-05 17:21:25 -04:00 by yindo · 0 comments
yindo commented 2026-06-05 17:21:25 -04:00
Owner
Copy Link
Copy Source

Originally created by @antonnak on GitHub (May 22, 2026).
Original GitHub issue: https://github.com/langchain-ai/deepagentsjs/issues/550

Summary

createCacheBreakpointMiddleware and createMemoryMiddleware write Anthropic-specific cache_control: { type: "ephemeral" } markers into the request payload based on a boot-time decision in createDeepAgent ("is the primary model Anthropic?"). When LangChain's modelFallbackMiddleware swaps request.model to a non-Anthropic provider mid-flight (e.g. on Anthropic 5xx), the markers leak through and the fallback provider rejects the request:

400 Unknown parameter: 'messages[0].content[N].cache_control'

The fallback chain that exists precisely to survive Anthropic outages is destroyed by a payload-shape bug. End users see a 5xx instead of a slower-but-correct answer from the fallback model.

Reproduction

Minimal: create an agent with Anthropic primary + OpenAI fallback via modelFallbackMiddleware. Force the primary call to fail. Capture the OpenAI request body — it carries cache_control blocks. OpenAI 400s on the unknown field.

Production trace where this surfaced: Anthropic returned 529 overloaded_error, modelFallbackMiddleware walked Opus → Sonnet → gpt-5.2, every gpt-5.2 attempt failed with the 400 above.

Root cause

Two writers, both with boot-time gating but no per-call check:

  1. libs/deepagents/src/middleware/cache.ts — createCacheBreakpointMiddleware().wrapModelCall tags systemMessage.content[N] with cache_control on every call. No request.model inspection. Installed conditionally in agent.ts:218-226 when isAnthropicModel(model) is true at boot.

  2. libs/deepagents/src/middleware/memory.ts:342-344 — writes cache_control onto the memory block, gated on the addCacheControl flag set once at agent creation (agent.ts:375: addCacheControl: anthropicModel).

Both gates are correct at boot. Neither survives modelFallbackMiddleware swapping request.model at request time.

Why fix it in the writers

modelFallbackMiddleware (LangChain side) spreads the request shape blindly: handler({ ...request, model }). It can't strip provider-specific markers because it doesn't know which fields are provider-specific — that knowledge lives in the writer.

The classification helper already exists: isAnthropicModel in libs/deepagents/src/utils.ts:9-18, which handles string / ConfigurableModel / ChatAnthropic instances. Reusing it inside wrapModelCall is one if statement per writer.

Proposed fix

```ts
// cache.ts — inside wrapModelCall, before existing logic
if (!isAnthropicModel(request.model)) return handler(request);

// memory.ts — replace the boot-time `addCacheControl` flag with a
// per-call check on request.model
```

Boot-time gate in agent.ts can stay (small perf win when primary is non-Anthropic — middleware isn't installed at all). The per-call check covers the fallback case where the middleware is installed but the model has been swapped.

Related

  • langchain-ai/deepagentsjs#280 — different toggle, but same area
  • langchain-ai/langchain#33709 — Python, same bug class

Workaround for users hitting this today

Strip cache_control from the request inside a custom middleware positioned immediately inside modelFallbackMiddleware. Not pretty but unblocks production until this lands.

Out of scope for this issue

request.modelSettings.cache_control (written by anthropicPromptCachingMiddleware in LangChain core) can also leak across the swap. That's a LangChain-side concern, not deepagents.

Originally created by @antonnak on GitHub (May 22, 2026). Original GitHub issue: https://github.com/langchain-ai/deepagentsjs/issues/550 ## Summary `createCacheBreakpointMiddleware` and `createMemoryMiddleware` write Anthropic-specific `cache_control: { type: "ephemeral" }` markers into the request payload based on a **boot-time** decision in `createDeepAgent` ("is the primary model Anthropic?"). When LangChain's `modelFallbackMiddleware` swaps `request.model` to a non-Anthropic provider mid-flight (e.g. on Anthropic 5xx), the markers leak through and the fallback provider rejects the request: ``` 400 Unknown parameter: 'messages[0].content[N].cache_control' ``` The fallback chain that exists *precisely* to survive Anthropic outages is destroyed by a payload-shape bug. End users see a 5xx instead of a slower-but-correct answer from the fallback model. ## Reproduction Minimal: create an agent with Anthropic primary + OpenAI fallback via `modelFallbackMiddleware`. Force the primary call to fail. Capture the OpenAI request body — it carries `cache_control` blocks. OpenAI 400s on the unknown field. Production trace where this surfaced: Anthropic returned 529 `overloaded_error`, `modelFallbackMiddleware` walked Opus → Sonnet → gpt-5.2, every gpt-5.2 attempt failed with the 400 above. ## Root cause Two writers, both with boot-time gating but no per-call check: 1. **`libs/deepagents/src/middleware/cache.ts`** — `createCacheBreakpointMiddleware().wrapModelCall` tags `systemMessage.content[N]` with `cache_control` on every call. No `request.model` inspection. Installed conditionally in `agent.ts:218-226` when `isAnthropicModel(model)` is true at boot. 2. **`libs/deepagents/src/middleware/memory.ts:342-344`** — writes `cache_control` onto the memory block, gated on the `addCacheControl` flag set once at agent creation (`agent.ts:375`: `addCacheControl: anthropicModel`). Both gates are correct at boot. Neither survives `modelFallbackMiddleware` swapping `request.model` at request time. ## Why fix it in the writers `modelFallbackMiddleware` (LangChain side) spreads the request shape blindly: `handler({ ...request, model })`. It can't strip provider-specific markers because it doesn't know which fields are provider-specific — that knowledge lives in the writer. The classification helper already exists: `isAnthropicModel` in `libs/deepagents/src/utils.ts:9-18`, which handles string / `ConfigurableModel` / `ChatAnthropic` instances. Reusing it inside `wrapModelCall` is one `if` statement per writer. ## Proposed fix \`\`\`ts // cache.ts — inside wrapModelCall, before existing logic if (!isAnthropicModel(request.model)) return handler(request); // memory.ts — replace the boot-time \`addCacheControl\` flag with a // per-call check on request.model \`\`\` Boot-time gate in `agent.ts` can stay (small perf win when primary is non-Anthropic — middleware isn't installed at all). The per-call check covers the fallback case where the middleware *is* installed but the model has been swapped. ## Related - `langchain-ai/deepagentsjs#280` — different toggle, but same area - `langchain-ai/langchain#33709` — Python, same bug class ## Workaround for users hitting this today Strip `cache_control` from the request inside a custom middleware positioned immediately inside `modelFallbackMiddleware`. Not pretty but unblocks production until this lands. ## Out of scope for this issue `request.modelSettings.cache_control` (written by `anthropicPromptCachingMiddleware` in LangChain core) can also leak across the swap. That's a LangChain-side concern, not deepagents.
yindo closed this issue 2026-06-05 17:21:25 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
enhancement
help wanted
needs-response
pull-request
Mirrored from GitHub Pull Request
 stale
triage: high-impact
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/deepagentsjs#276
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?