langchain-ai/docs
1
0
Fork 0
mirror of https://github.com/langchain-ai/docs.git synced 2026-07-22 04:15:25 -04:00
Code Issues 490 Packages Projects Releases Wiki Activity

[GH-ISSUE #3163] [langsmith]: OTEL Exporter Missing Cache Token Breakdown (claude-agent-sdk integration) #2709

New Issue
Open
opened 2026-06-05 17:26:24 -04:00 by yindo · 0 comments
yindo commented 2026-06-05 17:26:24 -04:00
Owner
Copy Link
Copy Source

Originally created by @saurabhbidwai-hoc on GitHub (Mar 17, 2026).
Original GitHub issue: https://github.com/langchain-ai/docs/issues/3163

Type of issue

issue / bug

Language

Python

Description

OTEL Exporter Missing Cache Token Breakdown (claude-agent-sdk integration)

Issue for: langchain-ai/langsmith-sdk

Title: langsmith[claude-agent-sdk] OTEL exporter does not emit cache_read_input_tokens / cache_creation_input_tokens, causing ~2.5x cost overestimate in observability platforms

Summary

The langsmith[claude-agent-sdk] OTEL exporter sends generation spans to observability platforms (Langfuse, etc.) with only {input, output, total} token counts. It does not include cache_read_input_tokens or cache_creation_input_tokens in the usage details.

This causes downstream platforms to price ALL input tokens at the full rate ($3/MTok for Sonnet 4.5), when in practice most are cache reads priced at $0.30/MTok (90% discount). The result is a ~2.5x cost overestimate.

Reproduction

Setup

  • claude-agent-sdk>=0.1.6
  • langsmith[claude-agent-sdk]>=0.7.0
  • langsmith[otel]>=0.7.0
  • langfuse>=3.0.0
  • Model: claude-sonnet-4-5-20250929
  • OTEL configured via configure_claude_agent_sdk() + propagate_attributes()

Observed Cost Discrepancy

Source Cost How Calculated
Claude SDK (ResultMessage.total_cost_usd) $1.47 Anthropic's authoritative billing (cache-aware)
Langfuse (from OTEL spans) $3.71 Sums per-turn tokens at full input rate

Evidence from Langfuse API

Session 69b94ab55306839e627a1c30 contains 2 traces with 46 total generations.

Each generation's usage only contains:

{
  "usageDetails": {"input": 44285, "output": 6275, "total": 50560},
  "usage": {"unit": "TOKENS", "input": 44285, "output": 6275, "total": 50560}
}

Missing fields:

  • cache_read_input_tokens - not present
  • cache_creation_input_tokens - not present

Langfuse's cost calculation:

  • Total input tokens across all turns: 1,056,135
  • Priced at $3/MTok (full rate): $3.17 input cost
  • Total output tokens: 27,444
  • Priced at $15/MTok: $0.41 output cost
  • Total: $3.58 (vs Anthropic's $1.47)

The 1,056,135 "input" tokens include massive amounts of cached context re-sent each turn. Anthropic bills these at $0.30/MTok, but Langfuse has no way to know because the OTEL spans don't include the breakdown.

Per-Turn Input Token Pattern (Shows Caching)

Turn 1:  30,740 input tokens
Turn 2:  31,640
Turn 3:  32,894
...
Turn 44: 43,565
Turn 45: 44,285

Each turn re-sends the growing conversation. Most of the ~30K+ base tokens hit the cache each turn, but the OTEL exporter reports them all as regular input tokens.

Expected Behavior

The OTEL exporter should emit cache-aware token breakdowns per generation span:

{
  "usageDetails": {
    "input": 2500,
    "output": 6275,
    "cache_read_input_tokens": 41000,
    "cache_creation_input_tokens": 785,
    "total": 50560
  }
}

This would allow observability platforms to:

  1. Price cached tokens correctly ($0.30/MTok vs $3/MTok for Sonnet)
  2. Show accurate cost tracking that matches Anthropic's billing
  3. Provide useful cache hit rate metrics

Related

  • PR #685 (anthropics/claude-agent-sdk-python): feat(types): preserve per-turn usage on AssistantMessage - Now provides cache_read_input_tokens and cache_creation_input_tokens on AssistantMessage.usage. The OTEL exporter should map these to the generation span usage details.

  • Issue #673 (anthropics/claude-agent-sdk-python): Claude Agent SDK: OTEL Exporter Missing Prompt Caching Token Breakdown

Impact

Any user of langsmith[claude-agent-sdk] with Anthropic's prompt caching (which is automatic for conversations) will see significantly inflated costs in their observability platform. For long multi-turn sessions, the overestimate can be 2-3x or more.

Originally created by @saurabhbidwai-hoc on GitHub (Mar 17, 2026). Original GitHub issue: https://github.com/langchain-ai/docs/issues/3163 ### Type of issue issue / bug ### Language Python ### Description # OTEL Exporter Missing Cache Token Breakdown (claude-agent-sdk integration) ## Issue for: `langchain-ai/langsmith-sdk` **Title:** `langsmith[claude-agent-sdk]` OTEL exporter does not emit `cache_read_input_tokens` / `cache_creation_input_tokens`, causing ~2.5x cost overestimate in observability platforms --- ## Summary The `langsmith[claude-agent-sdk]` OTEL exporter sends generation spans to observability platforms (Langfuse, etc.) with only `{input, output, total}` token counts. It does **not** include `cache_read_input_tokens` or `cache_creation_input_tokens` in the usage details. This causes downstream platforms to price ALL input tokens at the full rate ($3/MTok for Sonnet 4.5), when in practice most are cache reads priced at $0.30/MTok (90% discount). The result is a **~2.5x cost overestimate**. ## Reproduction ### Setup - `claude-agent-sdk>=0.1.6` - `langsmith[claude-agent-sdk]>=0.7.0` - `langsmith[otel]>=0.7.0` - `langfuse>=3.0.0` - Model: `claude-sonnet-4-5-20250929` - OTEL configured via `configure_claude_agent_sdk()` + `propagate_attributes()` ### Observed Cost Discrepancy | Source | Cost | How Calculated | |--------|------|----------------| | **Claude SDK** (`ResultMessage.total_cost_usd`) | **$1.47** | Anthropic's authoritative billing (cache-aware) | | **Langfuse** (from OTEL spans) | **$3.71** | Sums per-turn tokens at full input rate | ### Evidence from Langfuse API Session `69b94ab55306839e627a1c30` contains 2 traces with 46 total generations. **Each generation's usage only contains:** ```json { "usageDetails": {"input": 44285, "output": 6275, "total": 50560}, "usage": {"unit": "TOKENS", "input": 44285, "output": 6275, "total": 50560} } ``` **Missing fields:** - `cache_read_input_tokens` - not present - `cache_creation_input_tokens` - not present **Langfuse's cost calculation:** - Total input tokens across all turns: 1,056,135 - Priced at $3/MTok (full rate): $3.17 input cost - Total output tokens: 27,444 - Priced at $15/MTok: $0.41 output cost - **Total: $3.58** (vs Anthropic's $1.47) The 1,056,135 "input" tokens include massive amounts of cached context re-sent each turn. Anthropic bills these at $0.30/MTok, but Langfuse has no way to know because the OTEL spans don't include the breakdown. ### Per-Turn Input Token Pattern (Shows Caching) ``` Turn 1: 30,740 input tokens Turn 2: 31,640 Turn 3: 32,894 ... Turn 44: 43,565 Turn 45: 44,285 ``` Each turn re-sends the growing conversation. Most of the ~30K+ base tokens hit the cache each turn, but the OTEL exporter reports them all as regular `input` tokens. ## Expected Behavior The OTEL exporter should emit cache-aware token breakdowns per generation span: ```json { "usageDetails": { "input": 2500, "output": 6275, "cache_read_input_tokens": 41000, "cache_creation_input_tokens": 785, "total": 50560 } } ``` This would allow observability platforms to: 1. Price cached tokens correctly ($0.30/MTok vs $3/MTok for Sonnet) 2. Show accurate cost tracking that matches Anthropic's billing 3. Provide useful cache hit rate metrics ## Related - **PR #685** (`anthropics/claude-agent-sdk-python`): [feat(types): preserve per-turn usage on AssistantMessage](https://github.com/anthropics/claude-agent-sdk-python/pull/685) - Now provides `cache_read_input_tokens` and `cache_creation_input_tokens` on `AssistantMessage.usage`. The OTEL exporter should map these to the generation span usage details. - **Issue #673** (`anthropics/claude-agent-sdk-python`): Claude Agent SDK: OTEL Exporter Missing Prompt Caching Token Breakdown ## Impact Any user of `langsmith[claude-agent-sdk]` with Anthropic's prompt caching (which is automatic for conversations) will see significantly inflated costs in their observability platform. For long multi-turn sessions, the overestimate can be 2-3x or more.
yindo added the langsmithexternal labels 2026-06-05 17:26:24 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
deepagents
docs-infra
external
external
help wanted
integration
internal
internal
javascript
js/ts
langchain
langgraph
langsmith
needs-title
pull-request
Mirrored from GitHub Pull Request
 python
reference
No Label external langsmith
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/docs#2709
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?