anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[PR #12074] fix(session): include cache.write tokens in isOverflow() context calc… #14057

New Issue
Open
opened 2026-02-16 18:18:52 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 18:18:52 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/anomalyco/opencode/pull/12074

State: open
Merged: No

PR: fix(session): include cache.write tokens in isOverflow() context calculation

Relates to #10017
Relates to #10634

Title

fix(session): include cache.write tokens in isOverflow() context calculation

Summary

isOverflow() excludes cache.write (cache_creation_input_tokens) from its token count, causing compaction to trigger late — sometimes well past the intended threshold. This is a one-line fix adding input.tokens.cache.write to the sum on line 35 of compaction.ts.

Problem

When Anthropic models use prompt caching, the API response partitions total input tokens into three disjoint categories:

total_input = input_tokens + cache_read_input_tokens + cache_creation_input_tokens

This is documented by Anthropic: these three fields are mutually exclusive partitions — a token counted in cache_creation_input_tokens is NOT also counted in input_tokens.

OpenCode correctly extracts all three into MessageV2.Assistant.tokens:

  • input ← input_tokens
  • cache.read ← cache_read_input_tokens
  • cache.write ← cache_creation_input_tokens

However, isOverflow() only sums two of the three input partitions:

// compaction.ts line 35 (current)
const count = input.tokens.input + input.tokens.cache.read + input.tokens.output

This means cache.write tokens — which represent real input tokens occupying context space — are invisible to the overflow check. When a session has significant cache creation activity, isOverflow() systematically underreports context utilization and compaction triggers late.

Evidence

Debug log from a real session shows the magnitude of the discrepancy at a single turn:

Metric Value
input_tokens 6,321
cache_read_input_tokens 44,539
cache_creation_input_tokens 62,900
output_tokens 2,746
isOverflow() count (current) 53,606 (input + cache.read + output)
Actual input tokens 116,506 (input + cache.read + cache.write + output)
Reported utilization ~24% of usable capacity
Actual utilization ~54% of usable capacity

The 62,900 cache_creation_input_tokens were completely excluded from the overflow calculation. The function reported ~24% utilization when the actual context usage was ~54%.

This pattern compounds across turns. As the session progresses and cache stabilizes (more reads, fewer writes), the gap narrows — but by then the session may already be well past the point where compaction should have triggered.

Community Signals

Multiple community PRs have worked around symptoms consistent with late compaction:

  • #6562: Adds a configurable buffer to the overflow threshold — effectively compensating for the undercount by lowering the trigger point
  • #9656: Alternative compaction trigger adjustments

These workarounds address the symptom (compaction triggers too late) without identifying the root cause (cache.write exclusion from the token count).

The Fix

One-line change — add input.tokens.cache.write to the token sum:

// packages/opencode/src/session/compaction.ts, line 35
- const count = input.tokens.input + input.tokens.cache.read + input.tokens.output
+ const count = input.tokens.input + input.tokens.cache.read + input.tokens.cache.write + input.tokens.output

Why this is correct

  1. Anthropic's token partitioning is disjoint: input_tokens, cache_read_input_tokens, and cache_creation_input_tokens are mutually exclusive. All three represent real tokens sent to the model that occupy context window space.

  2. The type system already has the data: MessageV2.Assistant.tokens.cache.write is defined and populated. The StepFinishPart schema also tracks cache.write. The data flows correctly through the system — it's just not used in the overflow check.

  3. No impact on non-caching providers: For providers that don't support prompt caching, cache.write is 0, so the sum is unchanged.

  4. Reasoning tokens are intentionally excluded: Note that tokens.reasoning is also not included in the count. This is correct — reasoning tokens are billed separately and don't occupy the input context window. The current code correctly excludes reasoning but incorrectly excludes cache.write.

Testing Notes

  • Anthropic models with prompt caching: Verify compaction triggers at appropriate context utilization. Sessions should no longer run significantly past the intended overflow threshold before compaction kicks in.
  • Non-caching providers: Verify no behavioral change (cache.write = 0, sum unchanged).
  • TUI context display: The TUI percentage display uses a different calculation path and is not affected by this change.
  • Debug logging: Adding temporary logging of cache.write values alongside existing token logging can help verify the fix is working as expected.

Risk Assessment

Low risk. This is a pure additive fix to arithmetic that was already intended to sum all input tokens. It makes isOverflow() consistent with the token partitioning model that the rest of the codebase already implements correctly.

The primary behavioral change is that compaction will trigger earlier for Anthropic sessions — which is the correct behavior. Sessions that previously ran past the overflow threshold due to excluded cache.write tokens will now trigger compaction at the intended point.

**Original Pull Request:** https://github.com/anomalyco/opencode/pull/12074 **State:** open **Merged:** No --- # PR: fix(session): include cache.write tokens in isOverflow() context calculation Relates to #10017 Relates to #10634 --- ## Title `fix(session): include cache.write tokens in isOverflow() context calculation` ## Summary `isOverflow()` excludes `cache.write` (`cache_creation_input_tokens`) from its token count, causing compaction to trigger late — sometimes well past the intended threshold. This is a one-line fix adding `input.tokens.cache.write` to the sum on line 35 of `compaction.ts`. ## Problem When Anthropic models use prompt caching, the API response partitions total input tokens into three disjoint categories: ``` total_input = input_tokens + cache_read_input_tokens + cache_creation_input_tokens ``` This is [documented by Anthropic](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching#tracking-cache-performance): these three fields are mutually exclusive partitions — a token counted in `cache_creation_input_tokens` is NOT also counted in `input_tokens`. OpenCode correctly extracts all three into `MessageV2.Assistant.tokens`: - `input` ← `input_tokens` - `cache.read` ← `cache_read_input_tokens` - `cache.write` ← `cache_creation_input_tokens` However, `isOverflow()` only sums two of the three input partitions: ```typescript // compaction.ts line 35 (current) const count = input.tokens.input + input.tokens.cache.read + input.tokens.output ``` This means `cache.write` tokens — which represent real input tokens occupying context space — are invisible to the overflow check. When a session has significant cache creation activity, `isOverflow()` systematically underreports context utilization and compaction triggers late. ## Evidence Debug log from a real session shows the magnitude of the discrepancy at a single turn: | Metric | Value | |--------|-------| | `input_tokens` | 6,321 | | `cache_read_input_tokens` | 44,539 | | `cache_creation_input_tokens` | 62,900 | | `output_tokens` | 2,746 | | **isOverflow() count (current)** | **53,606** (input + cache.read + output) | | **Actual input tokens** | **116,506** (input + cache.read + cache.write + output) | | **Reported utilization** | ~24% of usable capacity | | **Actual utilization** | ~54% of usable capacity | The 62,900 `cache_creation_input_tokens` were completely excluded from the overflow calculation. The function reported ~24% utilization when the actual context usage was ~54%. This pattern compounds across turns. As the session progresses and cache stabilizes (more reads, fewer writes), the gap narrows — but by then the session may already be well past the point where compaction should have triggered. ### Community Signals Multiple community PRs have worked around symptoms consistent with late compaction: - **#6562**: Adds a configurable buffer to the overflow threshold — effectively compensating for the undercount by lowering the trigger point - **#9656**: Alternative compaction trigger adjustments These workarounds address the symptom (compaction triggers too late) without identifying the root cause (cache.write exclusion from the token count). ## The Fix One-line change — add `input.tokens.cache.write` to the token sum: ```diff // packages/opencode/src/session/compaction.ts, line 35 - const count = input.tokens.input + input.tokens.cache.read + input.tokens.output + const count = input.tokens.input + input.tokens.cache.read + input.tokens.cache.write + input.tokens.output ``` ### Why this is correct 1. **Anthropic's token partitioning is disjoint**: `input_tokens`, `cache_read_input_tokens`, and `cache_creation_input_tokens` are mutually exclusive. All three represent real tokens sent to the model that occupy context window space. 2. **The type system already has the data**: `MessageV2.Assistant.tokens.cache.write` is defined and populated. The `StepFinishPart` schema also tracks `cache.write`. The data flows correctly through the system — it's just not used in the overflow check. 3. **No impact on non-caching providers**: For providers that don't support prompt caching, `cache.write` is `0`, so the sum is unchanged. 4. **Reasoning tokens are intentionally excluded**: Note that `tokens.reasoning` is also not included in the count. This is correct — reasoning tokens are billed separately and don't occupy the input context window. The current code correctly excludes reasoning but incorrectly excludes cache.write. ## Testing Notes - **Anthropic models with prompt caching**: Verify compaction triggers at appropriate context utilization. Sessions should no longer run significantly past the intended overflow threshold before compaction kicks in. - **Non-caching providers**: Verify no behavioral change (cache.write = 0, sum unchanged). - **TUI context display**: The TUI percentage display uses a different calculation path and is not affected by this change. - **Debug logging**: Adding temporary logging of `cache.write` values alongside existing token logging can help verify the fix is working as expected. ## Risk Assessment **Low risk**. This is a pure additive fix to arithmetic that was already intended to sum all input tokens. It makes `isOverflow()` consistent with the token partitioning model that the rest of the codebase already implements correctly. The primary behavioral change is that compaction will trigger **earlier** for Anthropic sessions — which is the correct behavior. Sessions that previously ran past the overflow threshold due to excluded cache.write tokens will now trigger compaction at the intended point.
yindo added the pull-request label 2026-02-16 18:18:52 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#14057
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?