anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-25 20:55:42 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[FEATURE]: Anthropic (and others) caching improvement #3487

New Issue
Open
opened 2026-02-16 17:40:19 -05:00 by yindo · 2 comments
yindo commented 2026-02-16 17:40:19 -05:00
Owner
Copy Link
Copy Source

Originally created by @ormandj on GitHub (Dec 11, 2025).

Feature hasn't been suggested before.

  • I have verified this feature I'm about to request hasn't been suggested before.

Describe the enhancement you want to request

I've recently started utilizing opencode, and saw my token usage was somewhat higher with the same general workflow in opencode that I was using in claude code. After doing a little research, I determined the cache model and prompt structure being used was suboptimal for claude-based models.

I've submitted a PR that attempts to address this, and allows configuration at the provider and per-agent level. Some of my workstreams may run for long periods of time, and have gaps inbetween runs for certain types of agents (for example, review agents may not run frequently, but when they do have a large amount of static context used for their instructions), so allowing TTL to be overridden at the agent level made sense to me.

I did some basic testing with the patch, and it made a significant difference in non-cached vs. cached usage, which with claude pricing, can make a huge difference in the cost of using these LLMs. Unfortunately, the minimum cache size wasn't available programmatically, so I had to build a lookup table for the various models. Basic performance/cache testing is in the PR.

I tried to create the PR in a way that wouldn't negatively impact any other models/providers, but could also be used as a starting point for other models/providers that had specific cache implementation requirements. Later this model can be extended to configuration beyond caching.

Originally created by @ormandj on GitHub (Dec 11, 2025). ### Feature hasn't been suggested before. - [x] I have verified this feature I'm about to request hasn't been suggested before. ### Describe the enhancement you want to request I've recently started utilizing opencode, and saw my token usage was somewhat higher with the same general workflow in opencode that I was using in claude code. After doing a little research, I determined the cache model and prompt structure being used was suboptimal for claude-based models. I've submitted a [PR](https://github.com/sst/opencode/pull/5422) that attempts to address this, and allows configuration at the provider and per-agent level. Some of my workstreams may run for long periods of time, and have gaps inbetween runs for certain types of agents (for example, review agents may not run frequently, but when they do have a large amount of static context used for their instructions), so allowing TTL to be overridden at the agent level made sense to me. I did some basic testing with the patch, and it made a significant difference in non-cached vs. cached usage, which with claude pricing, can make a huge difference in the cost of using these LLMs. Unfortunately, the minimum cache size wasn't available programmatically, so I had to build a lookup table for the various models. Basic performance/cache testing is in the PR. I tried to create the PR in a way that wouldn't negatively impact any other models/providers, but could also be used as a starting point for other models/providers that had specific cache implementation requirements. Later this model can be extended to configuration beyond caching.
yindo added the discussion label 2026-02-16 17:40:19 -05:00
yindo commented 2026-02-16 17:40:19 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Dec 11, 2025):

This issue might be a duplicate of existing issues. Please check:

  • #4317: Feature request for generic /compact command, auto-compaction, and fork-aware conversations (addresses context management and compaction at a broader level)

The proposed solution in #5416 (TTL-based cache configuration for Anthropic) complements the broader compaction strategy discussed in #4317. Both issues aim to optimize token usage, but from different angles - #5416 focuses on cache TTL specifics for Anthropic models, while #4317 addresses core compaction infrastructure.

Feel free to ignore if this is specifically an Anthropic caching implementation that differs from the broader compaction approach.

@github-actions[bot] commented on GitHub (Dec 11, 2025): This issue might be a duplicate of existing issues. Please check: - #4317: Feature request for generic /compact command, auto-compaction, and fork-aware conversations (addresses context management and compaction at a broader level) The proposed solution in #5416 (TTL-based cache configuration for Anthropic) complements the broader compaction strategy discussed in #4317. Both issues aim to optimize token usage, but from different angles - #5416 focuses on cache TTL specifics for Anthropic models, while #4317 addresses core compaction infrastructure. Feel free to ignore if this is specifically an Anthropic caching implementation that differs from the broader compaction approach.
yindo commented 2026-02-16 17:40:19 -05:00
Author
Owner
Copy Link
Copy Source

@konekohana commented on GitHub (Jan 15, 2026):

I'm running into the same issue as well. I'm using OpenRouter for completions and I can see in their logs that the cache expires often with Anthropic models if I go away from the conversation for a few minutes. Using opencode would probably be much cheaper if I could combine it with the 1 hour TTL anthropic offers. I tried to set the header using the per model configuration, but it seems that this can only be used for string header values:

Error: Configuration is invalid at /home/hana/.config/opencode/opencode.json
↳ Invalid input: expected string, received object provider.myprovider.models.openrouter/anthropic/claude-sonnet-4.5.headers.cache_control
@konekohana commented on GitHub (Jan 15, 2026): I'm running into the same issue as well. I'm using OpenRouter for completions and I can see in their logs that the cache expires often with Anthropic models if I go away from the conversation for a few minutes. Using opencode would probably be much cheaper if I could combine it with the 1 hour TTL anthropic offers. I tried to set the header using the per model configuration, but it seems that this can only be used for string header values: ``` Error: Configuration is invalid at /home/hana/.config/opencode/opencode.json ↳ Invalid input: expected string, received object provider.myprovider.models.openrouter/anthropic/claude-sonnet-4.5.headers.cache_control ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label discussion
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#3487
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?