anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 10:15:31 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[PR #5422] feat(provider): add provider-specific cache configuration system (significant token usage reduction) #11395

New Issue
Open
opened 2026-02-16 18:16:13 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 18:16:13 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/anomalyco/opencode/pull/5422

State: open
Merged: No

Summary

Implements comprehensive ProviderConfig system for provider-specific caching and prompt optimization.

Closes #5416

Test Results with Claude Opus 4.5 (my primary target for improvement)

A/B testing with identical prompts (same session, same agent) comparing legacy vs optimized behavior:

Metric Legacy Optimized Improvement
Cache writes (post-warmup) 18,417 tokens ~10,340 tokens 44% reduction
Effective cost (3rd prompt) 13,021 tokens 3,495 tokens 73% reduction
Initial cache write 16,211 tokens 17,987 tokens +11% (expected)
Cache hit rate 100% 100% Same

The slightly larger initial cache write (+11%) is quickly amortized by dramatically fewer cache invalidations in subsequent requests.

Provider testing

I tested with providers I have access to. I can confirm it works with with Anthropic (Opus 4.5), Google (Gemini 3 Pro preview), and Mistral (Devstral 2).

Changes

  • Add ProviderConfig namespace with defaults for 19+ providers
  • Support three caching paradigms:
    • Explicit breakpoint (Anthropic, Bedrock, Google Vertex Anthropic)
    • Automatic prefix (OpenAI, Azure, GitHub Copilot, DeepSeek)
    • Implicit/content-based (Google/Gemini)
  • Add tool sorting for cache consistency across requests
  • Add tool caching for explicit breakpoint providers
  • Add user config overrides via opencode.json (provider and agent level)
  • Simplify system message handling with combineSystemMessages boolean

Config Priority

Provider defaults → User provider config → User agent config

Area for future optimization

Currently, models.dev doesn't provide information regarding minimum cache requirements or prompt requirements, so this had to be written out as configuration. It would be ideal if the model definitions were updated with this detail. Until that point, as providers/models are added or updated, for optimal performance the configuration should be updated to match.

New Files

  • src/provider/config.ts (874 lines)
  • test/provider/config.test.ts (215 tests)

Example Config

{
  "provider": {
    "anthropic": {
      "cache": {
        "enabled": true,
        "ttl": "1h",
        "minTokens": 2048
      }
    }
  },
  "agent": {
    "plan": {
      "cache": {
        "ttl": "1h"
      }
    }
  }
}
**Original Pull Request:** https://github.com/anomalyco/opencode/pull/5422 **State:** open **Merged:** No --- ## Summary Implements comprehensive ProviderConfig system for provider-specific caching and prompt optimization. Closes #5416 ## Test Results with Claude Opus 4.5 (my primary target for improvement) A/B testing with identical prompts (same session, same agent) comparing legacy vs optimized behavior: | Metric | Legacy | Optimized | Improvement | |--------|--------|-----------|-------------| | Cache writes (post-warmup) | 18,417 tokens | ~10,340 tokens | **44% reduction** | | Effective cost (3rd prompt) | 13,021 tokens | 3,495 tokens | **73% reduction** | | Initial cache write | 16,211 tokens | 17,987 tokens | +11% (expected) | | Cache hit rate | 100% | 100% | Same | The slightly larger initial cache write (+11%) is quickly amortized by dramatically fewer cache invalidations in subsequent requests. ## Provider testing I tested with providers I have access to. I can confirm it works with with Anthropic (Opus 4.5), Google (Gemini 3 Pro preview), and Mistral (Devstral 2). ## Changes - Add `ProviderConfig` namespace with defaults for 19+ providers - Support three caching paradigms: - **Explicit breakpoint** (Anthropic, Bedrock, Google Vertex Anthropic) - **Automatic prefix** (OpenAI, Azure, GitHub Copilot, DeepSeek) - **Implicit/content-based** (Google/Gemini) - Add tool sorting for cache consistency across requests - Add tool caching for explicit breakpoint providers - Add user config overrides via `opencode.json` (provider and agent level) - Simplify system message handling with `combineSystemMessages` boolean ## Config Priority Provider defaults → User provider config → User agent config ## Area for future optimization Currently, models.dev doesn't provide information regarding minimum cache requirements or prompt requirements, so this had to be written out as configuration. It would be ideal if the model definitions were updated with this detail. Until that point, as providers/models are added or updated, for optimal performance the configuration should be updated to match. ## New Files - `src/provider/config.ts` (874 lines) - `test/provider/config.test.ts` (215 tests) ## Example Config ```json { "provider": { "anthropic": { "cache": { "enabled": true, "ttl": "1h", "minTokens": 2048 } } }, "agent": { "plan": { "cache": { "ttl": "1h" } } } } ```
yindo added the pull-request label 2026-02-16 18:16:13 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#11395
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?