anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 10:15:31 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[FEATURE]: Google/VertexAI Context Caching #4172

New Issue
Open
opened 2026-02-16 17:42:53 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 17:42:53 -05:00
Owner
Copy Link
Copy Source

Originally created by @thisisryanswift on GitHub (Jan 4, 2026).

Originally assigned to: @thdxr on GitHub.

While caching support is already implemented for Anthropic, OpenAI, and Bedrock, it's missing for Google/VertexAI. Caching enables up to a 90% cost reduction and significantly reduces latency for long-running sessions.

Proposed Changes

1. Provider Transformation

In packages/opencode/src/provider/transform.ts, the applyCaching function currently excludes Google. We need to:

  • Update applyCaching to recognize Gemini models.
  • Add Google-specific caching configurations to the providerOptions object.
  • Ensure ProviderTransform.options correctly sets up the environment for caching when using @ai-sdk/google or @ai-sdk/google-vertex.

2. Usage Tracking

In packages/opencode/src/session/index.ts, the getUsage function needs to be updated to handle Gemini's metadata:

  • Currently, it handles anthropic and bedrock specific metadata for cache writes.
  • We need to add logic to extract cachedInputTokens (or the equivalent Google metadata) to ensure costs are calculated accurately and displayed correctly in the TUI/Stats.

3. LLM Orchestration

In packages/opencode/src/session/llm.ts, verify that the streamText call correctly passes through the necessary providerOptions or experimental_cache parameters required by the AI SDK for Google context caching.

Context

Gemini 3 Flash offers a 10x cost improvement for cached tokens ($0.05/1M vs $0.50/1M), making this optimization essential for high-volume usage. Reference: Gemini Pricing & Features.

Originally created by @thisisryanswift on GitHub (Jan 4, 2026). Originally assigned to: @thdxr on GitHub. While caching support is already implemented for Anthropic, OpenAI, and Bedrock, it's missing for Google/VertexAI. Caching enables up to a 90% cost reduction and significantly reduces latency for long-running sessions. ### Proposed Changes #### 1. Provider Transformation In `packages/opencode/src/provider/transform.ts`, the `applyCaching` function currently excludes Google. We need to: - Update `applyCaching` to recognize Gemini models. - Add Google-specific caching configurations to the `providerOptions` object. - Ensure `ProviderTransform.options` correctly sets up the environment for caching when using `@ai-sdk/google` or `@ai-sdk/google-vertex`. #### 2. Usage Tracking In `packages/opencode/src/session/index.ts`, the `getUsage` function needs to be updated to handle Gemini's metadata: - Currently, it handles `anthropic` and `bedrock` specific metadata for cache writes. - We need to add logic to extract `cachedInputTokens` (or the equivalent Google metadata) to ensure costs are calculated accurately and displayed correctly in the TUI/Stats. #### 3. LLM Orchestration In `packages/opencode/src/session/llm.ts`, verify that the `streamText` call correctly passes through the necessary `providerOptions` or `experimental_cache` parameters required by the AI SDK for Google context caching. ### Context Gemini 3 Flash offers a 10x cost improvement for cached tokens ($0.05/1M vs $0.50/1M), making this optimization essential for high-volume usage. Reference: [Gemini Pricing & Features](https://ai.google.dev/pricing).
yindo commented 2026-02-16 17:42:53 -05:00
Author
Owner
Copy Link
Copy Source

@thisisryanswift commented on GitHub (Jan 4, 2026):

After some additional discovery, implicit caching on Google models was already working server-side, but the cached token counts weren't being recorded in session logs since Google returns them in a different metadata field (usageMetadata.cachedContentTokenCount) than OpenCode was expecting.

Should be a one-line change to ensure OpenCode is recording cached tokens. Will submit a PR in a sec

@thisisryanswift commented on GitHub (Jan 4, 2026): After some additional discovery, implicit caching on Google models was already working server-side, but the cached token counts weren't being recorded in session logs since Google returns them in a different metadata field (usageMetadata.cachedContentTokenCount) than OpenCode was expecting. Should be a one-line change to ensure OpenCode is recording cached tokens. Will submit a PR in a sec
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#4172
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?