anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

Read tool fails to read image files even with vision-capable models (paste works) #8046

New Issue
Closed
opened 2026-02-16 18:09:00 -05:00 by yindo · 5 comments
yindo commented 2026-02-16 18:09:00 -05:00
Owner
Copy Link
Copy Source

Originally created by @deleon626 on GitHub (Jan 30, 2026).

Originally assigned to: @rekram1-node on GitHub.

Description

When using a vision-capable model through OpenCode, the Read tool fails to read image files from disk, reporting that the model cannot read images. However, pasting the same image directly into the chat works perfectly.

Steps to Reproduce

  1. Configure a vision-capable model (e.g., Gemini 3 Pro, Claude Sonnet 4.5) with proper modalities: 
    "modalities": { "input": ["text", "image", "pdf"], "output": ["text"] }
  2. Have an image file on disk (e.g., screenshot.png)
  3. Ask the model to read the image file using the Read tool
  4. Observe error: "Failed to read image: [path], model may not be able to read images"
  5. Paste the same image directly into chat - it works fine

Expected Behavior

The Read tool should recognize that the model supports vision input (via modalities config) and successfully read image files, just like pasted images work.

Actual Behavior

  • Pasted/dragged images: Works
  • @image.png syntax: Works
  • read(image.png) via Read tool: Fails with model capability error

Configuration

Models already have correct modalities configured:

"antigravity-gemini-3-pro": {
  "modalities": { "input": ["text", "image", "pdf"], "output": ["text"] }
}

Related Issues

  • #3084 - Marked closed, but issue persists
  • #5056 - Marked "fixed in next release" with commit 350a322, but still broken
  • #712 (oh-my-opencode) - Suggests routing images to look-at tool instead of read

Environment

  • OpenCode version: Current
  • OS: macOS
  • Models affected: All vision-capable models (Gemini, Claude, etc.)
  • Modalities: Properly configured with "input": ["text", "image", "pdf"]

Root Cause Analysis

The Read tool appears to:

  1. Read the image file as binary/text
  2. Check if model supports vision
  3. Fail during the handoff between file reading and vision processing

Meanwhile, pasted images bypass the Read tool entirely and are sent as API attachments directly.

Suggested Fix

Either:

  1. Fix Read tool to properly handle image files when model has vision modality
  2. Route image file extensions (.png, .jpg, .jpeg, .gif, .webp) directly to vision/attachment path instead of through Read tool
Originally created by @deleon626 on GitHub (Jan 30, 2026). Originally assigned to: @rekram1-node on GitHub. ## Description When using a vision-capable model through OpenCode, the Read tool fails to read image files from disk, reporting that the model cannot read images. However, pasting the same image directly into the chat works perfectly. ## Steps to Reproduce 1. Configure a vision-capable model (e.g., Gemini 3 Pro, Claude Sonnet 4.5) with proper modalities: ```jsonc "modalities": { "input": ["text", "image", "pdf"], "output": ["text"] } ``` 2. Have an image file on disk (e.g., `screenshot.png`) 3. Ask the model to read the image file using the Read tool 4. Observe error: "Failed to read image: [path], model may not be able to read images" 5. Paste the same image directly into chat - it works fine ## Expected Behavior The Read tool should recognize that the model supports vision input (via modalities config) and successfully read image files, just like pasted images work. ## Actual Behavior - Pasted/dragged images: **Works** - `@image.png` syntax: **Works** - `read(image.png)` via Read tool: **Fails** with model capability error ## Configuration Models already have correct modalities configured: ```jsonc "antigravity-gemini-3-pro": { "modalities": { "input": ["text", "image", "pdf"], "output": ["text"] } } ``` ## Related Issues - #3084 - Marked closed, but issue persists - #5056 - Marked "fixed in next release" with commit 350a322, but still broken - #712 (oh-my-opencode) - Suggests routing images to look-at tool instead of read ## Environment - OpenCode version: Current - OS: macOS - Models affected: All vision-capable models (Gemini, Claude, etc.) - Modalities: Properly configured with `"input": ["text", "image", "pdf"]` ## Root Cause Analysis The Read tool appears to: 1. Read the image file as binary/text 2. Check if model supports vision 3. Fail during the handoff between file reading and vision processing Meanwhile, pasted images bypass the Read tool entirely and are sent as API attachments directly. ## Suggested Fix Either: 1. Fix Read tool to properly handle image files when model has vision modality 2. Route image file extensions (.png, .jpg, .jpeg, .gif, .webp) directly to vision/attachment path instead of through Read tool
yindo closed this issue 2026-02-16 18:09:00 -05:00
yindo commented 2026-02-16 18:09:00 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Jan 30, 2026):

This issue might be a duplicate of existing issues. Please check:

  • #5359: Unable to read images for some models - Similar issue with pasted images showing the model cannot read images despite being image-capable
  • #10150: [BUG] Azure gpt-5.2-codex (vision-capable) is unable to read screenshots - Vision-capable model failing to receive images properly
  • #11304: Images returned from Read tool don't work with OpenAI-compatible providers - Read tool failing with vision-capable models (OpenAI-compatible providers)
  • #6149: First message with image doesn't use MCP vision tool even when available - Related issue about image handling priority and vision tool routing

Feel free to ignore if your specific case differs or involves additional context.

@github-actions[bot] commented on GitHub (Jan 30, 2026): This issue might be a duplicate of existing issues. Please check: - #5359: Unable to read images for some models - Similar issue with pasted images showing the model cannot read images despite being image-capable - #10150: [BUG] Azure gpt-5.2-codex (vision-capable) is unable to read screenshots - Vision-capable model failing to receive images properly - #11304: Images returned from Read tool don't work with OpenAI-compatible providers - Read tool failing with vision-capable models (OpenAI-compatible providers) - #6149: First message with image doesn't use MCP vision tool even when available - Related issue about image handling priority and vision tool routing Feel free to ignore if your specific case differs or involves additional context.
yindo commented 2026-02-16 18:09:00 -05:00
Author
Owner
Copy Link
Copy Source

@OpeOginni commented on GitHub (Jan 30, 2026):

Hey @deleon626 does this nly happen while using antigravity as your model provider?

@OpeOginni commented on GitHub (Jan 30, 2026): Hey @deleon626 does this nly happen while using `antigravity` as your model provider?
yindo commented 2026-02-16 18:09:00 -05:00
Author
Owner
Copy Link
Copy Source

@deleon626 commented on GitHub (Jan 31, 2026):

Hey @OpeOginni, apologies for the confusion - I wasn't using the antigravity provider. I'm experiencing this issue with Kimi K2.5 through the MoonshotAI provider via OpenRouter.

Configuration

"moonshotai/kimi-k2.5": {
  "name": "Kimi K2.5 (Moonshot AI)",
  "limit": { "context": 262144, "output": 262144 },
  "modalities": { "input": ["text", "image"], "output": ["text"] },
  "options": {
    "provider": {
      "order": ["moonshotai", "fireworks"],
      "allow_fallbacks": false
    }
  }
}

Environment

  • OpenCode version: 1.1.40
  • OS: macOS
  • Provider: openrouter → moonshotai/kimi-k2.5

Reproduction Steps

  1. Start a session with moonshotai/kimi-k2.5 model
  2. Reference an image using @path/to/image.png → Works fine
  3. Ask the agent to "read more images in that folder"
  4. Agent autonomously uses the Read tool to access images → Blows out context window

The Issue

Method Result
Manual copy-paste ✅ Works
@path/to/image.png explicit reference ✅ Works
Agent autonomously calls Read tool on image path ❌ Blows out context window

The problem only occurs when the agent autonomously decides to read an image - not when I explicitly reference it with @ or paste directly. This suggests the Read tool may be encoding/sending images differently than the explicit @ reference mechanism.

@deleon626 commented on GitHub (Jan 31, 2026): Hey @OpeOginni, apologies for the confusion - I wasn't using the `antigravity` provider. I'm experiencing this issue with **Kimi K2.5 through the MoonshotAI provider via OpenRouter**. ### Configuration ```json "moonshotai/kimi-k2.5": { "name": "Kimi K2.5 (Moonshot AI)", "limit": { "context": 262144, "output": 262144 }, "modalities": { "input": ["text", "image"], "output": ["text"] }, "options": { "provider": { "order": ["moonshotai", "fireworks"], "allow_fallbacks": false } } } ``` ### Environment - **OpenCode version:** 1.1.40 - **OS:** macOS - **Provider:** openrouter → moonshotai/kimi-k2.5 ### Reproduction Steps 1. Start a session with `moonshotai/kimi-k2.5` model 2. Reference an image using `@path/to/image.png` → **Works fine** 3. Ask the agent to "read more images in that folder" 4. Agent autonomously uses the `Read` tool to access images → **Blows out context window** ### The Issue | Method | Result | |--------|--------| | Manual copy-paste | ✅ Works | | `@path/to/image.png` explicit reference | ✅ Works | | Agent autonomously calls `Read` tool on image path | ❌ Blows out context window | The problem only occurs when the agent **autonomously decides** to read an image - not when I explicitly reference it with `@` or paste directly. This suggests the Read tool may be encoding/sending images differently than the explicit `@` reference mechanism.
yindo commented 2026-02-16 18:09:00 -05:00
Author
Owner
Copy Link
Copy Source

@deleon626 commented on GitHub (Jan 31, 2026):

Potential Root Cause Identified

I conducted an exhaustive investigation using parallel subagents and sequential thinking, and I believe I've identified the root cause of this issue.

Summary

Root Cause: Tool output attachments are converted to a non-standard type: "media" format that bypasses modality validation and causes providers to incorrectly tokenize base64 image data as text, resulting in 500-600x context window explosion.

Bug Location

File: packages/opencode/src/session/message-v2.ts:464

The toModelOutput() function unnecessarily converts type: "file" attachments to type: "media":

// Current (BUGGY) code:
...attachments.map((attachment) => ({
  type: "media",      // ❌ WRONG - creates non-standard type
  mediaType: attachment.mime,
  data: attachment.url.slice(commaIndex + 1),  // Strips prefix
}))

Why This Causes Context Blowout

  1. Modality Bypass: unsupportedParts() in transform.ts:213 only checks type: "file" and type: "image", not type: "media" - so images bypass vision capability validation
  2. Format Mismatch: OpenRouter expects type: "image_url" but receives type: "media"
  3. Provider Fallback: Unknown "media" type triggers JSON.stringify fallback
  4. Text Tokenization: Base64 image data (e.g., 10KB = 13,333 chars) is tokenized as text (~13,333 tokens) instead of as image (~1,500-2,000 tokens)

Why Paste/@ Works

Both use type: "file" which:

  • Passes through modality check in transform.ts:unsupportedParts()
  • Gets converted to proper type: "image_url" format by existing conversion pipeline
  • Provider recognizes format correctly

Proposed Fix

Preserve type: "file" instead of converting to type: "media":

// AFTER (FIXED):
...attachments.map((attachment) => ({
  type: "file",              // ← Preserve standard type
  mime: attachment.mime,       // ← Keep mime
  url: attachment.url,         // ← Keep full data URL with prefix
  filename: attachment.filename || "image"  // ← Add filename
}))

This single change:

  • ✅ Reuses existing modality check
  • ✅ Reuses existing conversion pipeline
  • ✅ Prevents context blowout
  • ✅ Works for all providers

I've written a comprehensive analysis report with more details: Desktop Report

Would you like me to open a PR with this fix?

@deleon626 commented on GitHub (Jan 31, 2026): ## Potential Root Cause Identified I conducted an exhaustive investigation using parallel subagents and sequential thinking, and I believe I've identified the root cause of this issue. ### Summary **Root Cause**: Tool output attachments are converted to a non-standard `type: "media"` format that bypasses modality validation and causes providers to incorrectly tokenize base64 image data as text, resulting in **500-600x context window explosion**. ### Bug Location **File**: `packages/opencode/src/session/message-v2.ts:464` The `toModelOutput()` function unnecessarily converts `type: "file"` attachments to `type: "media"`: ```typescript // Current (BUGGY) code: ...attachments.map((attachment) => ({ type: "media", // ❌ WRONG - creates non-standard type mediaType: attachment.mime, data: attachment.url.slice(commaIndex + 1), // Strips prefix })) ``` ### Why This Causes Context Blowout 1. **Modality Bypass**: `unsupportedParts()` in `transform.ts:213` only checks `type: "file"` and `type: "image"`, not `type: "media"` - so images bypass vision capability validation 2. **Format Mismatch**: OpenRouter expects `type: "image_url"` but receives `type: "media"` 3. **Provider Fallback**: Unknown "media" type triggers JSON.stringify fallback 4. **Text Tokenization**: Base64 image data (e.g., 10KB = 13,333 chars) is tokenized as text (~13,333 tokens) instead of as image (~1,500-2,000 tokens) ### Why Paste/@ Works Both use `type: "file"` which: - Passes through modality check in `transform.ts:unsupportedParts()` - Gets converted to proper `type: "image_url"` format by existing conversion pipeline - Provider recognizes format correctly ### Proposed Fix Preserve `type: "file"` instead of converting to `type: "media"`: ```typescript // AFTER (FIXED): ...attachments.map((attachment) => ({ type: "file", // ← Preserve standard type mime: attachment.mime, // ← Keep mime url: attachment.url, // ← Keep full data URL with prefix filename: attachment.filename || "image" // ← Add filename })) ``` This single change: - ✅ Reuses existing modality check - ✅ Reuses existing conversion pipeline - ✅ Prevents context blowout - ✅ Works for all providers I've written a comprehensive analysis report with more details: [Desktop Report](/Users/dennyleonardo/Desktop/opencode-issue-11306-analysis.md) Would you like me to open a PR with this fix?
yindo commented 2026-02-16 18:09:00 -05:00
Author
Owner
Copy Link
Copy Source

@zuojianghua commented on GitHub (Feb 2, 2026):

same issue at v1.1.48. it read image binary data. then llm context oversized

@zuojianghua commented on GitHub (Feb 2, 2026): same issue at v1.1.48. it read image binary data. then llm context oversized
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#8046
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?