anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

Model-generated images silently dropped: processor.ts doesn't handle Vercel AI SDK file stream event #8920

New Issue
Open
opened 2026-02-16 18:11:10 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 18:11:10 -05:00
Owner
Copy Link
Copy Source

Originally created by @yourbuddyconner on GitHub (Feb 9, 2026).

Originally assigned to: @rekram1-node on GitHub.

Summary

When models that support inline image generation (GPT-4o, Gemini 2.5 Flash Image, etc.) return images in their responses, the image data is silently dropped because the stream processor does not handle the Vercel AI SDK's "file" event type.

Root Cause

The stream processing loop in packages/opencode/src/session/processor.ts (lines 49-277) iterates over stream.fullStream events and handles:

start, reasoning-start, reasoning-delta, reasoning-end, tool-input-start, tool-input-delta, tool-input-end, tool-call, tool-result, tool-error, error, start-step, finish-step, text-start, text-delta, text-end, finish

The Vercel AI SDK's streamText().fullStream also emits a "file" event when models generate images inline. This event contains a GeneratedFile with base64, uint8Array, and mediaType fields. This event type is not handled — it falls through to the default case at line ~272 where it's logged as "unhandled" and discarded via continue.

Impact

  • Models that support native image generation (GPT-4o, Gemini image models, etc.) claim to have generated an image, but the user never sees it
  • The image data is irretrievably lost — it's not stored, not displayed, and not accessible
  • Users have no indication that content was dropped

Existing Type Support

The type system already supports this. packages/opencode/src/session/message.ts (lines 74-82) defines a FilePart type:

export const FilePart = z.object({
  type: z.literal("file"),
  mediaType: z.string(),
  filename: z.string().optional(),
  url: z.string(),
})

And FilePart is already part of the MessagePart discriminated union (line ~84-90). The schema is ready — the processor just needs to create FilePart instances from the "file" stream event.

Suggested Fix

In the for await loop in processor.ts, add a case for the "file" event type that:

  1. Creates a FilePart from the GeneratedFile (converting base64 + mediaType to a data URI for the url field)
  2. Calls Session.appendPart() to attach it to the current assistant message
  3. Publishes the part via the event bus so SSE clients (serve mode) and the TUI can display it

Related Issues

These issues describe the user-facing symptoms but none have identified this root cause:

  • #7646 — "Is there a way to create images?" — User tried Gemini image model, it claimed success but image was invisible
  • #9539 — "feat: support images in custom tool responses" — Related gap for custom tools (tool-side, not model-output-side)
  • #6604 — "MCP Tool Output not displayed in TUI" — Related display gap for tool output including images
Originally created by @yourbuddyconner on GitHub (Feb 9, 2026). Originally assigned to: @rekram1-node on GitHub. ## Summary When models that support inline image generation (GPT-4o, Gemini 2.5 Flash Image, etc.) return images in their responses, the image data is **silently dropped** because the stream processor does not handle the Vercel AI SDK's `"file"` event type. ## Root Cause The stream processing loop in [`packages/opencode/src/session/processor.ts` (lines 49-277)](https://github.com/anomalyco/opencode/blob/dev/packages/opencode/src/session/processor.ts#L49-L277) iterates over `stream.fullStream` events and handles: `start`, `reasoning-start`, `reasoning-delta`, `reasoning-end`, `tool-input-start`, `tool-input-delta`, `tool-input-end`, `tool-call`, `tool-result`, `tool-error`, `error`, `start-step`, `finish-step`, `text-start`, `text-delta`, `text-end`, `finish` The Vercel AI SDK's `streamText().fullStream` also emits a [`"file"` event](https://ai-sdk.dev/docs/reference/ai-sdk-core/stream-text) when models generate images inline. This event contains a `GeneratedFile` with `base64`, `uint8Array`, and `mediaType` fields. **This event type is not handled** — it falls through to the default case at line ~272 where it's logged as `"unhandled"` and discarded via `continue`. ## Impact - Models that support native image generation (GPT-4o, Gemini image models, etc.) claim to have generated an image, but the user never sees it - The image data is irretrievably lost — it's not stored, not displayed, and not accessible - Users have no indication that content was dropped ## Existing Type Support The type system already supports this. [`packages/opencode/src/session/message.ts` (lines 74-82)](https://github.com/anomalyco/opencode/blob/dev/packages/opencode/src/session/message.ts#L74-L82) defines a `FilePart` type: ```typescript export const FilePart = z.object({ type: z.literal("file"), mediaType: z.string(), filename: z.string().optional(), url: z.string(), }) ``` And `FilePart` is already part of the `MessagePart` discriminated union (line ~84-90). The schema is ready — the processor just needs to create `FilePart` instances from the `"file"` stream event. ## Suggested Fix In the `for await` loop in `processor.ts`, add a case for the `"file"` event type that: 1. Creates a `FilePart` from the `GeneratedFile` (converting `base64` + `mediaType` to a data URI for the `url` field) 2. Calls `Session.appendPart()` to attach it to the current assistant message 3. Publishes the part via the event bus so SSE clients (serve mode) and the TUI can display it ## Related Issues These issues describe the user-facing symptoms but none have identified this root cause: - #7646 — "Is there a way to create images?" — User tried Gemini image model, it claimed success but image was invisible - #9539 — "feat: support images in custom tool responses" — Related gap for custom tools (tool-side, not model-output-side) - #6604 — "MCP Tool Output not displayed in TUI" — Related display gap for tool output including images
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#8920
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?