anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[FEATURE]: Native Multimodal Context Support (Video/Audio) #7530

New Issue
Open
opened 2026-02-16 18:07:28 -05:00 by yindo · 5 comments
yindo commented 2026-02-16 18:07:28 -05:00
Owner
Copy Link
Copy Source

Originally created by @AimAmit on GitHub (Jan 25, 2026).

Originally assigned to: @thdxr on GitHub.

Feature hasn't been suggested before.

  • I have verified this feature I'm about to request hasn't been suggested before.

Describe the enhancement you want to request

Hi Team,

I recently moved to opencode from claude code and I'm liking it way of handling multiple models and working with it.

I am looking to bring the True Multimodal capabilities found in environments like Gemini AI Studio into the OpenCode workflow. Currently, handling video based tasks requires manual transcription or frame extraction, which adds significant friction. Also it can't understand audio in it.

Use Case example: A developer records a 30 second bug reproduction. They attach the video to OpenCode and ask: "Analyze this UI glitch and find the relevant CSS class in my project that might be causing the flickering."

Originally created by @AimAmit on GitHub (Jan 25, 2026). Originally assigned to: @thdxr on GitHub. ### Feature hasn't been suggested before. - [x] I have verified this feature I'm about to request hasn't been suggested before. ### Describe the enhancement you want to request Hi Team, I recently moved to opencode from claude code and I'm liking it way of handling multiple models and working with it. I am looking to bring the **True Multimodal** capabilities found in environments like Gemini AI Studio into the OpenCode workflow. Currently, handling video based tasks requires manual transcription or frame extraction, which adds significant friction. Also it can't understand audio in it. Use Case example: A developer records a 30 second bug reproduction. They attach the video to OpenCode and ask: "Analyze this UI glitch and find the relevant CSS class in my project that might be causing the flickering."
yindo added the discussion label 2026-02-16 18:07:28 -05:00
yindo commented 2026-02-16 18:07:28 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Jan 25, 2026):

This issue might be a duplicate of existing issues. Please check:

  • #9897: [DOCS] Missing documentation for modalities property in custom provider configuration - the infrastructure for multimodal support already exists but lacks documentation
  • #4695: [FEATURE]: Speech-to-Text Voice Input for Lazy People in OpenCode - addresses audio input component
  • #9356: My opencode app cannot read files - PDF file reading issues
  • #10294: Read tool throws error when reading pdf files - PDF support incomplete
  • #9825: [BUG] ai is saying that it can't read a pdf - model supports it but OpenCode doesn't leverage it
  • #6665: Working with REST & media data? - documents the friction with handling non-text content
  • #3178: Add n1n.ai API as a model provider - requests providers with native multimodal capabilities

Feel free to ignore if none of these address your specific case.

@github-actions[bot] commented on GitHub (Jan 25, 2026): This issue might be a duplicate of existing issues. Please check: - #9897: [DOCS] Missing documentation for `modalities` property in custom provider configuration - the infrastructure for multimodal support already exists but lacks documentation - #4695: [FEATURE]: Speech-to-Text Voice Input for Lazy People in OpenCode - addresses audio input component - #9356: My opencode app cannot read files - PDF file reading issues - #10294: Read tool throws error when reading pdf files - PDF support incomplete - #9825: [BUG] ai is saying that it can't read a pdf - model supports it but OpenCode doesn't leverage it - #6665: Working with REST & media data? - documents the friction with handling non-text content - #3178: Add n1n.ai API as a model provider - requests providers with native multimodal capabilities Feel free to ignore if none of these address your specific case.
yindo commented 2026-02-16 18:07:28 -05:00
Author
Owner
Copy Link
Copy Source

@NamedIdentity commented on GitHub (Jan 30, 2026):

Also on my wish list of features. I'd like to read PDFs and docx that contain images without having to call and wait on a parsing API.

Also be nice to load in mp3 and mp4 files for audio transcription.

Workarounds exist to call external APIs and tools, but the innate capabilities of models cannot be fully used in OpenCode.

Gemini can transcribe audio, but not in OpenCode.

@NamedIdentity commented on GitHub (Jan 30, 2026): Also on my wish list of features. I'd like to read PDFs and docx that contain images without having to call and wait on a parsing API. Also be nice to load in mp3 and mp4 files for audio transcription. Workarounds exist to call external APIs and tools, but the innate capabilities of models cannot be fully used in OpenCode. Gemini can transcribe audio, but not in OpenCode.
yindo commented 2026-02-16 18:07:28 -05:00
Author
Owner
Copy Link
Copy Source

@LCorleone commented on GitHub (Feb 6, 2026):

+1
Tool calling and image understanding are both intrinsic capabilities of Kimi 2.5. However, even after integrating Kimi 2.5, OpenCode still cannot proactively invoke its multimodal features.

@LCorleone commented on GitHub (Feb 6, 2026): +1 Tool calling and image understanding are both intrinsic capabilities of Kimi 2.5. However, even after integrating Kimi 2.5, OpenCode still cannot proactively invoke its multimodal features.
yindo commented 2026-02-16 18:07:28 -05:00
Author
Owner
Copy Link
Copy Source

@HadiCherkaoui commented on GitHub (Feb 9, 2026):

+1 from me
Being able to upload videos would be a game changer for models that support it like kimi k2.5 which is really good as visual coding
Tho idk if the openai or anthropic compatiable API for kimi for coding supports uploading videos...

@HadiCherkaoui commented on GitHub (Feb 9, 2026): +1 from me Being able to upload videos would be a game changer for models that support it like kimi k2.5 which is really good as visual coding Tho idk if the openai or anthropic compatiable API for kimi for coding supports uploading videos...
yindo commented 2026-02-16 18:07:28 -05:00
Author
Owner
Copy Link
Copy Source

@HadiCherkaoui commented on GitHub (Feb 9, 2026):

if its okey i would like to tackle this feature myself since based on the api docs of kimi they extended the openai schema to accept video input similar to how gemini does. I would like to experiment on this and make a patch as long as all agree on this. i would probably first either start with kimi or gemini tho gemini should be pretty easy

Update:
from the looks of it it seems pretty easy since kimi cli already implements native video so yeah ill just implement that

@HadiCherkaoui commented on GitHub (Feb 9, 2026): if its okey i would like to tackle this feature myself since based on the api docs of kimi they extended the openai schema to accept video input similar to how gemini does. I would like to experiment on this and make a patch as long as all agree on this. i would probably first either start with kimi or gemini tho gemini should be pretty easy Update: from the looks of it it seems pretty easy since kimi cli already implements native video so yeah ill just implement that
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label discussion
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#7530
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?