anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[DOCS] Missing documentation for modalities property in custom provider configuration #7114

New Issue
Open
opened 2026-02-16 18:06:13 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 18:06:13 -05:00
Owner
Copy Link
Copy Source

Originally created by @slycrel on GitHub (Jan 21, 2026).

Originally assigned to: @jayair on GitHub.

Pulled the code to have opencode help me get images working with my "OpenAI Compatible" models endpoint I'm forced into using from my IT department. Great and terrible. Got it working, I will submit a doc PR for the missing information outlined here.

Issue Type

Documentation

Description

The modalities property for enabling vision/multimodal support in custom OpenAI-compatible providers is not documented, making it difficult for users to configure image input support for their models.

Problem

When configuring a custom provider using @ai-sdk/openai-compatible, users cannot find documentation on how to enable vision/image input support. The configuration property exists and works, but is not mentioned in:

This leads users to believe image support is not available for custom providers, or to try incorrect properties like:

  • supportsImageInput: true (doesn't work)
  • capabilities: { vision: true } (doesn't work)

Error Message

Without the modalities config, users see:

ERROR: Cannot read "clipboard" (this model does not support image input). Inform the user.

Solution

The correct configuration is the modalities property:

{
  "provider": {
    "my-provider": {
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "https://api.example.com/v1"
      },
      "models": {
        "vision-model": {
          "name": "Vision Model",
          "modalities": {
            "input": ["text", "image"],
            "output": ["text"]
          }
        }
      }
    }
  }
}

The modalities property accepts:

  • input: Array of supported input types: ["text", "audio", "image", "video", "pdf"]
  • output: Array of supported output types: ["text", "audio", "image", "video", "pdf"]

Evidence

This property is:

  1. ✅ Defined in the schema: packages/opencode/src/provider/models.ts:53-58
  2. ✅ Used in capability detection: packages/opencode/src/provider/provider.ts:769-773
  3. ✅ Tested in the test suite: packages/opencode/test/provider/provider.test.ts:1254-1257
  4. ✅ Working in production (verified)

But is:

  • ❌ Not documented in the user-facing documentation
  • ❌ Not mentioned in custom provider examples
  • ❌ Not discoverable through normal config exploration

Related

  • PR #5531 mentions multimodal support for @ai-sdk/openai-compatible
  • I'm preparing a documentation PR to add this information

Impact

Without this documentation, users:

  • Cannot use vision features with custom providers
  • Waste time trying incorrect configuration properties
  • May abandon OpenCode for other tools that have clearer documentation
  • Create duplicate issues asking how to enable vision support
Originally created by @slycrel on GitHub (Jan 21, 2026). Originally assigned to: @jayair on GitHub. Pulled the code to have opencode help me get images working with my "OpenAI Compatible" models endpoint I'm forced into using from my IT department. Great and terrible. Got it working, I will submit a doc PR for the missing information outlined here. ## Issue Type Documentation ## Description The `modalities` property for enabling vision/multimodal support in custom OpenAI-compatible providers is not documented, making it difficult for users to configure image input support for their models. ## Problem When configuring a custom provider using `@ai-sdk/openai-compatible`, users cannot find documentation on how to enable vision/image input support. The configuration property exists and works, but is not mentioned in: - The OpenCode docs at https://opencode.ai/docs/providers - The built-in config documentation - Custom provider examples This leads users to believe image support is not available for custom providers, or to try incorrect properties like: - `supportsImageInput: true` (doesn't work) - `capabilities: { vision: true }` (doesn't work) ## Error Message Without the `modalities` config, users see: ``` ERROR: Cannot read "clipboard" (this model does not support image input). Inform the user. ``` ## Solution The **correct configuration** is the `modalities` property: ```json { "provider": { "my-provider": { "npm": "@ai-sdk/openai-compatible", "options": { "baseURL": "https://api.example.com/v1" }, "models": { "vision-model": { "name": "Vision Model", "modalities": { "input": ["text", "image"], "output": ["text"] } } } } } } ``` The `modalities` property accepts: - `input`: Array of supported input types: `["text", "audio", "image", "video", "pdf"]` - `output`: Array of supported output types: `["text", "audio", "image", "video", "pdf"]` ## Evidence This property is: 1. ✅ Defined in the schema: `packages/opencode/src/provider/models.ts:53-58` 2. ✅ Used in capability detection: `packages/opencode/src/provider/provider.ts:769-773` 3. ✅ Tested in the test suite: `packages/opencode/test/provider/provider.test.ts:1254-1257` 4. ✅ Working in production (verified) But is: - ❌ Not documented in the user-facing documentation - ❌ Not mentioned in custom provider examples - ❌ Not discoverable through normal config exploration ## Related - PR #5531 mentions multimodal support for `@ai-sdk/openai-compatible` - I'm preparing a documentation PR to add this information ## Impact Without this documentation, users: - Cannot use vision features with custom providers - Waste time trying incorrect configuration properties - May abandon OpenCode for other tools that have clearer documentation - Create duplicate issues asking how to enable vision support
yindo added the docs label 2026-02-16 18:06:13 -05:00
yindo commented 2026-02-16 18:06:13 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Jan 21, 2026):

This issue might be a duplicate of existing issues. Please check:

  • #8875: [FEATURE]: Add supportsAttachments/vision capability for custom providers - discusses the need for vision/attachment support in custom providers but as a feature request rather than documentation
  • #5937: Custom provider docs incorrect - mentions that custom provider documentation contains inaccuracies and outdated information

While #8875 is about adding the feature as a request, your issue correctly identifies that the modalities property already exists and works but is simply undocumented. This is complementary to #8875's feature request and highlights the documentation gap that's causing confusion.

Feel free to cross-reference these issues in your documentation PR!

@github-actions[bot] commented on GitHub (Jan 21, 2026): This issue might be a duplicate of existing issues. Please check: - #8875: [FEATURE]: Add supportsAttachments/vision capability for custom providers - discusses the need for vision/attachment support in custom providers but as a feature request rather than documentation - #5937: Custom provider docs incorrect - mentions that custom provider documentation contains inaccuracies and outdated information While #8875 is about adding the feature as a request, your issue correctly identifies that the `modalities` property already exists and works but is simply undocumented. This is complementary to #8875's feature request and highlights the documentation gap that's causing confusion. Feel free to cross-reference these issues in your documentation PR!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label docs
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#7114
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?