anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[FEATURE]: Native Model Fallback / Failover Support #4608

New Issue
Open
opened 2026-02-16 17:44:46 -05:00 by yindo · 11 comments
yindo commented 2026-02-16 17:44:46 -05:00
Owner
Copy Link
Copy Source

Originally created by @arisgrout on GitHub (Jan 10, 2026).

Originally assigned to: @thdxr on GitHub.

Problem

OpenCode supports:

  • Provider fallback only when the model ID is the same
  • Static agent-level model overrides

There is no way to define fallback between different models, e.g.:

“If model A errors or rate-limits → automatically retry with model B”

This causes long-running agent workflows to fail on transient provider/model issues and forces users to rely on external routers or proxies (like litellm).

Request

Add first-class model fallback support to OpenCode.

Example (global)

{
  "models": {
    "coder": {
      "fallback": [
        "anthropic/claude-3.5-sonnet",
        "openai/gpt-4o",
        "deepseek/deepseek-r1"
      ]
    }
  }
}

Example (agent-level)

{
  "agents": {
    "build": {
      "model": {
        "fallback": ["claude-sonnet", "gpt-4o-mini"]
      }
    }
  }
}

Behavior

  • Switch models on: rate limits, provider/model unavailability, 5xx errors
  • Do not retry on prompt or validation errors
  • Optional retry limit

Why in Core

  • Model routing is an orchestration concern
  • External routers break plugin auth flows and agent semantics
  • Complements existing provider-order routing (e.g. #2153)

Status

  • No existing plugin or PR implements true model fallback

Thanks,

Originally created by @arisgrout on GitHub (Jan 10, 2026). Originally assigned to: @thdxr on GitHub. ## Problem OpenCode supports: - Provider fallback **only** when the model ID is the same - Static agent-level model overrides There is **no way to define fallback between different models**, e.g.: > “If model A errors or rate-limits → automatically retry with model B” This causes long-running agent workflows to fail on transient provider/model issues and forces users to rely on external routers or proxies (like litellm). --- ## Request Add **first-class model fallback support** to OpenCode. ### Example (global) ```jsonc { "models": { "coder": { "fallback": [ "anthropic/claude-3.5-sonnet", "openai/gpt-4o", "deepseek/deepseek-r1" ] } } } ``` ### Example (agent-level) ```jsonc { "agents": { "build": { "model": { "fallback": ["claude-sonnet", "gpt-4o-mini"] } } } } ``` --- ## Behavior - Switch models on: rate limits, provider/model unavailability, 5xx errors - Do not retry on prompt or validation errors - Optional retry limit --- ## Why in Core - Model routing is an orchestration concern - External routers break plugin auth flows and agent semantics - Complements existing provider-order routing (e.g. #2153) --- ## Status - No existing plugin or PR implements true model fallback Thanks,
yindo added the discussion label 2026-02-16 17:44:46 -05:00
yindo commented 2026-02-16 17:44:46 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Jan 10, 2026):

This issue might be a duplicate of existing issues. Please check:

  • #310: Retry w/ different model
  • #3011: [Feature Request] Configurable Retry Mechanism via opencode.json
  • #1712: Implement exponential back-off when hitting rate limits

Feel free to ignore if none of these address your specific case.

@github-actions[bot] commented on GitHub (Jan 10, 2026): This issue might be a duplicate of existing issues. Please check: - #310: Retry w/ different model - #3011: [Feature Request] Configurable Retry Mechanism via opencode.json - #1712: Implement exponential back-off when hitting rate limits Feel free to ignore if none of these address your specific case.
yindo commented 2026-02-16 17:44:47 -05:00
Author
Owner
Copy Link
Copy Source

@smashah commented on GitHub (Jan 14, 2026):

Now that antigravity limits have been adjusted, and claude is limiting usage, this is needed now more than ever

@smashah commented on GitHub (Jan 14, 2026): Now that antigravity limits have been adjusted, and claude is limiting usage, this is needed now more than ever
yindo commented 2026-02-16 17:44:47 -05:00
Author
Owner
Copy Link
Copy Source

@imqqmi commented on GitHub (Jan 15, 2026):

I've created a duplicate feature, here's my proposal:
Problem:
some models/providers may have issues, ie service is temporarily offline or you've hit rate limits.

Solution:
Instead of having one model, use an array so that multiple models can be given as a fallback, when one model fails, the second or third can be attempted (and remember using the second for 15 minutes, then retry primary model).
This avoids manual switching between models, and subagents that don't offer the option to switch models/providers can try other models/providers.
A message can be flashed to the user that opencode is switching to a different model so the user knows what's going on.

This is not the same as the feature request for dynamic model switching where the agent takes initiative, this is a configuration setting where fallback models/providers can be given and opencode decides when to use them (running into issues > fallback).

Differences:

  • I think it's simpler to just add an array of models in the same "model": ["primary model","secondary model"] etc property instead of a fallback property.
  • Add a time out of 15 minutes to retry primary model(s) once the issues are fixed/rate limits are cleared.
@imqqmi commented on GitHub (Jan 15, 2026): I've created a duplicate feature, here's my proposal: Problem: some models/providers may have issues, ie service is temporarily offline or you've hit rate limits. Solution: Instead of having one model, use an array so that multiple models can be given as a fallback, when one model fails, the second or third can be attempted (and remember using the second for 15 minutes, then retry primary model). This avoids manual switching between models, and subagents that don't offer the option to switch models/providers can try other models/providers. A message can be flashed to the user that opencode is switching to a different model so the user knows what's going on. This is not the same as the feature request for dynamic model switching where the agent takes initiative, this is a configuration setting where fallback models/providers can be given and opencode decides when to use them (running into issues > fallback). --- Differences: - I think it's simpler to just add an array of models in the same "model": ["primary model","secondary model"] etc property instead of a fallback property. - Add a time out of 15 minutes to retry primary model(s) once the issues are fixed/rate limits are cleared.
yindo commented 2026-02-16 17:44:47 -05:00
Author
Owner
Copy Link
Copy Source

@stickerdaniel commented on GitHub (Jan 15, 2026):

Describe the enhancement you want to request

it would be great if users could specify multiple models in the config (opencode.jsonc / agent/*md) so that an array like

    "$schema": "https://opencode.ai/config.json",
    "default_agent": "plan",
    "model": [
        "anthropic/claude-opus-4-5",
        "google/antigravity-claude-opus-4-5",
        "github-copilot/claude-opus-4-5"
    ],

would be a valid config option.

Why?

  • Users that dont have a specific provider connected have a fallback model
  • Ideally, opencode detects if provider limits are reached and selects the other model as default when starting a new session
  • This way you can continue using e.g. Opus 4.5 without switching the model manually through other providers even if Anthropic's ridiculous rate limits are reached.
@stickerdaniel commented on GitHub (Jan 15, 2026): ### Describe the enhancement you want to request it would be great if users could specify multiple models in the config (opencode.jsonc / agent/*md) so that an array like ```{ "$schema": "https://opencode.ai/config.json", "default_agent": "plan", "model": [ "anthropic/claude-opus-4-5", "google/antigravity-claude-opus-4-5", "github-copilot/claude-opus-4-5" ], ``` would be a valid config option. ### Why? - Users that dont have a specific provider connected have a fallback model - Ideally, opencode detects if provider limits are reached and selects the other model as default when starting a new session - This way you can continue using e.g. Opus 4.5 without switching the model manually through other providers even if Anthropic's ridiculous rate limits are reached.
yindo commented 2026-02-16 17:44:47 -05:00
Author
Owner
Copy Link
Copy Source

@broskees commented on GitHub (Jan 25, 2026):

This is big

@broskees commented on GitHub (Jan 25, 2026): This is big
yindo commented 2026-02-16 17:44:47 -05:00
Author
Owner
Copy Link
Copy Source

@Simplereally commented on GitHub (Jan 30, 2026):

Bumping this. Big want. Don’t want to have to use oh my opencode to orchestrate this, would prefer clean native use.

@Simplereally commented on GitHub (Jan 30, 2026): Bumping this. Big want. Don’t want to have to use oh my opencode to orchestrate this, would prefer clean native use.
yindo commented 2026-02-16 17:44:47 -05:00
Author
Owner
Copy Link
Copy Source

@manascb1344 commented on GitHub (Jan 30, 2026):

Have a look at : #8669 and https://github.com/anomalyco/opencode/issues/1267

@manascb1344 commented on GitHub (Jan 30, 2026): Have a look at : #8669 and https://github.com/anomalyco/opencode/issues/1267
yindo commented 2026-02-16 17:44:47 -05:00
Author
Owner
Copy Link
Copy Source

@nwpr commented on GitHub (Feb 7, 2026):

Just commented this on the PR, but want to mention it in this issue too:

I’d like to suggest a complementary abstraction that could make model selection more flexible and predictable long-term: introducing “virtual models” as a first-class concept in the configuration.

Instead of only defining fallback behavior on concrete models or providers, a virtual model would act as an alias to a set of real provider-specific models, together with a selection strategy.

Key idea

  • Concrete models (e.g. anthropic/claude-opus-4.5) are always deterministic: selecting them always uses that exact model.
  • Virtual models encapsulate dynamic behavior and are explicitly opt-in.
  • The config defines:
    • the underlying models
    • how a model is chosen (round-robin, priority, sequential after rate limit, weighted, etc.)

Why this helps

  1. Clear semantics: Explicit model selection never changes behavior. Dynamic selection is only used when a virtual model is chosen.
  2. Operational control: Virtual models provide a single choke point for retries and switching policies, making it easier to reason about side effects, retries, and observability compared to implicit fallback on concrete models.
  3. Extensibility: New strategies can be added without complicating fallback rules.
  4. Consistency: Virtual models can be referenced anywhere a normal model can be used (global default, agent, session).

Example configuration

{
  "models": {
    "virtual/coding": {
      "type": "virtual",
      "strategy": "sequential",
      "models": [
        "anthropic/claude-opus-4.5",
        "openai/gpt-4.1",
        "openai/gpt-4o"
      ]
    },
    "virtual/fast": {
      "type": "virtual",
      "strategy": "round_robin",
      "models": [
        "openai/gpt-4o-mini",
        "openai/gpt-4.1-mini"
      ]
    }
  }
}

Semantics

  • Selecting anthropic/claude-opus-4.5 always uses that exact model.
  • Selecting virtual/coding delegates model choice to the virtual model’s strategy.
  • Possible strategies could include:
    • priority
    • round_robin
    • random
    • sequential (on rate limit, failure, etc...)
    • weighted

This aligns well with ongoing discussions around native fallback and multi-provider usage, and could build cleanly on top of the current implementation without breaking existing configs.

@nwpr commented on GitHub (Feb 7, 2026): Just commented this on the PR, but want to mention it in this issue too: I’d like to suggest a complementary abstraction that could make model selection more flexible and predictable long-term: introducing “virtual models” as a first-class concept in the configuration. Instead of only defining fallback behavior on concrete models or providers, a virtual model would act as an alias to a set of real provider-specific models, together with a **selection strategy**. ### Key idea - **Concrete models** (e.g. `anthropic/claude-opus-4.5`) are always deterministic: selecting them always uses that exact model. - **Virtual models** encapsulate dynamic behavior and are explicitly opt-in. - The config defines: - the underlying models - how a model is chosen (round-robin, priority, sequential after rate limit, weighted, etc.) ### Why this helps 1. **Clear semantics:** Explicit model selection never changes behavior. Dynamic selection is only used when a virtual model is chosen. 2. **Operational control:** Virtual models provide a single choke point for retries and switching policies, making it easier to reason about side effects, retries, and observability compared to implicit fallback on concrete models. 3. **Extensibility:** New strategies can be added without complicating fallback rules. 4. **Consistency:** Virtual models can be referenced anywhere a normal model can be used (global default, agent, session). ### Example configuration ```json { "models": { "virtual/coding": { "type": "virtual", "strategy": "sequential", "models": [ "anthropic/claude-opus-4.5", "openai/gpt-4.1", "openai/gpt-4o" ] }, "virtual/fast": { "type": "virtual", "strategy": "round_robin", "models": [ "openai/gpt-4o-mini", "openai/gpt-4.1-mini" ] } } } ``` ### Semantics - Selecting `anthropic/claude-opus-4.5` always uses that exact model. - Selecting `virtual/coding` delegates model choice to the virtual model’s strategy. - Possible strategies could include: - `priority` - `round_robin` - `random` - `sequential` (on rate limit, failure, etc...) - `weighted` This aligns well with ongoing discussions around native fallback and multi-provider usage, and could build cleanly on top of the current implementation without breaking existing configs.
yindo commented 2026-02-16 17:44:47 -05:00
Author
Owner
Copy Link
Copy Source

@gimlichael commented on GitHub (Feb 9, 2026):

Consider adapting the model from OpenClaw [KISS]: https://docs.openclaw.ai/concepts/model-failover

@gimlichael commented on GitHub (Feb 9, 2026): Consider adapting the model from OpenClaw [KISS]: https://docs.openclaw.ai/concepts/model-failover
yindo commented 2026-02-16 17:44:47 -05:00
Author
Owner
Copy Link
Copy Source

@rober-tm commented on GitHub (Feb 9, 2026):

Lo de modelos virtuales suena genial, hay planes de incluir esto? O sacar un plugin?

@rober-tm commented on GitHub (Feb 9, 2026): Lo de modelos virtuales suena genial, hay planes de incluir esto? O sacar un plugin?
yindo commented 2026-02-16 17:44:47 -05:00
Author
Owner
Copy Link
Copy Source

@xitex commented on GitHub (Feb 10, 2026):

This proposal refines the "Virtual Model" concept into a more pragmatic Strategy Profiles approach. It separates Operational Logic (retries, backoffs) from Model Selection (which models to use), allowing for clean configuration in both global settings and individual Agent Markdown files.

Proposal: Unified Model Fallback & Strategy Profiles

Problem Statement

Current implementations of model selection are too rigid. Users need a way to:

  1. Fallback to a second model when the first hits a rate limit (429) or is down (5xx).
  2. Rotate between providers of the same model (e.g., Claude 3.5 on Anthropic vs. AWS Bedrock vs. GCP).
  3. Define retry policies (exponential backoff) once, and apply them to various agents.

1. Global Strategy Profiles (opencode.jsonc)

Instead of hardcoding retry logic everywhere, we define Strategy Profiles. These profiles handle the behavior of the rotation.

{
  "strategies": {
    "robust_coding": {
      "max_retries": 3,
      "timeout": "30s",
      "backoff": {
        "type": "exponential",
        "initial": "500ms",
        "multiplier": 2
      },
      "fallback_on": [429, 500, 503],
      "on_fail": "throw", // or "continue_with_next"
      "cooldown": "15m" // If a model fails, don't try it again for 15 minutes
    },
    "fast_retry": {
      "max_retries": 1,
      "timeout": "10s",
      "fallback_on": ["any_error"]
    }
  }
}

2. Agent-Level Configuration (Markdown Frontmatter)

Agents can now accept an array of models and a reference to a strategy. This keeps the agent file clean and focused on the task.

~/.config/opencode/agents/reviewer.md

---
description: Code quality reviewer
# Simple Array: Uses default priority strategy
model: 
  - anthropic/claude-3-5-sonnet
  - openai/gpt-4o
  - deepseek/deepseek-chat

# Reference to the global strategy profile
strategy: robust_coding 
temperature: 0.1
---

You are a senior reviewer...

3. Support for Multiple Providers (Same Model)

Addressing the feedback from the thread (e.g., using Opus from different sources), we can treat different providers as a prioritized list:

---
model:
  - google/antigravity-claude-opus-4-5
  - anthropic/claude-opus-4-5
  - github-copilot/claude-opus-4-5
strategy: robust_coding
---

4. Execution Logic (Internal Behavior)

When an agent is invoked:

  1. Selection: The orchestrator picks the first model in the list.
  2. Execution: It applies the settings from the linked strategy.
  3. Fallback: If a fallback_on error occurs:
    • The current model is marked with a cooldown timestamp.
    • The next model in the array is selected.
    • The request is retried based on the backoff settings.
  4. Observability: OpenCode flashes a UI notification: "Anthropic Rate Limit reached. Falling back to GPT-4o..."

Key Advantages:

  • Clean Markdown: No need to clutter agent files with complex JSON/Logic.
  • Global Control: Update your "Standard Retry" policy in one place, and it affects all agents.
  • Provider Agnostic: Easily mix and match local models (Ollama) with cloud providers as fallbacks.
  • Backward Compatible: A single string in the model field works exactly as it did before.

Comparison with existing proposals:

Feature Current PRs This Proposal
Model Arrays Supported in some Supported (Simplified UX)
Retry Logic Hardcoded Strategy Profiles (Reusable)
Exponential Backoff Missing Built-in
Circuit Breaking No Yes (via cooldown field)
Agent MD support Minimal First-class citizen

This approach provides the "clean native use" users are asking for while offering the power of an orchestrator like LiteLLM without the extra infrastructure.

@xitex commented on GitHub (Feb 10, 2026): This proposal refines the "Virtual Model" concept into a more pragmatic **Strategy Profiles** approach. It separates **Operational Logic** (retries, backoffs) from **Model Selection** (which models to use), allowing for clean configuration in both global settings and individual Agent Markdown files. --- ### Proposal: Unified Model Fallback & Strategy Profiles #### Problem Statement Current implementations of model selection are too rigid. Users need a way to: 1. **Fallback** to a second model when the first hits a rate limit (429) or is down (5xx). 2. **Rotate** between providers of the same model (e.g., Claude 3.5 on Anthropic vs. AWS Bedrock vs. GCP). 3. **Define retry policies** (exponential backoff) once, and apply them to various agents. #### 1. Global Strategy Profiles (`opencode.jsonc`) Instead of hardcoding retry logic everywhere, we define **Strategy Profiles**. These profiles handle the *behavior* of the rotation. ```jsonc { "strategies": { "robust_coding": { "max_retries": 3, "timeout": "30s", "backoff": { "type": "exponential", "initial": "500ms", "multiplier": 2 }, "fallback_on": [429, 500, 503], "on_fail": "throw", // or "continue_with_next" "cooldown": "15m" // If a model fails, don't try it again for 15 minutes }, "fast_retry": { "max_retries": 1, "timeout": "10s", "fallback_on": ["any_error"] } } } ``` #### 2. Agent-Level Configuration (Markdown Frontmatter) Agents can now accept an array of models and a reference to a strategy. This keeps the agent file clean and focused on the task. **`~/.config/opencode/agents/reviewer.md`** ```yaml --- description: Code quality reviewer # Simple Array: Uses default priority strategy model: - anthropic/claude-3-5-sonnet - openai/gpt-4o - deepseek/deepseek-chat # Reference to the global strategy profile strategy: robust_coding temperature: 0.1 --- You are a senior reviewer... ``` #### 3. Support for Multiple Providers (Same Model) Addressing the feedback from the thread (e.g., using Opus from different sources), we can treat different providers as a prioritized list: ```yaml --- model: - google/antigravity-claude-opus-4-5 - anthropic/claude-opus-4-5 - github-copilot/claude-opus-4-5 strategy: robust_coding --- ``` #### 4. Execution Logic (Internal Behavior) When an agent is invoked: 1. **Selection:** The orchestrator picks the first model in the list. 2. **Execution:** It applies the settings from the linked `strategy`. 3. **Fallback:** If a `fallback_on` error occurs: - The current model is marked with a `cooldown` timestamp. - The next model in the array is selected. - The request is retried based on the `backoff` settings. 4. **Observability:** OpenCode flashes a UI notification: *"Anthropic Rate Limit reached. Falling back to GPT-4o..."* #### Key Advantages: - **Clean Markdown:** No need to clutter agent files with complex JSON/Logic. - **Global Control:** Update your "Standard Retry" policy in one place, and it affects all agents. - **Provider Agnostic:** Easily mix and match local models (Ollama) with cloud providers as fallbacks. - **Backward Compatible:** A single string in the `model` field works exactly as it did before. --- ### Comparison with existing proposals: | Feature | Current PRs | **This Proposal** | | :--- | :--- | :--- | | **Model Arrays** | Supported in some | **Supported** (Simplified UX) | | **Retry Logic** | Hardcoded | **Strategy Profiles** (Reusable) | | **Exponential Backoff**| Missing | **Built-in** | | **Circuit Breaking** | No | **Yes** (via `cooldown` field) | | **Agent MD support** | Minimal | **First-class citizen** | This approach provides the "clean native use" users are asking for while offering the power of an orchestrator like LiteLLM without the extra infrastructure.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label discussion
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#4608
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?