anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-23 18:55:37 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[PR #11739] feat: add runtime model fallback on retry exhaustion #13920

New Issue
Open
opened 2026-02-16 18:18:44 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 18:18:44 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/anomalyco/opencode/pull/11739

State: open
Merged: No

Summary

  • Adds fallback_models and max_retries_before_fallback config fields to agent configuration
  • After N consecutive retryable failures on the current model, the session processor automatically switches to the next model in the configured fallback chain
  • New "fallback" session status type surfaces the model transition in the UI

Motivation

When an LLM model is overloaded or rate-limited, OpenCode retries the same model indefinitely with exponential backoff (capped at 30s). Users with multiple provider accounts (e.g. Anthropic, OpenAI, GitHub Copilot) have no way to automatically fail over — they're stuck waiting on a down model.

Config example

{
  "agent": {
    "build": {
      "model": "moonshotai/kimi-k2",
      "fallback_models": [
        "anthropic/claude-sonnet-4-5",
        "openai/gpt-4o"
      ],
      "max_retries_before_fallback": 3
    }
  }
}

Changes

File What
config/config.ts Accept fallback_models (string array) and max_retries_before_fallback (positive int) in Config.Agent schema; added to knownKeys so they don't leak to options via .catchall()
agent/agent.ts Parse "provider/model" strings into { providerID, modelID } objects on Agent.Info; propagate from config
session/status.ts New "fallback" discriminated union member with from/to string fields
session/processor.ts Fallback logic in the retry catch block (see below)

Processor behavior

Stream request to current model
  ├─ Success → return normally
  └─ Retryable error →
       attemptsOnCurrentModel++
       ├─ < max_retries → exponential backoff, retry same model
       └─ ≥ max_retries AND fallbacks remain →
            Loop through remaining fallbacks:
              ├─ Provider.getModel() succeeds →
              │    Swap model on streamInput + input + assistantMessage
              │    Reset per-model backoff counter
              │    Set session status to "fallback"
              │    Continue retry loop with fresh model
              └─ Provider.getModel() fails → try next fallback
            All unresolvable → fall through to normal backoff

Design decisions

  • Per-model backoff: delay uses attemptsOnCurrentModel, not total attempt, so a fresh fallback model starts at 2s instead of inheriting the escalated delay
  • Assistant message metadata updated: modelID/providerID on the persisted message reflect the actual model that generated the response
  • Greedy fallback resolution: if one fallback can't be resolved (provider not configured), immediately tries the next — no wasted retry cycle on an exhausted model
  • No state leakage: all counters live in the create() closure, reset naturally when process() returns and a new processor is created for the next turn

Backward compatibility

Fully backward-compatible. Both new config fields are optional with sensible defaults. Agents without fallback_models configured behave identically to before.

**Original Pull Request:** https://github.com/anomalyco/opencode/pull/11739 **State:** open **Merged:** No --- ## Summary - Adds `fallback_models` and `max_retries_before_fallback` config fields to agent configuration - After N consecutive retryable failures on the current model, the session processor automatically switches to the next model in the configured fallback chain - New `"fallback"` session status type surfaces the model transition in the UI ## Motivation When an LLM model is overloaded or rate-limited, OpenCode retries the same model indefinitely with exponential backoff (capped at 30s). Users with multiple provider accounts (e.g. Anthropic, OpenAI, GitHub Copilot) have no way to automatically fail over — they're stuck waiting on a down model. ## Config example ```json { "agent": { "build": { "model": "moonshotai/kimi-k2", "fallback_models": [ "anthropic/claude-sonnet-4-5", "openai/gpt-4o" ], "max_retries_before_fallback": 3 } } } ``` ## Changes | File | What | |------|------| | `config/config.ts` | Accept `fallback_models` (string array) and `max_retries_before_fallback` (positive int) in `Config.Agent` schema; added to `knownKeys` so they don't leak to `options` via `.catchall()` | | `agent/agent.ts` | Parse `"provider/model"` strings into `{ providerID, modelID }` objects on `Agent.Info`; propagate from config | | `session/status.ts` | New `"fallback"` discriminated union member with `from`/`to` string fields | | `session/processor.ts` | Fallback logic in the retry catch block (see below) | ## Processor behavior ``` Stream request to current model ├─ Success → return normally └─ Retryable error → attemptsOnCurrentModel++ ├─ < max_retries → exponential backoff, retry same model └─ ≥ max_retries AND fallbacks remain → Loop through remaining fallbacks: ├─ Provider.getModel() succeeds → │ Swap model on streamInput + input + assistantMessage │ Reset per-model backoff counter │ Set session status to "fallback" │ Continue retry loop with fresh model └─ Provider.getModel() fails → try next fallback All unresolvable → fall through to normal backoff ``` ### Design decisions - **Per-model backoff**: delay uses `attemptsOnCurrentModel`, not total `attempt`, so a fresh fallback model starts at 2s instead of inheriting the escalated delay - **Assistant message metadata updated**: `modelID`/`providerID` on the persisted message reflect the actual model that generated the response - **Greedy fallback resolution**: if one fallback can't be resolved (provider not configured), immediately tries the next — no wasted retry cycle on an exhausted model - **No state leakage**: all counters live in the `create()` closure, reset naturally when `process()` returns and a new processor is created for the next turn ## Backward compatibility Fully backward-compatible. Both new config fields are optional with sensible defaults. Agents without `fallback_models` configured behave identically to before.
yindo added the pull-request label 2026-02-16 18:18:44 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#13920
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?