anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 10:15:31 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

[PR #13189] feat: add model fallback support with TTFT-based timeout #14549

New Issue
Open
opened 2026-02-16 18:19:20 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 18:19:20 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/anomalyco/opencode/pull/13189

State: open
Merged: No

What does this PR do?

This PR adds model fallback support for agents. When a primary model fails or is too slow to respond, the system automatically cycles through configured fallback models.
feat #7602 #9575

Key changes:

  • New config options: fallback_models (list of fallback models in priority order) and first_token_timeout (base timeout in ms for first-token detection, scaled by input size: base + inputChars * 0.5)
  • Fallback cycling logic (SessionFallback): builds a deduplicated model list, cycles through all models starting from the last successful one, and remembers which model worked per session/agent pair
  • TTFT timeout: measures time-to-first-token from after LLM.stream() returns (excluding framework init overhead), so the timeout reflects actual network/model latency
  • Short retry before switching: for retryable API errors, one retry attempt is made on the same model before falling back to the next
  • Error handling: context overflow and user abort errors skip fallback (not model-specific); FirstTokenTimeoutError is properly unwrapped from AbortError so it triggers fallback instead of
    being treated as user cancellation
  • Toast notification: when a fallback switch occurs, a toast is shown indicating the model change
  • Edge case fixes: clamps out-of-bounds fallbackStartIndex to prevent infinite loops when models are removed from config; resets both indices when the remembered model fails to resolve

Files changed:

  • src/session/fallback.ts — new module for fallback state management and model cycling
  • src/session/processor.ts — main integration: TTFT timer, fallback loop, error classification
  • src/agent/agent.ts / src/config/config.ts — schema additions for fallback_models and first_token_timeout
  • src/session/prompt.ts / src/session/message-v2.ts — pass fallback config to processor; handle FirstTokenTimeoutError

How did you verify your code works?

  • Added unit tests for SessionFallback (buildModelList, nextIndex cycling, recordSuccess/getStartIndex, full cycle simulations including remembered-model and out-of-bounds edge cases) — see test/session/fallback.test.ts
  • Added unit tests for TTFT computation (estimateInputChars, computeTtftTimeout with various input sizes) and FirstTokenTimeoutError serialization — see test/session/processor-ttft.test.ts
  • Manual testing with multiple model configurations to verify fallback cycling, TTFT timeout triggering, and toast notifications
**Original Pull Request:** https://github.com/anomalyco/opencode/pull/13189 **State:** open **Merged:** No --- ### What does this PR do? This PR adds model fallback support for agents. When a primary model fails or is too slow to respond, the system automatically cycles through configured fallback models. feat #7602 #9575 **Key changes:** - **New config options**: `fallback_models` (list of fallback models in priority order) and `first_token_timeout` (base timeout in ms for first-token detection, scaled by input size: `base + inputChars * 0.5`) - **Fallback cycling logic** (`SessionFallback`): builds a deduplicated model list, cycles through all models starting from the last successful one, and remembers which model worked per session/agent pair - **TTFT timeout**: measures time-to-first-token from after `LLM.stream()` returns (excluding framework init overhead), so the timeout reflects actual network/model latency - **Short retry before switching**: for retryable API errors, one retry attempt is made on the same model before falling back to the next - **Error handling**: context overflow and user abort errors skip fallback (not model-specific); `FirstTokenTimeoutError` is properly unwrapped from `AbortError` so it triggers fallback instead of being treated as user cancellation - **Toast notification**: when a fallback switch occurs, a toast is shown indicating the model change - **Edge case fixes**: clamps out-of-bounds `fallbackStartIndex` to prevent infinite loops when models are removed from config; resets both indices when the remembered model fails to resolve **Files changed:** - `src/session/fallback.ts` — new module for fallback state management and model cycling - `src/session/processor.ts` — main integration: TTFT timer, fallback loop, error classification - `src/agent/agent.ts` / `src/config/config.ts` — schema additions for `fallback_models` and `first_token_timeout` - `src/session/prompt.ts` / `src/session/message-v2.ts` — pass fallback config to processor; handle `FirstTokenTimeoutError` ### How did you verify your code works? - Added unit tests for `SessionFallback` (buildModelList, nextIndex cycling, recordSuccess/getStartIndex, full cycle simulations including remembered-model and out-of-bounds edge cases) — see `test/session/fallback.test.ts` - Added unit tests for TTFT computation (estimateInputChars, computeTtftTimeout with various input sizes) and `FirstTokenTimeoutError` serialization — see `test/session/processor-ttft.test.ts` - Manual testing with multiple model configurations to verify fallback cycling, TTFT timeout triggering, and toast notifications
yindo added the pull-request label 2026-02-16 18:19:20 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#14549
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?