0 Fork 0

mirror of https://github.com/MervinPraison/PraisonAIUI.git synced 2026-11-10 17:29:50 -10:00

Files

T

Mervin Praison b5e8bfc39e feat(docs): add comprehensive feature documentation updates

Added new documentation for various features including attachments,
browser automation, comparison with OpenClaw, and more. This totals
to 13 new or modified files with extensive content to enhance
user understanding and usage of the platform's capabilities.

2026-03-06 20:54:40 +00:00

2.4 KiB

Raw Permalink Blame History

Model Fallback

Configure model fallback chains for high availability — if the primary LLM fails, automatically try the next model in the chain.

Quick Start

# List available models
curl http://localhost:8083/api/models

# Set a fallback chain
curl -X PUT http://localhost:8083/api/models/fallback \
  -H "Content-Type: application/json" \
  -d '{"models": ["gpt-4o", "gpt-4o-mini", "claude-3-5-sonnet"]}'

How It Works

graph TD
    A["User Request"] --> B["gpt-4o (Primary)"]
    B -->|Success| C["Return Response"]
    B -->|Failure| D["gpt-4o-mini (Fallback 1)"]
    D -->|Success| C
    D -->|Failure| E["claude-3-5-sonnet (Fallback 2)"]
    E -->|Success| C
    E -->|Failure| F["Error: All models failed"]

When a request is sent:

Try the primary model first
If it fails (rate limit, timeout, error), try fallback 1
Continue down the chain until a model responds
If all models fail, return an error

Model Discovery

The ModelFallbackManager tries to discover models automatically:

LiteLLM — If installed, reads litellm.model_list for configured models
Static catalog — Falls back to common models (gpt-4o, gpt-4o-mini, claude-3-5-sonnet, gemini-2.0-flash)

Configuration

from praisonaiui.features.model_fallback import ModelFallbackManager

mgr = ModelFallbackManager()

# Get available models
models = mgr.get_models()
# [{"id": "gpt-4o", "provider": "openai"}, ...]

# Set fallback chain
mgr.set_fallback_chain(["gpt-4o", "gpt-4o-mini"])

# Get current chain
chain = mgr.get_fallback_chain()
# ["gpt-4o", "gpt-4o-mini"]

REST API

Endpoint	Method	Description
`/api/models`	GET	List available models + current fallback chain
`/api/models/fallback`	PUT	Set the fallback chain

List Models

curl http://localhost:8083/api/models

{
  "models": [
    {"id": "gpt-4o", "provider": "openai"},
    {"id": "gpt-4o-mini", "provider": "openai"},
    {"id": "claude-3-5-sonnet", "provider": "anthropic"},
    {"id": "gemini-2.0-flash", "provider": "google"}
  ],
  "fallback_chain": ["gpt-4o", "gpt-4o-mini"]
}

Gateway Chat — Chat uses models via fallback chain
Providers — LLM provider configuration

2.4 KiB Raw Permalink Blame History