Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #5324] [MERGED] Model router #5408

New Issue
Closed
opened 2026-06-05 15:21:16 -04:00 by yindo · 0 comments
yindo commented 2026-06-05 15:21:16 -04:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/5324
Author: @shatfield4
Created: 4/1/2026
Status: ✅ Merged
Merged: 5/20/2026
Merged by: @timothycarambat

Base: master ← Head: 5313-feat-model-router

📝 Commits (10+)

  • af0dee6 add model router backend: schema, models, API, evaluator, provider, and cooldown
  • b632e62 add model router frontend: settings pages, workspace integration, rule builder with DnD, and system LLM support
  • 275eaff add routing indicator, new rule types, and UX improvements for model router
  • be5c298 add provider credential validation to model router and fix icon alignment
  • cea76b1 add llm classifcation rules for model router
  • efe2eab lint
  • 42ddcae add vision routing, agent support, and configurable cooldown to model router
  • 01fe935 Merge branch 'master' into 5313-feat-model-router
  • 1fb3064 add english translations for model router
  • 4b8c526 move isPathMatch to path utils

📊 Changes

76 files changed (+9029 additions, -70 deletions)

View changed files

📝 README.md (+3 -2)
➕ frontend/src/components/LLMSelection/ModelRouterOptions/index.jsx (+74 -0)
📝 frontend/src/components/SettingsSidebar/MenuOption/index.jsx (+8 -5)
📝 frontend/src/components/SettingsSidebar/index.jsx (+6 -0)
➕ frontend/src/components/WorkspaceChat/ChatContainer/ChatHistory/ModelRouteNotification/index.jsx (+66 -0)
📝 frontend/src/components/WorkspaceChat/ChatContainer/ChatHistory/index.jsx (+20 -0)
📝 frontend/src/components/WorkspaceChat/ChatContainer/ChatTooltips/index.jsx (+6 -0)
➕ frontend/src/components/WorkspaceChat/ChatContainer/PromptInput/LLMSelector/RouterPickerSelection/index.jsx (+64 -0)
📝 frontend/src/components/WorkspaceChat/ChatContainer/PromptInput/LLMSelector/index.jsx (+35 -14)
📝 frontend/src/components/WorkspaceChat/ChatContainer/WorkspaceModelPicker/index.jsx (+27 -9)
📝 frontend/src/components/WorkspaceChat/ChatContainer/index.jsx (+9 -3)
📝 frontend/src/locales/ar/common.js (+180 -1)
📝 frontend/src/locales/ca/common.js (+183 -1)
📝 frontend/src/locales/cs/common.js (+180 -1)
📝 frontend/src/locales/da/common.js (+178 -1)
📝 frontend/src/locales/de/common.js (+180 -1)
📝 frontend/src/locales/en/common.js (+176 -1)
📝 frontend/src/locales/es/common.js (+180 -1)
📝 frontend/src/locales/et/common.js (+180 -1)
📝 frontend/src/locales/fa/common.js (+179 -1)

...and 56 more files

📄 Description

Pull Request Type

  • ✨ feat (New feature)
  • 🐛 fix (Bug fix)
  • ♻️ refactor (Code refactoring without changing behavior)
  • 💄 style (UI style changes)
  • 🔨 chore (Build, CI, maintenance)
  • 📝 docs (Documentation updates)

Relevant Issues

resolves #5313
Translations PR: #5666
Docs PR: https://github.com/Mintplex-Labs/anythingllm-docs/pull/245

Description

  • Add model router as a new provider type that routes messages to different LLM providers/models based on configurable rules
  • Support two rule types: calculated (deterministic conditions) and LLM-classified (semantic matching via fallback model)
  • Route based on prompt content, conversation token count, message count, time of day, or image attachments
  • CSV support for keyword matching ("python,javascript,coding" matches any)
  • Configurable cooldown to cache routing decisions and avoid reevaluation on every message (decided to keep this so that users can dial it in to work for their specific workflows)
  • Full settings UI for creating/editing routers and rules with drag-and-drop priority ordering
  • Inline provider credential setup when configuring routing targets
  • Works in both regular chat and agent mode, including per-turn re-routing in agent mode
  • Ephemeral routing notification shown in chat during streaming with animated border, disappears on refresh
  • Router selectable at system level or per-workspace via the model picker

Visuals (if applicable)

Model routing animation

https://github.com/user-attachments/assets/d3b0a25b-d0cb-4802-9615-593e3eb6e917

Chat UI with model router switching to different models

Screenshot 2026-04-08 at 4 06 39 PM

Agent mode chat UI with router switching to different models

Screenshot 2026-04-08 at 4 11 37 PM

Model router settings page

Screenshot 2026-04-08 at 4 07 00 PM

Editing router

Screenshot 2026-04-08 at 4 07 28 PM

Router rules configuration

Screenshot 2026-04-08 at 4 07 17 PM

Additional Information

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated (if applicable)
  • I have tested my code functionality
  • Docker build succeeds locally

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/5324 **Author:** [@shatfield4](https://github.com/shatfield4) **Created:** 4/1/2026 **Status:** ✅ Merged **Merged:** 5/20/2026 **Merged by:** [@timothycarambat](https://github.com/timothycarambat) **Base:** `master` ← **Head:** `5313-feat-model-router` --- ### 📝 Commits (10+) - [`af0dee6`](https://github.com/Mintplex-Labs/anything-llm/commit/af0dee612b79ca735f6209dcbf46b2e25167a9b6) add model router backend: schema, models, API, evaluator, provider, and cooldown - [`b632e62`](https://github.com/Mintplex-Labs/anything-llm/commit/b632e6235d992ef77ae850054850cf44e9a002fa) add model router frontend: settings pages, workspace integration, rule builder with DnD, and system LLM support - [`275eaff`](https://github.com/Mintplex-Labs/anything-llm/commit/275eaff1eb0191113fcd0270242ba010b35f2293) add routing indicator, new rule types, and UX improvements for model router - [`be5c298`](https://github.com/Mintplex-Labs/anything-llm/commit/be5c298f7bd1b41fc6cffdb3f0336ad1fed49cfd) add provider credential validation to model router and fix icon alignment - [`cea76b1`](https://github.com/Mintplex-Labs/anything-llm/commit/cea76b187b0cce383ee28ae2769773a3fc444c49) add llm classifcation rules for model router - [`efe2eab`](https://github.com/Mintplex-Labs/anything-llm/commit/efe2eabb7e0f900e65860d1303038391d232d732) lint - [`42ddcae`](https://github.com/Mintplex-Labs/anything-llm/commit/42ddcaeedbc2dde4f34be3189115943e9abeccce) add vision routing, agent support, and configurable cooldown to model router - [`01fe935`](https://github.com/Mintplex-Labs/anything-llm/commit/01fe935eca55961495fac865324457cdeff3187d) Merge branch 'master' into 5313-feat-model-router - [`1fb3064`](https://github.com/Mintplex-Labs/anything-llm/commit/1fb3064a3487f53c59dced27d1c66ea8b0a38be6) add english translations for model router - [`4b8c526`](https://github.com/Mintplex-Labs/anything-llm/commit/4b8c52672bc8127d4e56c3f89c004efef48aac93) move isPathMatch to path utils ### 📊 Changes **76 files changed** (+9029 additions, -70 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+3 -2) ➕ `frontend/src/components/LLMSelection/ModelRouterOptions/index.jsx` (+74 -0) 📝 `frontend/src/components/SettingsSidebar/MenuOption/index.jsx` (+8 -5) 📝 `frontend/src/components/SettingsSidebar/index.jsx` (+6 -0) ➕ `frontend/src/components/WorkspaceChat/ChatContainer/ChatHistory/ModelRouteNotification/index.jsx` (+66 -0) 📝 `frontend/src/components/WorkspaceChat/ChatContainer/ChatHistory/index.jsx` (+20 -0) 📝 `frontend/src/components/WorkspaceChat/ChatContainer/ChatTooltips/index.jsx` (+6 -0) ➕ `frontend/src/components/WorkspaceChat/ChatContainer/PromptInput/LLMSelector/RouterPickerSelection/index.jsx` (+64 -0) 📝 `frontend/src/components/WorkspaceChat/ChatContainer/PromptInput/LLMSelector/index.jsx` (+35 -14) 📝 `frontend/src/components/WorkspaceChat/ChatContainer/WorkspaceModelPicker/index.jsx` (+27 -9) 📝 `frontend/src/components/WorkspaceChat/ChatContainer/index.jsx` (+9 -3) 📝 `frontend/src/locales/ar/common.js` (+180 -1) 📝 `frontend/src/locales/ca/common.js` (+183 -1) 📝 `frontend/src/locales/cs/common.js` (+180 -1) 📝 `frontend/src/locales/da/common.js` (+178 -1) 📝 `frontend/src/locales/de/common.js` (+180 -1) 📝 `frontend/src/locales/en/common.js` (+176 -1) 📝 `frontend/src/locales/es/common.js` (+180 -1) 📝 `frontend/src/locales/et/common.js` (+180 -1) 📝 `frontend/src/locales/fa/common.js` (+179 -1) _...and 56 more files_ </details> ### 📄 Description ### Pull Request Type <!-- For change type, change [ ] to [x]. --> - [x] ✨ feat (New feature) - [ ] 🐛 fix (Bug fix) - [ ] ♻️ refactor (Code refactoring without changing behavior) - [ ] 💄 style (UI style changes) - [ ] 🔨 chore (Build, CI, maintenance) - [ ] 📝 docs (Documentation updates) ### Relevant Issues <!-- Use "resolves #xxx" to auto resolve on merge. Otherwise, please use "connect #xxx" --> resolves #5313 Translations PR: #5666 Docs PR: https://github.com/Mintplex-Labs/anythingllm-docs/pull/245 ### Description <!-- Describe the changes in this PR that are impactful to the repo. What problem does it solve? --> - Add model router as a new provider type that routes messages to different LLM providers/models based on configurable rules - Support two rule types: calculated (deterministic conditions) and LLM-classified (semantic matching via fallback model) - Route based on prompt content, conversation token count, message count, time of day, or image attachments - CSV support for keyword matching ("python,javascript,coding" matches any) - Configurable cooldown to cache routing decisions and avoid reevaluation on every message (decided to keep this so that users can dial it in to work for their specific workflows) - Full settings UI for creating/editing routers and rules with drag-and-drop priority ordering - Inline provider credential setup when configuring routing targets - Works in both regular chat and agent mode, including per-turn re-routing in agent mode - Ephemeral routing notification shown in chat during streaming with animated border, disappears on refresh - Router selectable at system level or per-workspace via the model picker ### Visuals (if applicable) <!-- Add screenshots or screen recordings to demonstrate the changes, especially for UI updates. --> #### Model routing animation https://github.com/user-attachments/assets/d3b0a25b-d0cb-4802-9615-593e3eb6e917 #### Chat UI with model router switching to different models <img width="1817" height="924" alt="Screenshot 2026-04-08 at 4 06 39 PM" src="https://github.com/user-attachments/assets/a236edbf-5360-4bc8-ab68-3fd07ea284ae" /> #### Agent mode chat UI with router switching to different models <img width="1817" height="924" alt="Screenshot 2026-04-08 at 4 11 37 PM" src="https://github.com/user-attachments/assets/e43987f7-7eb5-4999-8732-30beca3cef57" /> #### Model router settings page <img width="1817" height="926" alt="Screenshot 2026-04-08 at 4 07 00 PM" src="https://github.com/user-attachments/assets/e0de9aeb-5bae-4c66-8fe2-27c7b4fcc7da" /> #### Editing router <img width="1819" height="925" alt="Screenshot 2026-04-08 at 4 07 28 PM" src="https://github.com/user-attachments/assets/fa3d70c1-d5d6-4f72-80fb-2d704fc99542" /> #### Router rules configuration <img width="1818" height="928" alt="Screenshot 2026-04-08 at 4 07 17 PM" src="https://github.com/user-attachments/assets/cf52242b-d105-4eea-94f1-0fa1293e8afa" /> ### Additional Information <!-- Add any other context about the Pull Request here that was not captured above. --> ### Developer Validations <!-- All of the applicable items should be checked. --> - [x] I ran `yarn lint` from the root of the repo & committed changes - [x] Relevant documentation has been updated (if applicable) - [x] I have tested my code functionality - [x] Docker build succeeds locally --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-06-05 15:21:16 -04:00
yindo closed this issue 2026-06-05 15:21:16 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5408
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?