langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #31688] feat: LLM node support model list fallback #33347

New Issue
Open
opened 2026-02-21 20:53:07 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:53:07 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/31688

State: open
Merged: No

Important

  1. Make sure you have read our contribution guidelines
  2. Ensure there is an associated issue and you have been assigned to it
  3. Use the correct syntax to link this PR: Fixes #<issue number>.

Summary

Fixes https://github.com/langgenius/dify/issues/31118

This pr introduce new error handling of LLM node, which user can select fallback model list when model run throw exception.

Before: every LLM node need add exception branch or use tool to make model fallback.
LLM node run failed -> error branch -> some logic about handle fallback -> end

Now: it can config fallback model in LLM node which need fallback.
LLM node run failed -> traverse fallback model list -> retry with fallback model -> end

Some edge case:

  • If fallback model list is empty, it just like none error handling
  • If fallback model list traverse fully and also failed, it will make workflow failed

Changelog:

  1. backend add enum about ERROR_STRATEGY, add some metadata field to store fallback model info
  2. backend add unittest about this feature
  3. frontend add logic about parse fallback model info, and support display model attempt info

TODO:

  • communicate about this feature
  • add docs about this new error handling

Screenshots

Add Fallback Model option only in LLM node error handing

image

Fallback model error handling select model list to fallback

image

Fallback success, LLM node status will stay exception, but we can access model retry information

image image image

Fallback model error handling with empty fallback model list

image image

Fallback model list has been retried and still failed

image image

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran make lint and make type-check (backend) and cd web && npx lint-staged (frontend) to appease the lint gods
**Original Pull Request:** https://github.com/langgenius/dify/pull/31688 **State:** open **Merged:** No --- > [!IMPORTANT] > > 1. Make sure you have read our [contribution guidelines](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) > 1. Ensure there is an associated issue and you have been assigned to it > 1. Use the correct syntax to link this PR: `Fixes #<issue number>`. ## Summary Fixes https://github.com/langgenius/dify/issues/31118 This pr introduce new error handling of LLM node, which user can select fallback model list when model run throw exception. Before: every LLM node need add exception branch or use tool to make model fallback. LLM node run failed -> error branch -> some logic about handle fallback -> end Now: it can config fallback model in LLM node which need fallback. LLM node run failed -> traverse fallback model list -> retry with fallback model -> end Some edge case: - If fallback model list is empty, it just like none error handling - If fallback model list traverse fully and also failed, it will make workflow failed Changelog: 1. backend add enum about `ERROR_STRATEGY`, add some metadata field to store fallback model info 2. backend add unittest about this feature 3. frontend add logic about parse fallback model info, and support display model attempt info TODO: - [ ] communicate about this feature - [ ] add docs about this new error handling ## Screenshots Add `Fallback Model` option **only in LLM node** error handing <img width="413" height="523" alt="image" src="https://github.com/user-attachments/assets/844ffa5f-53b1-4210-87ab-b43b80c9152d" /> Fallback model error handling select model list to fallback <img width="1036" height="630" alt="image" src="https://github.com/user-attachments/assets/d87ed3bf-a359-4e40-abcf-0f7480741e42" /> Fallback success, LLM node status will stay `exception`, but we can access model retry information <img width="1111" height="774" alt="image" src="https://github.com/user-attachments/assets/eacc43dc-2da1-45ee-94f7-b67e0904d130" /> <img width="406" height="179" alt="image" src="https://github.com/user-attachments/assets/8558ff8f-9515-49fd-92fe-490ed9de4c86" /> <img width="1289" height="782" alt="image" src="https://github.com/user-attachments/assets/c6eeaa44-f9b5-4d5f-8ef3-e3d2d3478479" /> Fallback model error handling with empty fallback model list <img width="996" height="545" alt="image" src="https://github.com/user-attachments/assets/7ea63cff-8162-499d-b527-9745fc2b5c26" /> <img width="1282" height="405" alt="image" src="https://github.com/user-attachments/assets/1b58d45c-feb8-42a1-8e8f-c10f262cf7c9" /> Fallback model list has been retried and still failed <img width="1013" height="609" alt="image" src="https://github.com/user-attachments/assets/9560941b-04f9-46d5-98e5-0872c6e52247" /> <img width="1282" height="621" alt="image" src="https://github.com/user-attachments/assets/cae1e7e0-fa17-42b9-a958-7b22d3802f63" /> ## Checklist - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs) - [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!) - [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change. - [x] I've updated the documentation accordingly. - [x] I ran `make lint` and `make type-check` (backend) and `cd web && npx lint-staged` (frontend) to appease the lint gods
yindo added the pull-request label 2026-02-21 20:53:07 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#33347
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?