langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 20:45:26 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #31108] refactor: Optimize question classifier performance #33093

New Issue
Open
opened 2026-02-21 20:52:39 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:52:39 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/31108

State: open
Merged: No

Summary

This PR optimizes the question classifier performance by requiring the LLM to output only category_id instead of both category_id and category_name.

Problem: When category descriptions are very long, the model spends most of its time generating the category name text, which significantly increases the classifier's latency.

Solution: Modified the classifier to only require category_id in the LLM output. The category_name is retrieved from the internal classes map using the category_id.

Impact:

  • Significantly reduces classifier latency, especially for categories with long descriptions
  • Can reduce response time to sub-second levels
  • Reduces output tokens and API costs
  • Backward compatible - no functionality loss

Changes:

  • Modified JSON validation in question_classifier_node.py to only check for category_id (line 182)
  • Updated all prompt examples in template_prompts.py to remove category_name from expected output
  • The category_name is still returned in the final output by looking it up from the classes map

Fixes #31103

Screenshots

Before After
b14a13221b8a247624a6c8272f3926bc
2bfa6b224186b21ced0b9abecb410dbd

Checklist

  • This change requires a documentation update, included: https://github.com/langgenius/dify-docs
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran make lint and make type-check to ensure the code follows the project's coding standards.
**Original Pull Request:** https://github.com/langgenius/dify/pull/31108 **State:** open **Merged:** No --- Summary This PR optimizes the question classifier performance by requiring the LLM to output only category_id instead of both category_id and category_name. Problem: When category descriptions are very long, the model spends most of its time generating the category name text, which significantly increases the classifier's latency. Solution: Modified the classifier to only require category_id in the LLM output. The category_name is retrieved from the internal classes map using the category_id. Impact: - Significantly reduces classifier latency, especially for categories with long descriptions - Can reduce response time to sub-second levels - Reduces output tokens and API costs - Backward compatible - no functionality loss Changes: - Modified JSON validation in question_classifier_node.py to only check for category_id (line 182) - Updated all prompt examples in template_prompts.py to remove category_name from expected output - The category_name is still returned in the final output by looking it up from the classes map Fixes #31103 Screenshots | Before | After | |--------|-------| | <img width="452" height="442" alt="b14a13221b8a247624a6c8272f3926bc" src="https://github.com/user-attachments/assets/13a70733-fb0f-4c4e-8df5-656ebfb28664" /> | <img width="853" height="516" alt="2bfa6b224186b21ced0b9abecb410dbd" src="https://github.com/user-attachments/assets/4be314b5-eda3-4b1a-872a-7a644ea1518b" /> | Checklist - This change requires a documentation update, included: https://github.com/langgenius/dify-docs - I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!) - I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change. - I've updated the documentation accordingly. - I ran make lint and make type-check to ensure the code follows the project's coding standards.
yindo added the pull-request label 2026-02-21 20:52:39 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#33093
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?