langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-08-25 11:59:42 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[Proposal] Add a WFGY 16-problem RAG troubleshooting template to Dify #22255

New Issue
Open
opened 2026-02-21 20:16:19 -05:00 by yindo · 2 comments
yindo commented 2026-02-21 20:16:19 -05:00
Owner
Copy Link
Copy Source

Originally created by @onestardao on GitHub (Feb 20, 2026).

Originally assigned to: @onestardao on GitHub.

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Hi Dify team,

First, thanks for open-sourcing such a powerful visual platform for agents and RAG apps.:contentReference[oaicite:5]{index=5}

I'm the author of WFGY, an MIT-licensed text-only framework for RAG debugging.
The core resource is a 16-problem RAG failure checklist:

https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md

This checklist has already been referenced by:

  • Harvard MIMS Lab – ToolUniverse (LLM tools benchmark) as a robustness / RAG debugging entry,
  • QCRI LLM Lab – Multimodal RAG Survey,
  • Rankify from University of Innsbruck (academic RAG / re-ranking toolkit).

Proposal

Because Dify focuses on visually composing workflows, it seems like a natural place to expose a very small, optional “RAG Clinic” template based on the 16 problems:

  1. New template / recipe

    • Something like: RAG troubleshooting flow (WFGY 16-problem checklist)
    • A pre-built flow that:
      • logs user query, retrieved chunks, final answer,
      • routes “bad” cases to a classifier agent that uses the ProblemMap to label the failure type (No.1–16),
      • stores labels so users can see where their pipeline is breaking (retrieval vs reasoning vs state vs infra).

  2. Docs snippet

    • Short section in the RAG docs explaining the 4 regions and 16 problems,
    • plus a single link back to the ProblemMap README for details.

  3. Low maintenance

    • All WFGY logic can live inside one or two prompt blocks in the template.
    • No new Python dependencies; just text and examples.

I’d be happy to:

  • propose a concrete Dify flow JSON / YAML,
  • provide example prompts that implement the 16-problem classification,
  • keep the issue and PR fully optional and easy to remove if it does not fit.

If this direction sounds reasonable, I can start with a draft template and small doc addition.

2. Additional context or comments

No response

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @onestardao on GitHub (Feb 20, 2026). Originally assigned to: @onestardao on GitHub. ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. Hi Dify team, First, thanks for open-sourcing such a powerful visual platform for agents and RAG apps.:contentReference[oaicite:5]{index=5} I'm the author of **WFGY**, an MIT-licensed text-only framework for RAG debugging. The core resource is a **16-problem RAG failure checklist**: > https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md This checklist has already been referenced by: - **Harvard MIMS Lab – ToolUniverse** (LLM tools benchmark) as a robustness / RAG debugging entry, - **QCRI LLM Lab – Multimodal RAG Survey**, - **Rankify** from University of Innsbruck (academic RAG / re-ranking toolkit). ### Proposal Because Dify focuses on **visually composing workflows**, it seems like a natural place to expose a very small, optional **“RAG Clinic” template** based on the 16 problems: 1. **New template / recipe** - Something like: `RAG troubleshooting flow (WFGY 16-problem checklist)` - A pre-built flow that: - logs user query, retrieved chunks, final answer, - routes “bad” cases to a classifier agent that uses the ProblemMap to label the failure type (No.1–16), - stores labels so users can see where their pipeline is breaking (retrieval vs reasoning vs state vs infra). 2. **Docs snippet** - Short section in the RAG docs explaining the 4 regions and 16 problems, - plus a single link back to the ProblemMap README for details. 3. **Low maintenance** - All WFGY logic can live inside one or two prompt blocks in the template. - No new Python dependencies; just text and examples. I’d be happy to: - propose a concrete Dify flow JSON / YAML, - provide example prompts that implement the 16-problem classification, - keep the issue and PR fully optional and easy to remove if it does not fit. If this direction sounds reasonable, I can start with a draft template and small doc addition. ### 2. Additional context or comments _No response_ ### 3. Can you help us with this feature? - [x] I am interested in contributing to this feature.
yindo added the 💪 enhancement📚 documentation labels 2026-02-21 20:16:19 -05:00
yindo commented 2026-02-21 20:16:20 -05:00
Author
Owner
Copy Link
Copy Source

@darshjme-codes commented on GitHub (Feb 20, 2026):

Problem: Debugging a broken RAG flow in Dify is painful because there’s no visual way to tag failures like retrieval‑no‑results or hallucination.

Solution: Add a tiny “RAG Clinic” recipe that runs a classifier agent on the 16 WFGY problem labels. The flow can be defined as a JSON template and uses a single prompt to map the failure to a number 1‑16.

{
  "name": "RAG Clinic",
  "steps": [
    {"type":"input","name":"query"},
    {"type":"retrieve","name":"docs"},
    {"type":"prompt","name":"classify","prompt":"Given the query, retrieved chunks and answer, which of the 16 WFGY problems caused the failure? Return only the number."},
    {"type":"output","name":"problem_id"}
  ]
}

Reference: #1542

@darshjme-codes commented on GitHub (Feb 20, 2026): Problem: Debugging a broken RAG flow in Dify is painful because there’s no visual way to tag failures like retrieval‑no‑results or hallucination. Solution: Add a tiny “RAG Clinic” recipe that runs a classifier agent on the 16 WFGY problem labels. The flow can be defined as a JSON template and uses a single prompt to map the failure to a number 1‑16. ```json { "name": "RAG Clinic", "steps": [ {"type":"input","name":"query"}, {"type":"retrieve","name":"docs"}, {"type":"prompt","name":"classify","prompt":"Given the query, retrieved chunks and answer, which of the 16 WFGY problems caused the failure? Return only the number."}, {"type":"output","name":"problem_id"} ] } ``` Reference: #1542
yindo commented 2026-02-21 20:16:20 -05:00
Author
Owner
Copy Link
Copy Source

@onestardao commented on GitHub (Feb 20, 2026):

This is a really nice framing, thanks for writing it up so clearly.

I can help draft the "RAG Clinic" flow as a JSON template based on the 16 failure labels, plus a short doc that explains how to read the problem_id and what to log for each case.

If this sounds useful, let me know which repo / folder you prefer for the template (and whether you want it as a built-in recipe or just a documented example). I’m happy to keep it as simple and Dify-native as possible.

@onestardao commented on GitHub (Feb 20, 2026): This is a really nice framing, thanks for writing it up so clearly. I can help draft the "RAG Clinic" flow as a JSON template based on the 16 failure labels, plus a short doc that explains how to read the problem_id and what to log for each case. If this sounds useful, let me know which repo / folder you prefer for the template (and whether you want it as a built-in recipe or just a documented example). I’m happy to keep it as simple and Dify-native as possible.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 💪 enhancement 📚 documentation
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#22255
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?