Proposal: add WFGY 16 Problem Map as RAG failure mode map #1

New Issue

2026-02-16T00:15:50-05:00

yindo commented

2026-02-16 00:15:50 -05:00

Originally created by @onestardao on GitHub (Feb 12, 2026).

Hi, thanks for putting together this RAG resource list.

I wanted to ask whether a failure mode taxonomy and debugging guide for RAG systems would be in scope as an additional resource.

What WFGY Problem Map is

WFGY is an open source project that defines a 16 mode failure map (No.1 to No.16) for LLM plus RAG pipelines. The modes include:

retrieval hallucination even when relevant evidence is present
vector store ingestion and index fragmentation
bootstrap ordering and infra race conditions between API gateway and vector DB
secrets and config drift only visible on first production deploy
and other quietly dangerous failure modes

Problem Map README:

https://github.com/onestardao/WFGY/tree/main/ProblemMap#readme

For each mode, there is a description, typical symptoms and suggested minimal countermeasures.

Why it might help this list

Many RAG frameworks already exist here. A concrete failure mode map gives teams:

a shared vocabulary to describe RAG outages and mis behaviour
a checklist before shipping to production
a way to organise post mortems and mitigation playbooks

Possible entry

If you think this fits the scope, one possible entry under a “Debugging / reliability / hardening” section could be:

WFGY 16 Problem Map – RAG failure mode taxonomy
Open source map of 16 real world failure modes (No.1 to No.16) for LLM plus RAG systems, with debugging checklists and mitigation ideas.
https://github.com/onestardao/WFGY/tree/main/ProblemMap#readme

If this feels out of scope for the list, no worries and thanks for considering it.

Originally created by @onestardao on GitHub (Feb 12, 2026). Hi, thanks for putting together this RAG resource list. I wanted to ask whether a failure mode taxonomy and debugging guide for RAG systems would be in scope as an additional resource. What WFGY Problem Map is WFGY is an open source project that defines a 16 mode failure map (No.1 to No.16) for LLM plus RAG pipelines. The modes include: retrieval hallucination even when relevant evidence is present vector store ingestion and index fragmentation bootstrap ordering and infra race conditions between API gateway and vector DB secrets and config drift only visible on first production deploy and other quietly dangerous failure modes Problem Map README: https://github.com/onestardao/WFGY/tree/main/ProblemMap#readme For each mode, there is a description, typical symptoms and suggested minimal countermeasures. Why it might help this list Many RAG frameworks already exist here. A concrete failure mode map gives teams: a shared vocabulary to describe RAG outages and mis behaviour a checklist before shipping to production a way to organise post mortems and mitigation playbooks Possible entry If you think this fits the scope, one possible entry under a “Debugging / reliability / hardening” section could be: WFGY 16 Problem Map – RAG failure mode taxonomy Open source map of 16 real world failure modes (No.1 to No.16) for LLM plus RAG systems, with debugging checklists and mitigation ideas. https://github.com/onestardao/WFGY/tree/main/ProblemMap#readme If this feels out of scope for the list, no worries and thanks for considering it.

yindo referenced this issue

2026-02-16 00:15:51 -05:00

[PR #1] [MERGED] feat: setup initial papers/structure #2

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: run-llama/awesome-rag#1