Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #1477] [MERGED] Improve RAG responses via source backfilling #3692

New Issue
Closed
opened 2026-02-22 18:34:18 -05:00 by yindo · 0 comments
yindo commented 2026-02-22 18:34:18 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/1477
Author: @timothycarambat
Created: 5/21/2024
Status: ✅ Merged
Merged: 5/23/2024
Merged by: @timothycarambat

Base: master ← Head: patch/rag-improvements-via-src-backfilling

📝 Commits (3)

  • 53e11cd Improve RAG responses via source backfilling
  • 8b61090 Hide irrelevant citations from UI
  • e0987d7 Merge branch 'master' into patch/rag-improvements-via-src-backfilling

📊 Changes

3 files changed (+135 additions, -17 deletions)

View changed files

📝 server/utils/chats/index.js (+18 -9)
📝 server/utils/chats/stream.js (+18 -8)
📝 server/utils/helpers/chat/index.js (+99 -0)

📄 Description

resolves #1240

Pull Request Type

  • ✨ feat
  • 🐛 fix
  • ♻️ refactor
  • 💄 style
  • 🔨 chore
  • 📝 docs

Warning

This PR enables query to work with chat history now but only in the app. Behavior in the embed is the same.

In an effort to improve RAG results without adding unwarranted or "conditional" middleware like ReRankers, there is a lot of room for improvement of the traditional RAG pipeline AnythingLLM uses. This PR includes the ability to backfill sources during a chat session.

What is backfilling?

Currently, AnythingLLM will append Context snippets to the system prompt on each chat. These contexts come from the attached VectorStores .similaritySearch method. While this works for direct questions this does not scale well as chat's become higher context or more vague. This leads to a bad RAG experience in a traditional chat, but is far worse in query where a follow-up query, that is contextually relevant, is deemed irrelevant.

Example

prompt 1: "What is anythingllm?"

  • possibly get 4 good sources and get a good LLM response as it is operating with high context.

prompt 2: "Tell me some features"

  • This question is appropriate, topical, and a decent inquiry but standalone is quite vague when viewed outside of the context of the chat.
  • Possible get 0 - 1 maybe relevant sources
  • We now must rely on the maybe existent snippet for source and hope the previous LLM response has something worth looking at

introduced in this pr

prompt 1: "What is anythingllm?"

  • possibly get 4 good sources and get a good LLM response as it is operating with high context.

prompt 2: "Tell me some features"

  • Possible get 0 - 1 maybe relevant sources
  • We now can backfill the context window in this order of priority
    • Pinned documents
    • This queries search results
    • Work backwards in chat history citations to backfill the source window to the workspace's assigned snippet window value.

With this, we can now ensure that follow-up questions will be more relevant and contextually pertinent as a conversation continues.

"What if the contextual topic changes"?

  • This is handled as we prioritize sources from search results first, then backfill in by more recent chats with historical references
  • If you have a relevant contextual change in the chat, those sources will take precedence over the historical ones and will become the new historical reference

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/1477 **Author:** [@timothycarambat](https://github.com/timothycarambat) **Created:** 5/21/2024 **Status:** ✅ Merged **Merged:** 5/23/2024 **Merged by:** [@timothycarambat](https://github.com/timothycarambat) **Base:** `master` ← **Head:** `patch/rag-improvements-via-src-backfilling` --- ### 📝 Commits (3) - [`53e11cd`](https://github.com/Mintplex-Labs/anything-llm/commit/53e11cdc0a715ad19a014dff6737fab36b2bfd31) Improve RAG responses via source backfilling - [`8b61090`](https://github.com/Mintplex-Labs/anything-llm/commit/8b610902fe3c6106dec18a3d85bf3aff6a958ccd) Hide irrelevant citations from UI - [`e0987d7`](https://github.com/Mintplex-Labs/anything-llm/commit/e0987d7b5b38ae3d385fd084c4bc7cd1b7bc93af) Merge branch 'master' into patch/rag-improvements-via-src-backfilling ### 📊 Changes **3 files changed** (+135 additions, -17 deletions) <details> <summary>View changed files</summary> 📝 `server/utils/chats/index.js` (+18 -9) 📝 `server/utils/chats/stream.js` (+18 -8) 📝 `server/utils/helpers/chat/index.js` (+99 -0) </details> ### 📄 Description resolves #1240 ### Pull Request Type <!-- For change type, change [ ] to [x]. --> - [ ] ✨ feat - [ ] 🐛 fix - [ ] ♻️ refactor - [ ] 💄 style - [x] 🔨 chore - [ ] 📝 docs > [!WARNING] > This PR enables `query` to work with chat history now but **only** in the app. Behavior in the embed is the same. In an effort to improve RAG results without adding unwarranted or "conditional" middleware like ReRankers, there is a lot of room for improvement of the traditional RAG pipeline AnythingLLM uses. This PR includes the ability to **backfill** sources during a chat session. ## What is backfilling? Currently, AnythingLLM will append `Context` snippets to the `system` prompt on each chat. These contexts come from the attached VectorStores `.similaritySearch` method. While this works for direct questions this does not scale well as chat's become higher context or more vague. This leads to a bad RAG experience in a traditional chat, but is far worse in `query` where a follow-up query, that is contextually relevant, is deemed irrelevant. ## Example **prompt 1**: "What is anythingllm?" - possibly get 4 good sources and get a good LLM response as it is operating with high context. **prompt 2**: "Tell me some features" - This question is appropriate, topical, and a decent inquiry but standalone is quite vague when viewed outside of the context of the chat. - Possible get 0 - 1 maybe relevant sources - We now must rely on the _maybe_ existent snippet for source and hope the previous LLM response has something worth looking at _introduced in this pr_ **prompt 1**: "What is anythingllm?" - possibly get 4 good sources and get a good LLM response as it is operating with high context. **prompt 2**: "Tell me some features" - Possible get 0 - 1 maybe relevant sources - We now can backfill the context window in this order of priority - Pinned documents - This queries search results - Work backwards in chat history citations to backfill the source window to the workspace's assigned snippet window value. With this, we can now ensure that follow-up questions will be more relevant and contextually pertinent as a conversation continues. **"What if the contextual topic changes"?** - This is handled as we prioritize sources from search results first, then backfill in by more recent chats with historical references - If you have a relevant contextual change in the chat, those sources will take precedence over the historical ones and will become the new historical reference --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-22 18:34:18 -05:00
yindo closed this issue 2026-02-22 18:34:18 -05:00
yindo changed title from [PR #1477] Improve RAG responses via source backfilling to [PR #1477] [MERGED] Improve RAG responses via source backfilling 2026-06-05 15:14:44 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#3692
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?