Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3587] [FEAT]: Automatically pin documents that have been found based on vector search for improved RAG #2319

New Issue
Closed
opened 2026-02-22 18:29:09 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:29:09 -05:00
Owner
Copy Link
Copy Source

Originally created by @illov on GitHub (Apr 3, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3587

What would you like to see?

A option in the workspace: If a related snippet is found in the vector search then that document is fed to the LLM in full as a pinned document to get the full context. Also in this approach the snippet limit should me removed because all documents that have info should be found.

Originally created by @illov on GitHub (Apr 3, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3587 ### What would you like to see? A option in the workspace: If a related snippet is found in the vector search then that document is fed to the LLM in full as a pinned document to get the full context. Also in this approach the snippet limit should me removed because all documents that have info should be found.
yindo added the enhancementfeature request labels 2026-02-22 18:29:09 -05:00
yindo closed this issue 2026-02-22 18:29:09 -05:00
yindo commented 2026-02-22 18:29:09 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 3, 2025):

This would blow up almost any persons local LLM use (or even cloud) since you could go from n tokens with y context, but the document y comes from is 100K tokens your next request would be n + 100K suddenly.

For cloud users -> This would inflate the per query costs to something insane for long-turn chats.
For Local -> This would spike all resources and result in a full lockup or a crash since we would be max-context filling.

This also ignores how we do pruning to make sure context works - so now you will lose history, system prompt content, and document snippets.

For this, turn on re-ranking. It is the closest you can get to full-context without blowing up your LLM costs or your server running the LLM. The nuance to enable this is too complex for a toggle for a user - so that is why we show a big warning model when you pin a document for the first time. Pinning should always be a last resort or a tool for people who know what they are doing.

@timothycarambat commented on GitHub (Apr 3, 2025): This would blow up almost any persons local LLM use (or even cloud) since you could go from `n` tokens with `y` context, but the document `y` comes from is `100K tokens` your next request would be `n + 100K` suddenly. For cloud users -> This would inflate the per query costs to something insane for long-turn chats. For Local -> This would spike all resources and result in a full lockup or a crash since we would be max-context filling. This also ignores how we do pruning to make sure context works - so now you will lose history, system prompt content, **and** document snippets. For this, turn on re-ranking. It is the closest you can get to full-context without blowing up your LLM costs or your server running the LLM. The nuance to enable this is too complex for a toggle for a user - so that is why we show a big warning model when you pin a document for the first time. Pinning should always be a last resort or a tool for people who know what they are doing.
yindo changed title from [FEAT]: Automatically pin documents that have been found based on vector search for improved RAG to [GH-ISSUE #3587] [FEAT]: Automatically pin documents that have been found based on vector search for improved RAG 2026-06-05 14:45:52 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2319
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?