Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1240] [CHORE]: Improve RAG retention via history context #776

New Issue
Closed
opened 2026-02-22 18:21:18 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:21:18 -05:00
Owner
Copy Link
Copy Source

Originally created by @timothycarambat on GitHub (Apr 30, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1240

Originally assigned to: @timothycarambat on GitHub.

What would you like to see?

When we send a chat like What is AnythingLLM the system will find relevant context, if any and append it under the system prompt.

{SYSTEM}
<context>
<context>
# No history
<user prompt>

However, on follow up request, where we may not get a vector search result due to ambiguity like tell me some features
We will get a flow like

{SYSTEM}
# NO CONTEXT!
<reply from prompt 1 from LLM>
<user prompt>

This puts 100% of the response result on the <reply from prompt 1 from LLM> response where we could leverage the previous citations in the history to help offload the responsibility and scope of the answering.

Originally created by @timothycarambat on GitHub (Apr 30, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1240 Originally assigned to: @timothycarambat on GitHub. ### What would you like to see? When we send a chat like `What is AnythingLLM` the system will find relevant context, if any and append it under the system prompt. ```shell {SYSTEM} <context> <context> # No history <user prompt> ``` However, on follow up request, where we may not get a vector search result due to ambiguity like `tell me some features` We will get a flow like ```shell {SYSTEM} # NO CONTEXT! <reply from prompt 1 from LLM> <user prompt> ``` This puts 100% of the response result on the `<reply from prompt 1 from LLM>` response where we could leverage the previous citations in the history to help offload the responsibility and scope of the answering.
yindo added the enhancementcore-team-onlyfeature request labels 2026-02-22 18:21:18 -05:00
yindo closed this issue 2026-02-22 18:21:18 -05:00
yindo commented 2026-02-22 18:21:18 -05:00
Author
Owner
Copy Link
Copy Source

@Propheticus commented on GitHub (May 23, 2024):

This would be a good enhancement. Instead of having to rely on condensed pieces of second hand information, allowing it to access the same full source citations found earlier would reduce the tendency to hallucinate or guess from general knowledge.

Now the implementation questions are:

  • When will you add previous chunks? Only when no chunks where found at all or will you also backfill chunks when the # retrieved is below the max # set.
  • How many messages back to look for previous chunks? (only most recent message with chunks VS as far back as needed to fill till max # is reached)
@Propheticus commented on GitHub (May 23, 2024): This would be a good enhancement. Instead of having to rely on condensed pieces of second hand information, allowing it to access the same full source citations found earlier would reduce the tendency to hallucinate or guess from general knowledge. Now the implementation questions are: - When will you add previous chunks? Only when no chunks where found at all or will you also backfill chunks when the # retrieved is below the max # set. - How many messages back to look for previous chunks? (only most recent message with chunks VS as far back as needed to fill till max # is reached)
yindo commented 2026-02-22 18:21:18 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 23, 2024):

When will you add previous chunks? Only when no chunks where found at all or will you also backfill chunks when the # retrieved is below the max # set.

Backfill when window is not full at all times, even when empty for the prompt itself since there is no semantic way to determine if it should or should not be appended. If its irrelevant the model should emit response tokens to simply not include references to citations that wont help generate the answer. If we omit appending if the inital query is empty we are pretty much back where we started!

How many messages back to look for previous chunks? (only most recent message with chunks VS as far back as needed to fill till max # is reached)

As far back as needed to backfill with unqiue sources. However the chat history itself only goes back so many messages, so its not like each request will loop up a 1,000 chat long session. By default the appended LLM history is 20 message pairs

@timothycarambat commented on GitHub (May 23, 2024): > When will you add previous chunks? Only when no chunks where found at all or will you also backfill chunks when the # retrieved is below the max # set. Backfill when window is not full at all times, even when empty for the prompt itself since there is no semantic way to determine if it should or should not be appended. If its irrelevant the model should emit response tokens to simply not include references to citations that wont help generate the answer. If we omit appending if the inital query is empty we are pretty much back where we started! > How many messages back to look for previous chunks? (only most recent message with chunks VS as far back as needed to fill till max # is reached) As far back as needed to backfill with unqiue sources. However the chat history itself only goes back so many messages, so its not like each request will loop up a 1,000 chat long session. By default the appended LLM history is 20 message pairs
yindo commented 2026-02-22 18:21:19 -05:00
Author
Owner
Copy Link
Copy Source

@Propheticus commented on GitHub (May 23, 2024):

Agree on both.

@Propheticus commented on GitHub (May 23, 2024): Agree on both.
yindo changed title from [CHORE]: Improve RAG retention via history context to [GH-ISSUE #1240] [CHORE]: Improve RAG retention via history context 2026-06-05 14:37:08 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label core-team-only enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#776
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?