Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1425] [FEAT]: Use System Folder as RAG's Knowledge Base #910

New Issue
Closed
opened 2026-02-22 18:22:10 -05:00 by yindo · 4 comments
yindo commented 2026-02-22 18:22:10 -05:00
Owner
Copy Link
Copy Source

Originally created by @shgsousa on GitHub (May 16, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1425

What would you like to see?

I’m utilizing Anything LLM as a knowledge assistant to discuss business processes and share my own insights, which I periodically record in documents. However, I’m facing a challenge keeping the system up-to-date with the evolving nature of these files, as they are not static documents.

My ideal solution would be a RAG system that enables me to:

  1. Directly access answers from a set of files within a designated folder (and possibly subfolders) on my computer. To avoid scanning, splitting, and embedding unnecessary files in case of subfolders, I’d like the ability to specify file extension filters.
  2. Automatically detect changes to the documents and rebuild the vector database at a scheduled interval, during agent use, or when deemed necessary.
  3. Ensure that the vector database update occurs in memory, in the background, without disrupting agent usage that can continue working with outdated content until the new context data becomes available. Once the new embeddings and indexing are complete. This would enable Anything LLM to reduce the time of locking the system for usage only to the time it takes to serialize changes to the vector database, and loading the new vector database to memory for the operation of the agent.

Implementing this feature would significantly enhance the suitability of Anything LLM for my intended use case. Additionally, I’m impressed with the project’s progress and intend to contribute financially from time to time.

Originally created by @shgsousa on GitHub (May 16, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1425 ### What would you like to see? I’m utilizing Anything LLM as a knowledge assistant to discuss business processes and share my own insights, which I periodically record in documents. However, I’m facing a challenge keeping the system up-to-date with the evolving nature of these files, as they are not static documents. My ideal solution would be a RAG system that enables me to: 1. Directly access answers from a set of files within a designated folder (and possibly subfolders) on my computer. To avoid scanning, splitting, and embedding unnecessary files in case of subfolders, I’d like the ability to specify file extension filters. 2. Automatically detect changes to the documents and rebuild the vector database at a scheduled interval, during agent use, or when deemed necessary. 3. Ensure that the vector database update occurs in memory, in the background, without disrupting agent usage that can continue working with outdated content until the new context data becomes available. Once the new embeddings and indexing are complete. This would enable Anything LLM to reduce the time of locking the system for usage only to the time it takes to serialize changes to the vector database, and loading the new vector database to memory for the operation of the agent. Implementing this feature would significantly enhance the suitability of Anything LLM for my intended use case. Additionally, I’m impressed with the project’s progress and intend to contribute financially from time to time.
yindo added the enhancementfeature request labels 2026-02-22 18:22:10 -05:00
yindo commented 2026-02-22 18:22:10 -05:00
Author
Owner
Copy Link
Copy Source

@lctuck commented on GitHub (Jul 10, 2024):

This is right on target. Adding, tracking, updating files is a real issue. Pointing to a folder(s)/subfolder(s) would absolutely be a next level enhancement for this wonderful piece of tech.

@lctuck commented on GitHub (Jul 10, 2024): This is right on target. Adding, tracking, updating files is a real issue. Pointing to a folder(s)/subfolder(s) would absolutely be a next level enhancement for this wonderful piece of tech.
yindo commented 2026-02-22 18:22:10 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 10, 2024):

If anyone is watching this, live file sync for remote files (web, Github, etc) is already "live" in beta https://docs.useanything.com/beta-preview/active-features/live-document-sync and the soon to be merged branch desktop-local-file-sync will expand that to any locally upload file for desktop only.

Permissions disable us from being able to do this with local files for self-hosting and obviously remote hosted instances. So local file sync will only ever be on desktop

@timothycarambat commented on GitHub (Jul 10, 2024): If anyone is watching this, live file sync for remote files (web, Github, etc) is already "live" in beta https://docs.useanything.com/beta-preview/active-features/live-document-sync and the soon to be merged branch [desktop-local-file-sync](https://github.com/Mintplex-Labs/anything-llm/tree/desktop-local-file-sync) will expand that to any locally upload file for desktop only. Permissions disable us from being able to do this with local files for self-hosting and obviously remote hosted instances. So local file sync will only ever be on desktop
yindo commented 2026-02-22 18:22:10 -05:00
Author
Owner
Copy Link
Copy Source

@tino commented on GitHub (Sep 7, 2024):

The mentioned branch doesn't exist anymore, however I don't yet see this in v1.6.4? Or am I missing something?

@tino commented on GitHub (Sep 7, 2024): The mentioned branch doesn't exist anymore, however I don't yet see this in v1.6.4? Or am I missing something?
yindo commented 2026-02-22 18:22:10 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Sep 9, 2024):

It is in the app under experimental:
https://docs.anythingllm.com/beta-preview/enable-feature

Then you can enable it. It is under there because without a doubt someone will try to proactively track 10K files and wonder why it uses so many resources & that is currently out of scope for its current support.

https://docs.anythingllm.com/beta-preview/active-features/live-document-sync

@timothycarambat commented on GitHub (Sep 9, 2024): It is in the app under experimental: https://docs.anythingllm.com/beta-preview/enable-feature Then you can enable it. It is under there because without a doubt someone will try to proactively track 10K files and wonder why it uses so many resources & that is currently out of scope for its current support. https://docs.anythingllm.com/beta-preview/active-features/live-document-sync
yindo changed title from [FEAT]: Use System Folder as RAG's Knowledge Base to [GH-ISSUE #1425] [FEAT]: Use System Folder as RAG's Knowledge Base 2026-06-05 14:37:53 -04:00
yindo closed this issue 2026-06-05 14:37:54 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#910
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?