Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4265] [FEAT]: LEANN Vector DB as integrated option #2718

New Issue
Open
opened 2026-02-22 18:30:56 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:30:56 -05:00
Owner
Copy Link
Copy Source

Originally created by @korgano on GitHub (Aug 9, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4265

What would you like to see?

LEANN is a new vector DB solution that purports to have huge improvements in DB size while maintaining performance:

LEANN is an innovative vector database that democratizes personal AI. Transform your laptop into a powerful RAG system that can index and search through millions of documents while using 97% less storage than traditional solutions without accuracy loss.

LEANN achieves this through graph-based selective recomputation with high-degree preserving pruning, computing embeddings on-demand instead of storing them all. Illustration Fig → | Paper →

LEANN: A Low-Storage Vector Index

Embedding-based search is widely used in applications such as recommendation and retrieval-augmented generation (RAG). Recently, there is a growing demand to support these capabilities over personal data stored locally on devices. However, maintaining the necessary data structure associated with the embedding-based search is often infeasible due to its high storage overhead. For example, indexing 100 GB of raw data requires 150 to 700 GB of storage, making local deployment impractical. Reducing this overhead while maintaining search quality and latency becomes a critical challenge.

In this paper, we present LEANN, a storage-efficient approximate nearest neighbor (ANN) search index optimized for resource-constrained personal devices. LEANN combines a compact graph-based structure with an efficient on-the-fly recomputation strategy to enable fast and accurate retrieval with minimal storage overhead. Our evaluation shows that LEANN reduces index size to under 5% of the original raw data, achieving up to 50 times smaller storage than standard indexes, while maintaining 90% top-3 recall in under 2 seconds on real-world question answering benchmarks.

This seems like a great solution for improving the quality of RAG with large DBs like code bases and other large data sets.

Originally created by @korgano on GitHub (Aug 9, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4265 ### What would you like to see? [LEANN](https://github.com/yichuan-w/LEANN) is a new vector DB solution that purports to have huge improvements in DB size while maintaining performance: > LEANN is an innovative vector database that democratizes personal AI. Transform your laptop into a powerful RAG system that can index and search through millions of documents while using 97% less storage than traditional solutions without accuracy loss. > > LEANN achieves this through graph-based selective recomputation with high-degree preserving pruning, computing embeddings on-demand instead of storing them all. [Illustration Fig →](https://github.com/yichuan-w/LEANN#%EF%B8%8F-architecture--how-it-works) | [Paper →](https://arxiv.org/abs/2506.08276) [LEANN: A Low-Storage Vector Index](https://arxiv.org/html/2506.08276v1) > Embedding-based search is widely used in applications such as recommendation and retrieval-augmented generation (RAG). Recently, there is a growing demand to support these capabilities over personal data stored locally on devices. However, maintaining the necessary data structure associated with the embedding-based search is often infeasible due to its high storage overhead. For example, indexing 100 GB of raw data requires 150 to 700 GB of storage, making local deployment impractical. Reducing this overhead while maintaining search quality and latency becomes a critical challenge. > > In this paper, we present LEANN, a storage-efficient approximate nearest neighbor (ANN) search index optimized for resource-constrained personal devices. LEANN combines a compact graph-based structure with an efficient on-the-fly recomputation strategy to enable fast and accurate retrieval with minimal storage overhead. Our evaluation shows that LEANN reduces index size to under 5% of the original raw data, achieving up to 50 times smaller storage than standard indexes, while maintaining 90% top-3 recall in under 2 seconds on real-world question answering benchmarks. This seems like a great solution for improving the quality of RAG with large DBs like code bases and other large data sets.
yindo added the enhancementfeature request labels 2026-02-22 18:30:56 -05:00
yindo commented 2026-02-22 18:30:56 -05:00
Author
Owner
Copy Link
Copy Source

@onestardao commented on GitHub (Aug 10, 2025):

interesting approach — LEANN’s on-demand embedding recomputation could really cut storage costs, but in a live RAG pipeline there are a couple of things to watch for (ProblemMap No.14 — vector index drift & inconsistency):

Cold-start latency — first-time queries on unseen vectors will have to wait for recomputation, so you might need caching for high-traffic queries.

Graph drift — when documents are updated frequently, ensure the graph structure and embeddings stay in sync; otherwise, retrieval accuracy can degrade without obvious signs.

Hybrid search tuning — mixing recomputed vectors with stored ones can cause score skew unless you normalize similarity metrics.

if those edge cases are handled well, this could be a solid alternative to traditional FAISS / HNSW indexes, especially on limited hardware.

@onestardao commented on GitHub (Aug 10, 2025): interesting approach — LEANN’s on-demand embedding recomputation could really cut storage costs, but in a live RAG pipeline there are a couple of things to watch for (ProblemMap No.14 — vector index drift & inconsistency): Cold-start latency — first-time queries on unseen vectors will have to wait for recomputation, so you might need caching for high-traffic queries. Graph drift — when documents are updated frequently, ensure the graph structure and embeddings stay in sync; otherwise, retrieval accuracy can degrade without obvious signs. Hybrid search tuning — mixing recomputed vectors with stored ones can cause score skew unless you normalize similarity metrics. if those edge cases are handled well, this could be a solid alternative to traditional FAISS / HNSW indexes, especially on limited hardware.
yindo changed title from [FEAT]: LEANN Vector DB as integrated option to [GH-ISSUE #4265] [FEAT]: LEANN Vector DB as integrated option 2026-06-05 14:48:08 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2718
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?