Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1748] [BUG]: Data Syncing in Pinecone Vector Database Service #1139

New Issue
Closed
opened 2026-02-22 18:23:19 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:23:19 -05:00
Owner
Copy Link
Copy Source

Originally created by @chalitbkb on GitHub (Jun 23, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1748

How are you running AnythingLLM?

All versions

What happened?

Here are the questions:

Question One:

Why doesn't the vector database service like "pinecone" sync data? For instance, let's say I use "anything" on the desktop version across two computers and connect it to the pinecone API. When I upload data to pinecone from computer one, which already has the vector database information, computer two also connects to the same API but doesn't upload any data. The question is, why do I need to upload the same data again when it's already available and I want to use it immediately on the second computer?

image

Is it possible to have an option to use the same database to reduce time and complexity instead of uploading new data? Sometimes, I just want to fetch from the existing database.

Question Two:

When comparing "LanceDB" and "Pinecone," is there a difference in performance when it comes to data search efficiency?

Are there known steps to reproduce?

No response

Originally created by @chalitbkb on GitHub (Jun 23, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1748 ### How are you running AnythingLLM? All versions ### What happened? Here are the questions: Question One: Why doesn't the vector database service like "pinecone" sync data? For instance, let's say I use "anything" on the desktop version across two computers and connect it to the pinecone API. When I upload data to pinecone from computer one, which already has the vector database information, computer two also connects to the same API but doesn't upload any data. The question is, why do I need to upload the same data again when it's already available and I want to use it immediately on the second computer? ![image](https://github.com/Mintplex-Labs/anything-llm/assets/32437546/5d16c2fe-4564-4f03-a088-aae54e9cd9ae) Is it possible to have an option to use the same database to reduce time and complexity instead of uploading new data? Sometimes, I just want to fetch from the existing database. Question Two: When comparing "LanceDB" and "Pinecone," is there a difference in performance when it comes to data search efficiency? ### Are there known steps to reproduce? _No response_
yindo added the possible bug label 2026-02-22 18:23:19 -05:00
yindo closed this issue 2026-02-22 18:23:19 -05:00
yindo commented 2026-02-22 18:23:20 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 23, 2024):

Is it possible to have an option to use the same database to reduce time and complexity instead of uploading new data? Sometimes, I just want to fetch from the existing database.

You can connect to the same DB, but AnythingLLM cannot currently import embeddings because we have no idea on the metadata schema that was used and since embeddings are basically ad-hoc there is no determination how data was uploaded. Obviously we could put some special tag on all AnythingLLM uploaded data so we know it conforms, but if your main goal is to share an embedding workspace across devices - that is what the Docker multi-user version is for.

When comparing "LanceDB" and "Pinecone," is there a difference in performance when it comes to data search efficiency?

In a word, No. The search efficiency/comparisons of vectors vs a collection is equally performant for basically all of the DB providers 👍. We offer a multitude of providers because people have preferences and LanceDB is not portable outside the application/instance of AnythingLLM

@timothycarambat commented on GitHub (Jun 23, 2024): > Is it possible to have an option to use the same database to reduce time and complexity instead of uploading new data? Sometimes, I just want to fetch from the existing database. You can connect to the same DB, but AnythingLLM cannot currently import embeddings because we have no idea on the metadata schema that was used and since embeddings are basically ad-hoc there is no determination how data was uploaded. Obviously we _could_ put some special tag on all AnythingLLM uploaded data so we know it conforms, but if your main goal is to share an embedding workspace across devices - that is what the Docker multi-user version is for. > When comparing "LanceDB" and "Pinecone," is there a difference in performance when it comes to data search efficiency? In a word, No. The search efficiency/comparisons of vectors vs a collection is equally performant for basically all of the DB providers 👍. We offer a multitude of providers because people have preferences and LanceDB is not portable outside the application/instance of AnythingLLM
yindo changed title from [BUG]: Data Syncing in Pinecone Vector Database Service to [GH-ISSUE #1748] [BUG]: Data Syncing in Pinecone Vector Database Service 2026-06-05 14:39:10 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1139
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?