Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3958] [BUG]: Weaviate embeddings persist in vector DB after removal #2519

New Issue
Closed
opened 2026-02-22 18:30:02 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:30:02 -05:00
Owner
Copy Link
Copy Source

Originally created by @Sarmingsteiner on GitHub (Jun 5, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3958

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

I’ve noticed two related issues when using AnythingLLM with a Weaviate vector store backend:

Deleted documents remain in the vector store

After removing a document from a workspace (and even deleting it entirely via the UI), its embeddings are still active and continue to surface in retrieval/citations.

Expected behavior: Deleting a document from a workspace (or permanently deleting it) should also hide/remove its embeddings from the vector store so it can no longer appear in retrieval results or citation annotations.

Cached embeddings are re-embedded in another workspace

When I embed a document once (and it is tagged as “cached”), then use that same file in a different workspace, Anything LLM re-embeds it from scratch instead of reusing the existing cached embedding.

Expected behavior: If a document’s embedding already exists in the cache (regardless of workspace), AnythingLLM should detect and reuse it rather than re-embedding, saving compute and avoiding duplicates in the vector store.

Both issues lead to stale data being returned and unnecessary re-processing, especially noticeable in multi-workspace setups.

Are there known steps to reproduce?

Configure Anything LLM to use Weaviate as the vector store (e.g., via VECTOR_DB=weaviate in .env).

a. Upload or embed Document X.
b. Confirm that Document X appears in the citation annotation results.
c. Delete Document X from Workspace A via the UI (Remove from workspace → Delete permanently).
d. Perform a retrieval query (e.g., use a prompt that would retrieve Document X).
→ Observe that Document X’s embedding is still returned in the citations.

a. Add an (already embedded) Document to the new Workspace B.
b. Check the logs or network calls to see that AnythingLLM is performing a fresh embed request, even though it was previously “cached” in Workspace A.
c. Query retrievals and observe that there are now duplicate embeddings for Document X.

Originally created by @Sarmingsteiner on GitHub (Jun 5, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3958 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? I’ve noticed two related issues when using AnythingLLM with a Weaviate vector store backend: **Deleted documents remain in the vector store** After removing a document from a workspace (and even deleting it entirely via the UI), its embeddings are still active and continue to surface in retrieval/citations. Expected behavior: Deleting a document from a workspace (or permanently deleting it) should also hide/remove its embeddings from the vector store so it can no longer appear in retrieval results or citation annotations. **Cached embeddings are re-embedded in another workspace** When I embed a document once (and it is tagged as “cached”), then use that same file in a different workspace, Anything LLM re-embeds it from scratch instead of reusing the existing cached embedding. Expected behavior: If a document’s embedding already exists in the cache (regardless of workspace), AnythingLLM should detect and reuse it rather than re-embedding, saving compute and avoiding duplicates in the vector store. Both issues lead to stale data being returned and unnecessary re-processing, especially noticeable in multi-workspace setups. ### Are there known steps to reproduce? Configure Anything LLM to use Weaviate as the vector store (e.g., via VECTOR_DB=weaviate in .env). 1) a. Upload or embed Document X. b. Confirm that Document X appears in the citation annotation results. c. Delete Document X from Workspace A via the UI (Remove from workspace → Delete permanently). d. Perform a retrieval query (e.g., use a prompt that would retrieve Document X). → Observe that Document X’s embedding is still returned in the citations. 2) a. Add an (already embedded) Document to the new Workspace B. b. Check the logs or network calls to see that AnythingLLM is performing a fresh embed request, even though it was previously “cached” in Workspace A. c. Query retrievals and observe that there are now duplicate embeddings for Document X.
yindo added the possible buginvestigating labels 2026-02-22 18:30:02 -05:00
yindo closed this issue 2026-02-22 18:30:02 -05:00
yindo commented 2026-02-22 18:30:02 -05:00
Author
Owner
Copy Link
Copy Source

@anytrace-ai-support-engineer-yc-s25 commented on GitHub (Jun 29, 2025):

Summary of the issue

Two issues are mentioned:
Issue 1: Deleted documents remain in the vector store
Issue 2: Cached embeddings are re-embedded in another workspace

Verified steps

Issue 1

I'm unable to reproduce this issue; following the steps, the embeddings were correctly deleted upon document deletion via UI.

Issue 2

I was able to reproduce this issue following the steps. This is due to an inverted skipCache condition in server/utils/vectorDbProviders/weaviate/index.js.

Proposed fix:

// server/utils/vectorDbProviders/weaviate/index.js
       193          if (!pageContent || pageContent.length == 0) return false;
       194    
       195          console.log("Adding new vectorized document into namespace", namespace);
       196 -        if (skipCache) {
       196 +        if (!skipCache) {
       197            const cacheResult = await cachedVectorInformation(fullFilePath);
       198            if (cacheResult.exists) {
       199              const { client } = await this.connect();

I was able to validate this fix locally. With this fix, the new workspace is able to use the cached document from an existing workspace:

[backend] info: [Event Logged] - workspace_created
[backend] info: [DOCUMENT DEBUG] Starting removal of 0 documents from workspace: workspace_b
[backend] info: Adding new vectorized document into namespace workspace_b
[backend] info: Cached vectorized results of custom-documents/anytrace_dummy_document.pdf-aae0e82b-0ce1-4b63-a7ec-6a8691b5ff83.json found! Using cached data to save on embed costs.
[backend] info: [TELEMETRY SENT] {"event":"documents_embedded_in_workspace","distinctId":"80966f5b-06c4-473c-a38b-7644cdb6a41d","properties":{"LLMSelection":"openai","Embedder":"native","VectorDbSelection":"weaviate","TTSSelection":"native","LLMModel":"gpt-4o","runtime":"docker"}}

Next steps for @Sarmingsteiner

Issue 1

Please retry. If you continue to see the issue, plea provide the exact local setup and steps to reproduce.

Issue 2

None

Next steps for Multiplex-Labs

Issue 1

TBD

Issue 2

Fix the bug blocking the use of cached documents. Consider the fix:

// server/utils/vectorDbProviders/weaviate/index.js
       193          if (!pageContent || pageContent.length == 0) return false;
       194    
       195          console.log("Adding new vectorized document into namespace", namespace);
       196 -        if (skipCache) {
       196 +        if (!skipCache) {
       197            const cacheResult = await cachedVectorInformation(fullFilePath);
       198            if (cacheResult.exists) {
       199              const { client } = await this.connect();

@timothycarambat @Mintplex-Labs

If you'd like Anytrace (YC S25) to keep investigating your support tickets, reach out to taira@anytrace.ai to discuss how we can keep supporting your team!

@anytrace-ai-support-engineer-yc-s25 commented on GitHub (Jun 29, 2025): ### Summary of the issue Two issues are mentioned: **Issue 1**: Deleted documents remain in the vector store **Issue 2**: Cached embeddings are re-embedded in another workspace ### Verified steps #### Issue 1 I'm unable to reproduce this issue; following the steps, the embeddings were correctly deleted upon document deletion via UI. #### Issue 2 I was able to reproduce this issue following the steps. This is due to [an inverted `skipCache` condition in `server/utils/vectorDbProviders/weaviate/index.js`](https://github.com/Mintplex-Labs/anything-llm/blob/91e498229cc6893744ae718df3479cd742c0e2e3/server/utils/vectorDbProviders/weaviate/index.js#L196-L247). **Proposed fix:** ``` // server/utils/vectorDbProviders/weaviate/index.js 193 if (!pageContent || pageContent.length == 0) return false; 194 195 console.log("Adding new vectorized document into namespace", namespace); 196 - if (skipCache) { 196 + if (!skipCache) { 197 const cacheResult = await cachedVectorInformation(fullFilePath); 198 if (cacheResult.exists) { 199 const { client } = await this.connect(); ``` I was able to validate this fix locally. With this fix, the new workspace is able to use the cached document from an existing workspace: ``` [backend] info: [Event Logged] - workspace_created [backend] info: [DOCUMENT DEBUG] Starting removal of 0 documents from workspace: workspace_b [backend] info: Adding new vectorized document into namespace workspace_b [backend] info: Cached vectorized results of custom-documents/anytrace_dummy_document.pdf-aae0e82b-0ce1-4b63-a7ec-6a8691b5ff83.json found! Using cached data to save on embed costs. [backend] info: [TELEMETRY SENT] {"event":"documents_embedded_in_workspace","distinctId":"80966f5b-06c4-473c-a38b-7644cdb6a41d","properties":{"LLMSelection":"openai","Embedder":"native","VectorDbSelection":"weaviate","TTSSelection":"native","LLMModel":"gpt-4o","runtime":"docker"}} ``` ### Next steps for @Sarmingsteiner #### Issue 1 Please retry. If you continue to see the issue, plea provide the exact local setup and steps to reproduce. #### Issue 2 None ### Next steps for Multiplex-Labs #### Issue 1 TBD #### Issue 2 Fix the bug blocking the use of cached documents. Consider the fix: ``` // server/utils/vectorDbProviders/weaviate/index.js 193 if (!pageContent || pageContent.length == 0) return false; 194 195 console.log("Adding new vectorized document into namespace", namespace); 196 - if (skipCache) { 196 + if (!skipCache) { 197 const cacheResult = await cachedVectorInformation(fullFilePath); 198 if (cacheResult.exists) { 199 const { client } = await this.connect(); ``` ### @timothycarambat @Mintplex-Labs If you'd like Anytrace (YC S25) to keep investigating your support tickets, reach out to taira@anytrace.ai to discuss how we can keep supporting your team!
yindo changed title from [BUG]: Weaviate embeddings persist in vector DB after removal to [GH-ISSUE #3958] [BUG]: Weaviate embeddings persist in vector DB after removal 2026-06-05 14:47:02 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2519
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?