Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #988] [BUG]: addDocumentToNamespace LanceDBError: Append with different schema #604

New Issue
Closed
opened 2026-02-22 18:20:25 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:20:25 -05:00
Owner
Copy Link
Copy Source

Originally created by @majestichou on GitHub (Mar 28, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/988

How are you running AnythingLLM?

Docker (local)

What happened?

I use the sfr-embedding-mistral:q4_k_m embedding model and LanceDB. I uploaded a document to the workspace and clicked the "Save and Embed" button. The error occurred.
[OllamaEmbedder] Embedding 2 chunks of text with sfr-embedding-mistral:q4_k_m.
Inserting vectorized chunks into LanceDB collection.

addDocumentToNamespace LanceDBError: Append with different schema: original=Field(id=0, name=vector, type=fixed_size_list:float:384)
Field(id=1, name=id, type=string)
Field(id=2, name=url, type=string)
Field(id=3, name=title, type=string)
Field(id=4, name=docAuthor, type=string)
Field(id=5, name=description, type=string)
Field(id=6, name=docSource, type=string)
Field(id=7, name=chunkSource, type=string)
Field(id=8, name=published, type=string)
Field(id=9, name=wordCount, type=double)
Field(id=10, name=token_count_estimate, type=double)
Field(id=11, name=text, type=string)
 new=Field(id=0, name=vector, type=fixed_size_list:float:4096)
Field(id=1, name=id, type=string)
Field(id=2, name=url, type=string)
Field(id=3, name=title, type=string)
Field(id=4, name=docAuthor, type=string)
Field(id=5, name=description, type=string)
Field(id=6, name=docSource, type=string)
Field(id=7, name=chunkSource, type=string)
Field(id=8, name=published, type=string)
Field(id=9, name=wordCount, type=double)
Field(id=10, name=token_count_estimate, type=double)
Field(id=11, name=text, type=string)

As far as I know, the output of the sfr-embedding-mistral:q4_k_m model has dimension 4096, which cannot be changed. Can the data storage dimension of the LanceDB database be changed?
Maybe changing the data storage dimension from 384 to 4096 will fix this?

Are there known steps to reproduce?

No response

Originally created by @majestichou on GitHub (Mar 28, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/988 ### How are you running AnythingLLM? Docker (local) ### What happened? I use the **`sfr-embedding-mistral:q4_k_m`** embedding model and **`LanceDB`**. I uploaded a document to the workspace and clicked the "Save and Embed" button. The error occurred. [OllamaEmbedder] Embedding 2 chunks of text with sfr-embedding-mistral:q4_k_m. Inserting vectorized chunks into LanceDB collection. ``` addDocumentToNamespace LanceDBError: Append with different schema: original=Field(id=0, name=vector, type=fixed_size_list:float:384) Field(id=1, name=id, type=string) Field(id=2, name=url, type=string) Field(id=3, name=title, type=string) Field(id=4, name=docAuthor, type=string) Field(id=5, name=description, type=string) Field(id=6, name=docSource, type=string) Field(id=7, name=chunkSource, type=string) Field(id=8, name=published, type=string) Field(id=9, name=wordCount, type=double) Field(id=10, name=token_count_estimate, type=double) Field(id=11, name=text, type=string) new=Field(id=0, name=vector, type=fixed_size_list:float:4096) Field(id=1, name=id, type=string) Field(id=2, name=url, type=string) Field(id=3, name=title, type=string) Field(id=4, name=docAuthor, type=string) Field(id=5, name=description, type=string) Field(id=6, name=docSource, type=string) Field(id=7, name=chunkSource, type=string) Field(id=8, name=published, type=string) Field(id=9, name=wordCount, type=double) Field(id=10, name=token_count_estimate, type=double) Field(id=11, name=text, type=string) ``` As far as I know, the output of the sfr-embedding-mistral:q4_k_m model has dimension 4096, which cannot be changed. Can the data storage dimension of the LanceDB database be changed? Maybe changing the data storage dimension from 384 to 4096 will fix this? ### Are there known steps to reproduce? _No response_
yindo added the possible bug label 2026-02-22 18:20:25 -05:00
yindo closed this issue 2026-02-22 18:20:25 -05:00
yindo commented 2026-02-22 18:20:26 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 28, 2024):

You cannot create a workspace and embed documents with one embedder, unembed all documents, swap embedders, and re-embed into that same workspace. The lanceDB schema is not removed when the last document is un-embedded (maybe it should to prevent this issues).

Just make a new workspace and it will work 👍

Default embedder is 384, so that is likely what happened here

@timothycarambat commented on GitHub (Mar 28, 2024): You cannot create a workspace and embed documents with one embedder, unembed all documents, swap embedders, and re-embed into that same workspace. The lanceDB schema is not removed when the last document is un-embedded (maybe it should to prevent this issues). Just make a new workspace and it will work 👍 Default embedder is 384, so that is likely what happened here
yindo commented 2026-02-22 18:20:26 -05:00
Author
Owner
Copy Link
Copy Source

@majestichou commented on GitHub (Mar 28, 2024):

You cannot create a workspace and embed documents with one embedder, unembed all documents, swap embedders, and re-embed into that same workspace. The lanceDB schema is not removed when the last document is un-embedded (maybe it should to prevent this issues).

Just make a new workspace and it will work 👍

Default embedder is 384, so that is likely what happened here

YES！Just make a new workspace and it will work 👍 Thanks a lot! You did a great job!

@majestichou commented on GitHub (Mar 28, 2024): > You cannot create a workspace and embed documents with one embedder, unembed all documents, swap embedders, and re-embed into that same workspace. The lanceDB schema is not removed when the last document is un-embedded (maybe it should to prevent this issues). > > Just make a new workspace and it will work 👍 > > Default embedder is 384, so that is likely what happened here YES！Just make a new workspace and it will work 👍 Thanks a lot! You did a great job!
yindo changed title from [BUG]: addDocumentToNamespace LanceDBError: Append with different schema to [GH-ISSUE #988] [BUG]: addDocumentToNamespace LanceDBError: Append with different schema 2026-06-05 14:36:14 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#604
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?