Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1867] [FEAT]: Can we set embedding model for each workspace? #1215

New Issue
Closed
opened 2026-02-22 18:23:44 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:23:44 -05:00
Owner
Copy Link
Copy Source

Originally created by @yanwun on GitHub (Jul 15, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1867

What would you like to see?

Because sometimes have language problem in each workspace, so we need to change the embedding model to support this, but from now on, we only can adjust the embedding for entrie system workspace.
I really hope that it will happend.

Originally created by @yanwun on GitHub (Jul 15, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1867 ### What would you like to see? Because sometimes have language problem in each workspace, so we need to change the embedding model to support this, but from now on, we only can adjust the embedding for entrie system workspace. I really hope that it will happend.
yindo added the enhancementwontfixfeature request labels 2026-02-22 18:23:44 -05:00
yindo closed this issue 2026-02-22 18:23:44 -05:00
yindo commented 2026-02-22 18:23:44 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 15, 2024):

Unfortunately, We won't support this for a couple of reasons - mostly because of headaches that will inevitably occur if we permit this.

  • When you embed a doc, we cache its embeddings so they can be reused without hitting the embedder again. This makes all subsequent upserts very fast, if we enable people to change embedders you can have 5 docs, all cached at different dimensions - upserts will fail and it will be very frustrating

  • The alternative is to disable caching, which now makes the system very slow. Additionally, you could have a workspace with documents, swap embedders, now you cannot even query since your prompt vectorization is a different dimension from the workspace - again frustrating.

We already have people swap the embedder system-wide and get this issue on existing workspaces or cached documents and its a point of frustration. If you want to swap embedders you should basically be resetting every file and vector db attached to AnythingLLM or else these issues will occur.

And then of course, most vector dbs get their schemas from the first embed they see. If you removed all documents from a vector db, swap embedders, and try to embed a fresh new document it will still fail because the vector db only expects xyz dimensions and you are using a new model now.

So hopefully you can see now why we dont allow flexibility with embedders as we do with LLMs - swapping embedders quickly is basically the quickest way to make your AnythingLLM instance unusable without a total reset. :/

Open to discussion of the topic if the use case is strong enough for flexible selection of embedders, but it will open the door to many more self-inflicted issues and that is not something we particularly are excited to support

@timothycarambat commented on GitHub (Jul 15, 2024): Unfortunately, We won't support this for a couple of reasons - mostly because of headaches that will inevitably occur if we permit this. - When you embed a doc, we cache its embeddings so they can be reused without hitting the embedder again. This makes all subsequent upserts very fast, if we enable people to change embedders you can have 5 docs, all cached at different dimensions - upserts will fail and it will be very frustrating - The alternative is to disable caching, which now makes the system very slow. Additionally, you could have a workspace with documents, swap embedders, now you cannot even _query_ since your prompt vectorization is a different dimension from the workspace - again frustrating. We already have people swap the embedder system-wide and get this issue on existing workspaces or cached documents and its a point of frustration. If you want to swap embedders you should basically be resetting every file and vector db attached to AnythingLLM or else these issues will occur. And then of course, most vector dbs get their schemas from the first embed they see. If you removed all documents from a vector db, swap embedders, and try to embed a fresh new document it will **still fail** because the vector db only expects xyz dimensions and you are using a new model now. So hopefully you can see now why we dont allow flexibility with embedders as we do with LLMs - swapping embedders quickly is basically the quickest way to make your AnythingLLM instance unusable without a total reset. :/ Open to discussion of the topic if the use case is strong enough for flexible selection of embedders, but it will open the door to many more self-inflicted issues and that is not something we particularly are excited to support
yindo commented 2026-02-22 18:23:44 -05:00
Author
Owner
Copy Link
Copy Source

@yanwun commented on GitHub (Jul 16, 2024):

Maybe can try that cache the embedded files in the workspace level, and when the embedding model change just warning users it will remove the cache and reset the embedding.
I can not understand that why this happend? Sorry, I am newer to this field.
"And then of course, most vector dbs get their schemas from the first embed they see. If you removed all documents from a vector db, swap embedders, and try to embed a fresh new document it will still fail because the vector db only expects xyz dimensions and you are using a new model now."
Because I just think that when delete the embedding in the system it does not mean that the embedding reset.
And the system will be slow at the embedding? or in streamining chat? because the embedding for each worksapce is independent, if in embedding, I think user will accept it.

Thanks for answering the request.

@yanwun commented on GitHub (Jul 16, 2024): Maybe can try that cache the embedded files in the workspace level, and when the embedding model change just warning users it will remove the cache and reset the embedding. I can not understand that why this happend? Sorry, I am newer to this field. "And then of course, most vector dbs get their schemas from the first embed they see. If you removed all documents from a vector db, swap embedders, and try to embed a fresh new document it will still fail because the vector db only expects xyz dimensions and you are using a new model now." Because I just think that when delete the embedding in the system it does not mean that the embedding reset. And the system will be slow at the embedding? or in streamining chat? because the embedding for each worksapce is independent, if in embedding, I think user will accept it. Thanks for answering the request.
yindo commented 2026-02-22 18:23:44 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 16, 2024):

Maybe can try that cache the embedded files in the workspace level, and when the embedding model change just warning users it will remove the cache and reset the embedding.

We do this! But at the system level. if you swap embedders we warn you of these potential damages and do force reset everything. So that part is at least covered.

Because I just think that when delete the embedding in the system it does not mean that the embedding reset.

Yes, correct, we still have the cache.

And the system will be slow at the embedding?

Exactly, because the embedder will need to run again on the text chunk that was previous embedded. Caching improves this speed a lot.

I think user will accept it.

The reason it exists is because early early one users hated how long it took with large documents. Simple 20 line text file was fast, but a multi-page PDF was so slow and it was slow every time it needed to be uploaded and embedded into a workspace. That is why we did caching in the first place :)

@timothycarambat commented on GitHub (Jul 16, 2024): > Maybe can try that cache the embedded files in the workspace level, and when the embedding model change just warning users it will remove the cache and reset the embedding. We do this! But at the system level. if you swap embedders we warn you of these potential damages and do force reset everything. So that part is at least covered. > Because I just think that when delete the embedding in the system it does not mean that the embedding reset. Yes, correct, we still have the cache. > And the system will be slow at the embedding? Exactly, because the embedder will need to run again on the text chunk that was previous embedded. Caching improves this speed a lot. > I think user will accept it. The reason it exists is because early early one users _hated_ how long it took with large documents. Simple 20 line text file was fast, but a multi-page PDF was so slow and it was slow _every_ time it needed to be uploaded and embedded into a workspace. That is why we did caching in the first place :)
yindo changed title from [FEAT]: Can we set embedding model for each workspace? to [GH-ISSUE #1867] [FEAT]: Can we set embedding model for each workspace? 2026-06-05 14:39:34 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request wontfix
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1215
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?