Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1630] [BUG]: LLM Preferences credentials and model list is populated with the Embedder Preferences. #1060

New Issue
Closed
opened 2026-02-22 18:22:56 -05:00 by yindo · 4 comments
yindo commented 2026-02-22 18:22:56 -05:00
Owner
Copy Link
Copy Source

Originally created by @avinashkurup on GitHub (Jun 7, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1630

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

As per my understanding the Embedder preferences are to be included when the LLM model does not do embedding by default. I am providing both of the preferences, i.e. for LLM as well as embedder preferences, but I notice that the former is being over-written by the latter.
Screenshot from 2024-06-07 10-42-26
LLM preference is over-written with the Embedder model that I had updated in the Embedder preferences

Screenshot from 2024-06-07 10-46-03
Embedded the freshly uploaded document with the provided Embedder model.

Screenshot from 2024-06-07 10-47-41
Faced the above error.

Are there known steps to reproduce?

  1. Set LLM preferences in Settings.
  2. Set Embedder preferences in Settings.
  3. Checked LLM preferences but it was over-written by Embedder preferences.
  4. On prompting faced this error in the screenshot.
Originally created by @avinashkurup on GitHub (Jun 7, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1630 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? As per my understanding the Embedder preferences are to be included when the LLM model does not do embedding by default. I am providing both of the preferences, i.e. for LLM as well as embedder preferences, but I notice that the former is being over-written by the latter. ![Screenshot from 2024-06-07 10-42-26](https://github.com/Mintplex-Labs/anything-llm/assets/681657/c4cb3d7a-04fb-4bd0-9230-c7e327573c05) LLM preference is over-written with the Embedder model that I had updated in the Embedder preferences ![Screenshot from 2024-06-07 10-46-03](https://github.com/Mintplex-Labs/anything-llm/assets/681657/96ae70a8-770c-4bcc-9f13-d42339b21153) Embedded the freshly uploaded document with the provided Embedder model. ![Screenshot from 2024-06-07 10-47-41](https://github.com/Mintplex-Labs/anything-llm/assets/681657/1638854b-bc98-485d-b114-33fa9d6c2e3b) Faced the above error. ### Are there known steps to reproduce? 1. Set LLM preferences in Settings. 2. Set Embedder preferences in Settings. 3. Checked LLM preferences but it was over-written by Embedder preferences. 4. On prompting faced this error in the screenshot.
yindo added the possible buginvestigating labels 2026-02-22 18:22:57 -05:00
yindo closed this issue 2026-02-22 18:22:57 -05:00
yindo commented 2026-02-22 18:22:57 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 7, 2024):

@avinashkurup I think there is a misunderstanding.
LLM and Embedders are totally different and are not interchangeable

If you want to use LiteLLM/Ollama for your embedder go to Embedder preferences and use LiteLLM and use an embedding model like ollama-nomic-embed-text.

Then in LLM preferences, you should set the model preference to be a chat model like llama3 8B.

You cannot use an embedding model for your LLM or a chat model for your embedder. They are not interchangeable. The use of any LLM has no determination on the embedder used, they are set up separately.

@timothycarambat commented on GitHub (Jun 7, 2024): @avinashkurup I think there is a misunderstanding. LLM and Embedders are **totally different and are not interchangeable** If you want to use LiteLLM/Ollama for your embedder go to `Embedder preferences` and use LiteLLM and use an **embedding model** like `ollama-nomic-embed-text`. Then in LLM preferences, you should set the model preference to be a **chat model** like `llama3 8B`. You cannot use an embedding model for your LLM or a chat model for your embedder. They are not interchangeable. The use of any LLM has no determination on the embedder used, they are set up separately.
yindo commented 2026-02-22 18:22:57 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 7, 2024):

Its also worth noting that LiteLLM (or any provider for that fact) differentiates embedding models from chat models in the /models endpoint. So it is up to the user to know the difference - which I think it all that occurred here.

@timothycarambat commented on GitHub (Jun 7, 2024): Its also worth noting that LiteLLM (or any provider for that fact) differentiates **embedding** models from **chat** models in the `/models` endpoint. So it is up to the user to know the difference - which I think it all that occurred here.
yindo commented 2026-02-22 18:22:57 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Jun 7, 2024):

I just confirmed that selecting LiteLLM and saving the LLM model and then changing the embedder preference to another LiteLLM model does not cause it to overwrite the LLM model. Just be sure to use only chat models for the LLM and only embedding models for the embedder preference since we cannot differentiate the models that come from LiteLLM.

@shatfield4 commented on GitHub (Jun 7, 2024): I just confirmed that selecting LiteLLM and saving the LLM model and then changing the embedder preference to another LiteLLM model does not cause it to overwrite the LLM model. Just be sure to use only chat models for the LLM and only embedding models for the embedder preference since we cannot differentiate the models that come from LiteLLM.
yindo commented 2026-02-22 18:22:57 -05:00
Author
Owner
Copy Link
Copy Source

@avinashkurup commented on GitHub (Jun 12, 2024):

Thank you @timothycarambat and @shatfield4, The issue was that we were using different models in LiteLLM for embedding and chat, which resulted in generating separate keys for each model. These keys were then provided to anythingLLM, causing the problem.

To resolve this, we consolidated the different models under a single unified API key. This single key was then provided to anythingLLM, which fixed the issue.

@avinashkurup commented on GitHub (Jun 12, 2024): Thank you @timothycarambat and @shatfield4, The issue was that we were using different models in LiteLLM for embedding and chat, which resulted in generating separate keys for each model. These keys were then provided to anythingLLM, causing the problem. To resolve this, we consolidated the different models under a single unified API key. This single key was then provided to anythingLLM, which fixed the issue.
yindo changed title from [BUG]: LLM Preferences credentials and model list is populated with the Embedder Preferences. to [GH-ISSUE #1630] [BUG]: LLM Preferences credentials and model list is populated with the Embedder Preferences. 2026-06-05 14:38:46 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1060
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?