Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #698] [FEAT]: Support multiple providers at once for workspaces #400

New Issue
Closed
opened 2026-02-22 18:19:19 -05:00 by yindo · 9 comments
yindo commented 2026-02-22 18:19:19 -05:00
Owner
Copy Link
Copy Source

Originally created by @sheneman on GitHub (Feb 8, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/698

What would you like to see?

It would be great if AnythingLLM allowed the admin to configure URLs and API keys for multiple LLM providers. For example, I would love it if one workspace used OpenAI and another workspace used a Local AI endpoint for something like Mixtral 8x 7B. It seems that for any given AnythingLLM instance, there is only one preferred LLM provider and that setting applies across all workspaces.

Thank you!

Originally created by @sheneman on GitHub (Feb 8, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/698 ### What would you like to see? It would be great if AnythingLLM allowed the admin to configure URLs and API keys for multiple LLM providers. For example, I would love it if one workspace used OpenAI and another workspace used a Local AI endpoint for something like Mixtral 8x 7B. It seems that for any given AnythingLLM instance, there is only one preferred LLM provider and that setting applies across all workspaces. Thank you!
yindo added the enhancementfeature request labels 2026-02-22 18:19:19 -05:00
yindo closed this issue 2026-02-22 18:19:19 -05:00
yindo commented 2026-02-22 18:19:19 -05:00
Author
Owner
Copy Link
Copy Source

@icsy7867 commented on GitHub (Feb 13, 2024):

Was also looking into this. For my business use case, I am using ollama as the backend and I have an ollama instance running a mistral instance and another using codellama. I would love to have a workspace dedicated to our RAG chat bot and another for code generation.

@icsy7867 commented on GitHub (Feb 13, 2024): Was also looking into this. For my business use case, I am using ollama as the backend and I have an ollama instance running a mistral instance and another using codellama. I would love to have a workspace dedicated to our RAG chat bot and another for code generation.
yindo commented 2026-02-22 18:19:19 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 14, 2024):

@icsy7867 Are the Ollama models running on the same server? If so, you can do that already via the workspace's settings. If the model is available, you can override the system model.

This issue is more towards connecting a totally different provider. Unless you mean you have Ollama running somewhere and the Mistral API service is another thing you want to connect, not just Mistral running on Ollama.

Screenshot 2024-02-14 at 8 45 29 AM
@timothycarambat commented on GitHub (Feb 14, 2024): @icsy7867 Are the Ollama models running on the same server? If so, you can do that already via the workspace's settings. If the model is available, you can override the system model. This issue is more towards connecting a totally different provider. Unless you mean you have Ollama running somewhere and the Mistral **API** service is another thing you want to connect, not just Mistral running on Ollama. <img width="682" alt="Screenshot 2024-02-14 at 8 45 29 AM" src="https://github.com/Mintplex-Labs/anything-llm/assets/16845892/940fc4dd-2c38-4ac3-96dc-dd4b307680bd">
yindo commented 2026-02-22 18:19:19 -05:00
Author
Owner
Copy Link
Copy Source

@icsy7867 commented on GitHub (Feb 14, 2024):

@icsy7867 Are the Ollama models running on the same server? If so, you can do that already via the workspace's settings. If the model is available, you can override the system model.

This issue is more towards connecting a totally different provider. Unless you mean you have Ollama running somewhere and the Mistral API service is another thing you want to connect, not just Mistral running on Ollama.

I wasn't under the impression that an ollama instance could run multiple model simultaneosly. Currently I am running two separate docker containers. Will give it a whirl!

@icsy7867 commented on GitHub (Feb 14, 2024): > @icsy7867 Are the Ollama models running on the same server? If so, you can do that already via the workspace's settings. If the model is available, you can override the system model. > > This issue is more towards connecting a totally different provider. Unless you mean you have Ollama running somewhere and the Mistral **API** service is another thing you want to connect, not just Mistral running on Ollama. > I wasn't under the impression that an ollama instance could run multiple model simultaneosly. Currently I am running two separate docker containers. Will give it a whirl!
yindo commented 2026-02-22 18:19:20 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 14, 2024):

You can! You would just use the ollama serve command and it will run an openAI compatible API at http://127.0.0.1:11434. Plug that into AnythingLLM and you can list what models you want to use. I believe Ollama hot loads them - obviously more models = more RAM

@timothycarambat commented on GitHub (Feb 14, 2024): You can! You would just use the `ollama serve` command and it will run an openAI compatible API at http://127.0.0.1:11434. Plug that into AnythingLLM and you can list what models you want to use. I believe Ollama hot loads them - obviously more models = more RAM
yindo commented 2026-02-22 18:19:20 -05:00
Author
Owner
Copy Link
Copy Source

@sheneman commented on GitHub (Feb 14, 2024):

Yeah, in that case ollama is a single provider of multiple models, which is great. But just to be clear, I'm looking for is the ability to configure per-workspace LLM providers so I can associate entirely different providers (OpenAI, Anthropic, ollama, etc.) with specific workspaces within a single AnythingLLM instance. I don't believe that AnythingLLM supports that yet? Currently I'd have to spin up a different instance of AnythingLLM for each provider.

@sheneman commented on GitHub (Feb 14, 2024): Yeah, in that case ollama is a single provider of multiple models, which is great. But just to be clear, I'm looking for is the ability to configure per-workspace LLM providers so I can associate entirely different providers (OpenAI, Anthropic, ollama, etc.) with specific workspaces within a single AnythingLLM instance. I don't believe that AnythingLLM supports that yet? Currently I'd have to spin up a different instance of AnythingLLM for each provider.
yindo commented 2026-02-22 18:19:20 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 14, 2024):

@sheneman Absolutely, im clear on the scope and detail of this issue being distinct providers and not just using a different model from the singular system provider which is currently live.

We are on the same page for this issue - I was just clarifying!

@timothycarambat commented on GitHub (Feb 14, 2024): @sheneman Absolutely, im clear on the scope and detail of this issue being **distinct providers** and not just using a different model from the singular _system provider_ which is currently live. We are on the same page for this issue - I was just clarifying!
yindo commented 2026-02-22 18:19:20 -05:00
Author
Owner
Copy Link
Copy Source

@sheneman commented on GitHub (Feb 14, 2024):

@timothycarambat - Awesome. Anything LLM is absolutely wonderful by the way!

@sheneman commented on GitHub (Feb 14, 2024): @timothycarambat - Awesome. Anything LLM is absolutely wonderful by the way!
yindo commented 2026-02-22 18:19:20 -05:00
Author
Owner
Copy Link
Copy Source

@hellkrusher commented on GitHub (Mar 28, 2024):

A the very least I would like to be able to assign different Token context window values for each Chat Model selected. Thank you.

@hellkrusher commented on GitHub (Mar 28, 2024): A the very least I would like to be able to assign different Token context window values for each Chat Model selected. Thank you.
yindo commented 2026-02-22 18:19:20 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 28, 2024):

Related https://github.com/Mintplex-Labs/anything-llm/issues/969

@timothycarambat commented on GitHub (Mar 28, 2024): Related https://github.com/Mintplex-Labs/anything-llm/issues/969
yindo changed title from [FEAT]: Support multiple providers at once for workspaces to [GH-ISSUE #698] [FEAT]: Support multiple providers at once for workspaces 2026-06-05 14:35:07 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#400
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?