Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3871] [FEAT]: A drop down for keeping endpoints info saved under Generic OpenAI LLM service. #2472

New Issue
Open
opened 2026-02-22 18:29:49 -05:00 by yindo · 4 comments
yindo commented 2026-02-22 18:29:49 -05:00
Owner
Copy Link
Copy Source

Originally created by @OzMan21 on GitHub (May 23, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3871

What would you like to see?

When under the LLM global settings with the Generic OpenAI selected, it would be a great utility to have a dropdown that'd keep saved various endpoints and their respective API's and models, and make them easily selectable. Currently it is such that, we need to manually enter this information each time, and this is not only taking a lot of time to source this info from multiple different screens, but it also is creating probability for mistakes which then break operation.

In my case, I use RunPod to have various workers running different models, each a different endpoint, then there can be other servers running in the local network serving other LLM endpoints, so this utility is much desired, I imagine, to quickly and safely switch endpoints.

Thank you for this amazing UI and all the work Mindplex/Tim and all the developers have done! We love you very much for all you do.

Originally created by @OzMan21 on GitHub (May 23, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3871 ### What would you like to see? When under the LLM global settings with the Generic OpenAI selected, it would be a great utility to have a dropdown that'd keep saved various endpoints and their respective API's and models, and make them easily selectable. Currently it is such that, we need to manually enter this information each time, and this is not only taking a lot of time to source this info from multiple different screens, but it also is creating probability for mistakes which then break operation. In my case, I use RunPod to have various workers running different models, each a different endpoint, then there can be other servers running in the local network serving other LLM endpoints, so this utility is much desired, I imagine, to quickly and safely switch endpoints. Thank you for this amazing UI and all the work Mindplex/Tim and all the developers have done! We love you very much for all you do.
yindo added the enhancementfeature request labels 2026-02-22 18:29:49 -05:00
yindo commented 2026-02-22 18:29:49 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 25, 2025):

This is a great solution, we long since have been needing to move to "stored connections" so that we can persist them and have many separate connections - this would be under that.

Moving https://github.com/Mintplex-Labs/anything-llm/issues/2002 https://github.com/Mintplex-Labs/anything-llm/issues/1567 into this issue since it is more current

@timothycarambat commented on GitHub (May 25, 2025): This is a great solution, we long since have been needing to move to "stored connections" so that we can persist them and have many separate connections - this would be under that. Moving https://github.com/Mintplex-Labs/anything-llm/issues/2002 https://github.com/Mintplex-Labs/anything-llm/issues/1567 into this issue since it is more current
yindo commented 2026-02-22 18:29:49 -05:00
Author
Owner
Copy Link
Copy Source

@WickedSquirrel commented on GitHub (Jun 13, 2025):

Exactly what I came here to say

I use multiple models of the same external provider and have to always go back and copy and paste in the modellname I want to use.

@WickedSquirrel commented on GitHub (Jun 13, 2025): Exactly what I came here to say I use multiple models of the same external provider and have to always go back and copy and paste in the modellname I want to use.
yindo commented 2026-02-22 18:29:49 -05:00
Author
Owner
Copy Link
Copy Source

@chaserhkj commented on GitHub (Sep 6, 2025):

I second this improvement. I would also suggest to support switching between different configured providers and models in the chat interface as well, currently you can switch between the models of other providers, but not between different generic providers (or different models of the same provider)

This is relevant to my setup because I use LiteLLM proxy to do cost management and load balancing both on my own hardware and on different providers for a number of different LLM apps. And currently in AnythingLLM there is no way to use more than one model from such a setup.

@chaserhkj commented on GitHub (Sep 6, 2025): I second this improvement. I would also suggest to support switching between different configured providers and models in the chat interface as well, currently you can switch between the models of other providers, but not between different generic providers (or different models of the same provider) This is relevant to my setup because I use [LiteLLM proxy](https://docs.litellm.ai/docs/simple_proxy) to do cost management and load balancing both on my own hardware and on different providers for a number of different LLM apps. And currently in AnythingLLM there is no way to use more than one model from such a setup.
yindo changed title from [FEAT]: A drop down for keeping endpoints info saved under Generic OpenAI LLM service. to [GH-ISSUE #3871] [FEAT]: A drop down for keeping endpoints info saved under Generic OpenAI LLM service. 2026-06-05 14:46:45 -04:00
yindo commented 2026-06-05 14:46:46 -04:00
Author
Owner
Copy Link
Copy Source

@shanevcantwell commented on GitHub (Mar 26, 2026):

Hi Tim! I've been using AnythingLLM for close to 2 years now, so when looking to plug my own langgraph project into an existing UI to handle the chat context accumulation and threading/editing, I knew exactly what I wanted to use. My custom LAS project works great through the Generic OpenAI adapter. However, I also use llama.cpp llama-server that already goes through the single Generic OpenAI. I need a way to have 2 (or more, since I also switch between 2 workstations that multiply the possible configurations) unique host configurations where neither fits into any other existing LLM service adapter. Cheers

<!-- gh-comment-id:4138772317 --> @shanevcantwell commented on GitHub (Mar 26, 2026): Hi Tim! I've been using AnythingLLM for close to 2 years now, so when looking to plug my own langgraph project into an existing UI to handle the chat context accumulation and threading/editing, I knew exactly what I wanted to use. My custom LAS project works great through the Generic OpenAI adapter. However, I also use llama.cpp llama-server that already goes through the single Generic OpenAI. I need a way to have 2 (or more, since I also switch between 2 workstations that multiply the possible configurations) unique host configurations where neither fits into any other existing LLM service adapter. Cheers
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2472
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?