Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3111] [FEAT]: Allow Customization of Base URL for OpenAI-Compatible LLM Models #1995

New Issue
Closed
opened 2026-02-22 18:27:38 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:27:38 -05:00
Owner
Copy Link
Copy Source

Originally created by @CaoMeiYouRen on GitHub (Feb 4, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3111

What would you like to see?

Issue Title: Allow Customization of Base URL for OpenAI-Compatible LLM Models

Description:
Currently, when configuring an OpenAI-compatible LLM model (e.g., local/self-hosted models or third-party services that mimic the OpenAI API), the base URL is hardcoded to OpenAI's default endpoint. This limits flexibility for users who want to integrate models hosted on alternative URLs, such as local instances (e.g., Ollama, LocalAI),custom cloud deployments, or other OpenAI-compatible API providers.

Suggested Solution:
Add an optional Base URL field in the LLM configuration settings for OpenAI-compatible models. This field would allow users to override the default base URL (e.g., https://api.openai.com/v1) with a custom endpoint (e.g., http://localhost:11434/v1).

Example Implementation:

In the model configuration UI, include a text input labeled "Custom Base URL" (optional).

If the field is left empty, fall back to the default OpenAI URL.

Ensure the configured base URL is properly validated and used for all API requests to the model.

Why This Matters:

Enables integration with self-hosted or niche OpenAI-compatible models.

Supports local development/testing (e.g., using localhost endpoints).

Increases compatibility with emerging LLM API providers that adopt the OpenAI schema.

Additional Context:
Many developers and organizations now leverage OpenAI-compatible APIs for cost, privacy, or customization reasons. Allowing base URL customization would align Anything-LLM with tools like LiteLLM or LangChain, which prioritize interoperability.

Thank you for considering this feature! 🙌

Originally created by @CaoMeiYouRen on GitHub (Feb 4, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3111 ### What would you like to see? Issue Title: Allow Customization of Base URL for OpenAI-Compatible LLM Models  Description: Currently, when configuring an OpenAI-compatible LLM model (e.g., local/self-hosted models or third-party services that mimic the OpenAI API), the base URL is hardcoded to OpenAI's default endpoint. This limits flexibility for users who want to integrate models hosted on alternative URLs, such as local instances (e.g., Ollama, LocalAI),custom cloud deployments, or other OpenAI-compatible API providers.  Suggested Solution: Add an optional Base URL field in the LLM configuration settings for OpenAI-compatible models. This field would allow users to override the default base URL (e.g., https://api.openai.com/v1) with a custom endpoint (e.g., http://localhost:11434/v1).  Example Implementation:  In the model configuration UI, include a text input labeled "Custom Base URL" (optional).  If the field is left empty, fall back to the default OpenAI URL.  Ensure the configured base URL is properly validated and used for all API requests to the model.  Why This Matters:  Enables integration with self-hosted or niche OpenAI-compatible models.  Supports local development/testing (e.g., using localhost endpoints).  Increases compatibility with emerging LLM API providers that adopt the OpenAI schema.  Additional Context: Many developers and organizations now leverage OpenAI-compatible APIs for cost, privacy, or customization reasons. Allowing base URL customization would align Anything-LLM with tools like LiteLLM or LangChain, which prioritize interoperability.  Thank you for considering this feature! 🙌
yindo added the enhancementfeature request labels 2026-02-22 18:27:38 -05:00
yindo closed this issue 2026-02-22 18:27:38 -05:00
yindo commented 2026-02-22 18:27:38 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 4, 2025):

The Generic OpenAI LLM provider exists for this use case

Image
@timothycarambat commented on GitHub (Feb 4, 2025): The Generic OpenAI LLM provider exists for this use case <img width="1612" alt="Image" src="https://github.com/user-attachments/assets/caeb51ec-bf6e-48e8-aeb3-0c44b082aff0" />
yindo commented 2026-02-22 18:27:38 -05:00
Author
Owner
Copy Link
Copy Source

@mpetruc commented on GitHub (May 10, 2025):

is there a way to add more than one Generic OpenAI LLM provider?

@mpetruc commented on GitHub (May 10, 2025): is there a way to add more than one Generic OpenAI LLM provider?
yindo changed title from [FEAT]: Allow Customization of Base URL for OpenAI-Compatible LLM Models to [GH-ISSUE #3111] [FEAT]: Allow Customization of Base URL for OpenAI-Compatible LLM Models 2026-06-05 14:43:53 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1995
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?