Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5234] [FEAT]: Add custom base URL for Anthropic provider and optional model discovery for Generic OpenAI #4980

New Issue
Open
opened 2026-06-05 14:51:16 -04:00 by yindo · 2 comments
yindo commented 2026-06-05 14:51:16 -04:00
Owner
Copy Link
Copy Source

Originally created by @raucodes on GitHub (Mar 19, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5234

What would you like to see?

What would you like to see?

I would like to request two related improvements for self-hosted / compatible API backends:

  1. Add custom base URL / custom host support to the Anthropic provider
  2. Add optional model discovery / model selection support to the Generic OpenAI provider

Why is this needed?

There are now local/self-hosted backends that expose APIs compatible with OpenAI and/or Anthropic.

For example, some backends support:

  • OpenAI-compatible /v1/chat/completions
  • OpenAI-compatible /v1/models
  • Anthropic-compatible /v1/messages
  • streaming
  • reasoning / thinking
  • tool calling

A good example is oMLX, which advertises itself as a drop-in replacement for both OpenAI and Anthropic APIs.

Problem today

1. Anthropic provider

The Anthropic connector appears to be tied to Anthropic-hosted endpoints only, with no way to set a custom host / base URL.

That makes it impossible to use Anthropic-compatible self-hosted backends directly.

2. Generic OpenAI provider

The Generic OpenAI connector works with custom hosts, but it appears to require a manually entered model name and does not seem to expose the same model selection UX as more specific providers.

This means that even when a backend supports GET /v1/models, users cannot conveniently pick from available models in the UI.

Expected behavior

Anthropic provider

Please allow:

  • API Key
  • Model
  • Base URL / Custom Host

Example:

  • https://api.anthropic.com
  • http://localhost:8080
  • https://my-self-hosted-backend.example.com

Generic OpenAI provider

Please optionally support:

  • calling GET /v1/models
  • populating the model dropdown from that endpoint
  • falling back to manual model entry when model discovery is unavailable

Why this would help

This would make AnythingLLM much more flexible for:

  • self-hosted backends
  • OpenAI-compatible servers
  • Anthropic-compatible servers
  • private/local inference infrastructure
  • advanced users who do not want to rely only on hosted APIs

It would also reduce friction when testing backends that already expose standards-compatible endpoints.

Additional context

Right now, Generic OpenAI is often the best workaround for self-hosted backends because it supports custom hosts, but it may miss provider-specific UX features such as model selection.

At the same time, the Anthropic provider could be a better fit for Anthropic-compatible servers, but only if custom base URLs are supported.

Originally created by @raucodes on GitHub (Mar 19, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5234 ### What would you like to see? ### What would you like to see? I would like to request two related improvements for self-hosted / compatible API backends: 1. **Add custom base URL / custom host support to the Anthropic provider** 2. **Add optional model discovery / model selection support to the Generic OpenAI provider** ### Why is this needed? There are now local/self-hosted backends that expose APIs compatible with OpenAI and/or Anthropic. For example, some backends support: - OpenAI-compatible `/v1/chat/completions` - OpenAI-compatible `/v1/models` - Anthropic-compatible `/v1/messages` - streaming - reasoning / thinking - tool calling A good example is **oMLX**, which advertises itself as a drop-in replacement for both OpenAI and Anthropic APIs. ### Problem today #### 1. Anthropic provider The Anthropic connector appears to be tied to Anthropic-hosted endpoints only, with no way to set a custom host / base URL. That makes it impossible to use Anthropic-compatible self-hosted backends directly. #### 2. Generic OpenAI provider The Generic OpenAI connector works with custom hosts, but it appears to require a manually entered model name and does not seem to expose the same model selection UX as more specific providers. This means that even when a backend supports `GET /v1/models`, users cannot conveniently pick from available models in the UI. ### Expected behavior #### Anthropic provider Please allow: - API Key - Model - **Base URL / Custom Host** Example: - `https://api.anthropic.com` - `http://localhost:8080` - `https://my-self-hosted-backend.example.com` #### Generic OpenAI provider Please optionally support: - calling `GET /v1/models` - populating the model dropdown from that endpoint - falling back to manual model entry when model discovery is unavailable ### Why this would help This would make AnythingLLM much more flexible for: - self-hosted backends - OpenAI-compatible servers - Anthropic-compatible servers - private/local inference infrastructure - advanced users who do not want to rely only on hosted APIs It would also reduce friction when testing backends that already expose standards-compatible endpoints. ### Additional context Right now, Generic OpenAI is often the best workaround for self-hosted backends because it supports custom hosts, but it may miss provider-specific UX features such as model selection. At the same time, the Anthropic provider could be a better fit for Anthropic-compatible servers, but only if custom base URLs are supported.
yindo added the enhancementfeature request labels 2026-06-05 14:51:16 -04:00
yindo commented 2026-06-05 14:51:17 -04:00
Author
Owner
Copy Link
Copy Source

@svar2388 commented on GitHub (Mar 19, 2026):

I just ran into this issue as well. I hooked up NanoGPT to the generic OpenAI connector but was surprised that it would not enumerate the models from the /v1/models endpoint, instead having to manually select only a single model.

<!-- gh-comment-id:4090899331 --> @svar2388 commented on GitHub (Mar 19, 2026): I just ran into this issue as well. I hooked up NanoGPT to the generic OpenAI connector but was surprised that it would not enumerate the models from the `/v1/models` endpoint, instead having to manually select only a single model.
yindo commented 2026-06-05 14:51:17 -04:00
Author
Owner
Copy Link
Copy Source

@1nhale commented on GitHub (Mar 27, 2026):

I also ran into this same issue. Hosted an azure anthropic on foundry but cant change the baseurl and also API key is binded to anthropic hosted models only.

<!-- gh-comment-id:4141345548 --> @1nhale commented on GitHub (Mar 27, 2026): I also ran into this same issue. Hosted an azure anthropic on foundry but cant change the baseurl and also API key is binded to anthropic hosted models only.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#4980
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?