Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2094] [FEAT]: Custom Endpoint API Claude #1363

New Issue
Closed
opened 2026-02-22 18:24:26 -05:00 by yindo · 7 comments
yindo commented 2026-02-22 18:24:26 -05:00
Owner
Copy Link
Copy Source

Originally created by @chalitbkb on GitHub (Aug 12, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2094

What would you like to see?

https://xxxxxx.ngrok-free.app/api/claude

Is it possible to run the Claude API from a custom endpoint that I set up myself? From the image, you can see that I am using a custom endpoint and the result as shown. I'm not sure if the current version of "ChatGPTNextWeb" supports custom endpoints.

Or should we handle the backend API code to ensure the API responses are formatted correctly to support this? I've tried, but it's not working.

image

Originally created by @chalitbkb on GitHub (Aug 12, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2094 ### What would you like to see? >> https://xxxxxx.ngrok-free.app/api/claude Is it possible to run the Claude API from a custom endpoint that I set up myself? From the image, you can see that I am using a custom endpoint and the result as shown. I'm not sure if the current version of "ChatGPTNextWeb" supports custom endpoints. Or should we handle the backend API code to ensure the API responses are formatted correctly to support this? I've tried, but it's not working. ![image](https://github.com/user-attachments/assets/4a4f8856-192a-48e2-8896-f97d16c2be8a)
yindo added the enhancementfeature request labels 2026-02-22 18:24:26 -05:00
yindo closed this issue 2026-02-22 18:24:27 -05:00
yindo commented 2026-02-22 18:24:27 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 12, 2024):

In general, if you are using any endpoint it should be in openai endpoint format since we cannot possible support every random endpoint that someone may come up with. Other tools enforce this kind of spec or produce this kind of spec for easy integrations with other tools since you can always expect /v1/models to return models and such.

Is this endpoint simply the base and https://xxxx.ngrok.app/api/claude/v1/models would still work? If that is that case simply use https://xxxx.ngrok.app/api/claude as the base URL in the generic OpenAI connector

@timothycarambat commented on GitHub (Aug 12, 2024): In general, if you are using any endpoint it should _be in openai endpoint format_ since we cannot possible support every random endpoint that someone may come up with. Other tools enforce this kind of spec or produce this kind of spec for easy integrations with other tools since you can always expect `/v1/models` to return models and such. Is this endpoint simply the base and `https://xxxx.ngrok.app/api/claude/v1/models` would still work? If that is that case simply use `https://xxxx.ngrok.app/api/claude` as the base URL in the generic OpenAI connector
yindo commented 2026-02-22 18:24:27 -05:00
Author
Owner
Copy Link
Copy Source

@chalitbkb commented on GitHub (Aug 13, 2024):

In general, if you are using any endpoint it should be in openai endpoint format since we cannot possible support every random endpoint that someone may come up with. Other tools enforce this kind of spec or produce this kind of spec for easy integrations with other tools since you can always expect /v1/models to return models and such.

Is this endpoint simply the base and https://xxxx.ngrok.app/api/claude/v1/models would still work? If that is that case simply use https://xxxx.ngrok.app/api/claude as the base URL in the generic OpenAI connector

@timothycarambat

Where should I configure the endpoint in the "anything-llm Desktop app"? I can't find any section that allows for setting a custom API endpoint. Also, the custom endpoint already includes the predefined API key from the provider. If that's the case, how should I handle the API key configuration in the "anything-llm Desktop app"?

@chalitbkb commented on GitHub (Aug 13, 2024): > In general, if you are using any endpoint it should _be in openai endpoint format_ since we cannot possible support every random endpoint that someone may come up with. Other tools enforce this kind of spec or produce this kind of spec for easy integrations with other tools since you can always expect `/v1/models` to return models and such. > > Is this endpoint simply the base and `https://xxxx.ngrok.app/api/claude/v1/models` would still work? If that is that case simply use `https://xxxx.ngrok.app/api/claude` as the base URL in the generic OpenAI connector @timothycarambat Where should I configure the endpoint in the "anything-llm Desktop app"? I can't find any section that allows for setting a custom API endpoint. Also, the custom endpoint already includes the predefined API key from the provider. If that's the case, how should I handle the API key configuration in the "anything-llm Desktop app"?
yindo commented 2026-02-22 18:24:27 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 13, 2024):

In this case, you should use the "Generic OpenAI connector" assuming the API is openAI compatible, is the service you are using us the same "OpenAi" URL schema?

@timothycarambat commented on GitHub (Aug 13, 2024): In this case, you should use the "Generic OpenAI connector" assuming the API is openAI compatible, is the service you are using us the same "OpenAi" URL schema?
yindo commented 2026-02-22 18:24:27 -05:00
Author
Owner
Copy Link
Copy Source

@chalitbkb commented on GitHub (Aug 14, 2024):

In this case, you should use the "Generic OpenAI connector" assuming the API is openAI compatible, is the service you are using us the same "OpenAi" URL schema?

@timothycarambat

This is what I tried in Postman. There were no issues, and the response format met all of Anthropic's specifications. I guess "anything-llm Desktop" doesn't have custom options for Anthropic providers.

image

image

https://docs.anthropic.com/en/api/messages-streaming
image

This is the option I see, which only requires an apikey. There doesn't seem to be a field for specifying a custom endpoint.
image

@chalitbkb commented on GitHub (Aug 14, 2024): > In this case, you should use the "Generic OpenAI connector" assuming the API is openAI compatible, is the service you are using us the same "OpenAi" URL schema? @timothycarambat This is what I tried in Postman. There were no issues, and the response format met all of Anthropic's specifications. I guess "anything-llm Desktop" doesn't have custom options for Anthropic providers. ![image](https://github.com/user-attachments/assets/8d5d9b8a-d480-4cf0-9c8a-70939b76df50) ![image](https://github.com/user-attachments/assets/a613ee1d-65b1-4ab2-bf80-c9b67c73db6f) https://docs.anthropic.com/en/api/messages-streaming ![image](https://github.com/user-attachments/assets/f751d6c5-e670-4bc0-9d58-f2e0a6cac8e9) This is the option I see, which only requires an apikey. There doesn't seem to be a field for specifying a custom endpoint. ![image](https://github.com/user-attachments/assets/76245ec4-2860-4f85-9a77-790a4e2f3233)
yindo commented 2026-02-22 18:24:27 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 14, 2024):

I still have not had the questions answered that would allow me to answer this best.

  • Does this base URL comply to the OpenAI schema? If it does, you can use the OpenAI Generic Connector, even if it is Anthropic.

We dont allow overriding of the Anthropic right now because...well how do you even have a custom Anthropic URL? It looks like you are simply port forwarding some process running?

@timothycarambat commented on GitHub (Aug 14, 2024): I still have not had the questions answered that would allow me to answer this best. - Does this base URL comply to the OpenAI schema? If it does, you can use the OpenAI Generic Connector, even if it is Anthropic. We dont allow overriding of the Anthropic right now because...well how do you even _have_ a custom Anthropic URL? It looks like you are simply port forwarding some process running?
yindo commented 2026-02-22 18:24:27 -05:00
Author
Owner
Copy Link
Copy Source

@chalitbkb commented on GitHub (Aug 14, 2024):

I still have not had the questions answered that would allow me to answer this best.

  • Does this base URL comply to the OpenAI schema? If it does, you can use the OpenAI Generic Connector, even if it is Anthropic.

We dont allow overriding of the Anthropic right now because...well how do you even have a custom Anthropic URL? It looks like you are simply port forwarding some process running?

@timothycarambat

I apologize for any confusion earlier. To provide more clarity, I have created an additional set of APIs that act as intermediaries. These APIs incorporate custom management functionalities, such as handling the API key directly within the API or having multiple backup API keys in a single API setup. Despite these customizations, the response format strictly adheres to the original provider's specifications without any changes.

Summary:
I have created a custom API layer that acts as an intermediary, with some specific management features like built-in API keys or multiple backup API keys within the same API. This custom API maintains the original response format as specified by the service provider without any modifications. However, it seems that the "anything-llm Desktop app" doesn't currently offer options for custom Anthropic providers or endpoints.

Regarding your questions:

  1. Yes, my custom endpoint complies with the OpenAI schema in terms of request and response formats.

  2. The base URL I'm using is indeed a custom implementation that includes port forwarding to a process I'm running. It's not directly from Anthropic, but rather an intermediary layer I've set up to manage and route requests while maintaining compatibility with the original API specifications.

  3. I understand that using the Generic OpenAI Connector might be a potential solution. However, I'm still unsure about how to handle the API key configuration in the "anything-llm Desktop app" since my custom endpoint already includes a predefined API key.

Does the "anything-llm Desktop app" support configuring such a custom endpoint? If so, where can I set this up?

@chalitbkb commented on GitHub (Aug 14, 2024): > I still have not had the questions answered that would allow me to answer this best. > > * Does this base URL comply to the OpenAI schema? If it does, you can use the OpenAI Generic Connector, even if it is Anthropic. > > We dont allow overriding of the Anthropic right now because...well how do you even _have_ a custom Anthropic URL? It looks like you are simply port forwarding some process running? @timothycarambat I apologize for any confusion earlier. To provide more clarity, I have created an additional set of APIs that act as intermediaries. These APIs incorporate custom management functionalities, such as handling the API key directly within the API or having multiple backup API keys in a single API setup. Despite these customizations, the response format strictly adheres to the original provider's specifications without any changes. Summary: I have created a custom API layer that acts as an intermediary, with some specific management features like built-in API keys or multiple backup API keys within the same API. This custom API maintains the original response format as specified by the service provider without any modifications. However, it seems that the "anything-llm Desktop app" doesn't currently offer options for custom Anthropic providers or endpoints. Regarding your questions: 1. Yes, my custom endpoint complies with the OpenAI schema in terms of request and response formats. 2. The base URL I'm using is indeed a custom implementation that includes port forwarding to a process I'm running. It's not directly from Anthropic, but rather an intermediary layer I've set up to manage and route requests while maintaining compatibility with the original API specifications. 3. I understand that using the Generic OpenAI Connector might be a potential solution. However, I'm still unsure about how to handle the API key configuration in the "anything-llm Desktop app" since my custom endpoint already includes a predefined API key. Does the "anything-llm Desktop app" support configuring such a custom endpoint? If so, where can I set this up?
yindo commented 2026-02-22 18:24:28 -05:00
Author
Owner
Copy Link
Copy Source

@chalitbkb commented on GitHub (Aug 18, 2024):

@timothycarambat

Hello again, I've thoroughly checked with "anything-llm" and found no option for users to set their own API endpoint. Additionally, I used my own API endpoint, integrating it with other "LLMs" that support custom API endpoints, and it worked well. However, throughout our discussion, my only requirement has been to find the field to input the API endpoint URL on "anything-llm," but I can't find it. It only allows users to set the "apikey."

@chalitbkb commented on GitHub (Aug 18, 2024): @timothycarambat Hello again, I've thoroughly checked with "anything-llm" and found no option for users to set their own API endpoint. Additionally, I used my own API endpoint, integrating it with other "LLMs" that support custom API endpoints, and it worked well. However, throughout our discussion, my only requirement has been to find the field to input the API endpoint URL on "anything-llm," but I can't find it. It only allows users to set the "apikey."
yindo changed title from [FEAT]: Custom Endpoint API Claude to [GH-ISSUE #2094] [FEAT]: Custom Endpoint API Claude 2026-06-05 14:40:22 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1363
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?