Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #662] [FEAT]: OPENAI BASE params #376

New Issue
Closed
opened 2026-02-22 18:19:13 -05:00 by yindo · 11 comments
yindo commented 2026-02-22 18:19:13 -05:00
Owner
Copy Link
Copy Source

Originally created by @sherry0429 on GitHub (Jan 31, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/662

Originally assigned to: @timothycarambat on GitHub.

What would you like to see?

npm package openai have a config api base, like a proxy settings.

if someone only want to use OpenAI API rather than any llm service, this config will help a lot.

Originally created by @sherry0429 on GitHub (Jan 31, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/662 Originally assigned to: @timothycarambat on GitHub. ### What would you like to see? npm package openai have a config api base, like a proxy settings. if someone only want to use OpenAI API rather than any llm service, this config will help a lot.
yindo added the enhancementfeature request labels 2026-02-22 18:19:13 -05:00
yindo closed this issue 2026-02-22 18:19:13 -05:00
yindo commented 2026-02-22 18:19:13 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 1, 2024):

Ah, so like being able to hookup any "generic" service that also is fully compatible with the OpenAI API schema via just overriding the baseURL?

@timothycarambat commented on GitHub (Feb 1, 2024): Ah, so like being able to hookup any "generic" service that also is fully compatible with the OpenAI API schema via just overriding the baseURL?
yindo commented 2026-02-22 18:19:13 -05:00
Author
Owner
Copy Link
Copy Source

@sherry0429 commented on GitHub (Feb 1, 2024):

Yes, this technology is widely used in OpenAI LLM integrators. They define an intermediary site and generate an intermediary API key for users. Users use this pair (host, api_key) to access their servers, and they forward the requests to OpenAI. For example, in the request:

curl https://api.openai.com/v1/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-3.5-turbo-instruct",
    "prompt": "Say this is a test",
    "max_tokens": 7,
    "temperature": 0
  }'

The https://{OPENAI_API_BASE}/v1/completions and $OPENAI_API_KEY will be replaced with the values from the LLM integrator, which implements all of OpenAI's public interfaces.

In anythingLLM, When users utilize OpenAI models, they can overwrite the request's host address, enabling them to utilize any proxy service integrated with OpenAI and implementing OpenAI's public API. This proves beneficial for those facing poor network conditions and desiring to utilize custom services or plugins.

@sherry0429 commented on GitHub (Feb 1, 2024): Yes, this technology is widely used in OpenAI LLM integrators. They define an intermediary site and generate an intermediary API key for users. Users use this pair (host, api_key) to access their servers, and they forward the requests to OpenAI. For example, in the request: ``` curl https://api.openai.com/v1/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-3.5-turbo-instruct", "prompt": "Say this is a test", "max_tokens": 7, "temperature": 0 }' ``` The https://{OPENAI_API_BASE}/v1/completions and $OPENAI_API_KEY will be replaced with the values from the LLM integrator, which implements all of OpenAI's public interfaces. In anythingLLM, When users utilize OpenAI models, they can overwrite the request's host address, enabling them to utilize any proxy service integrated with OpenAI and implementing OpenAI's public API. This proves beneficial for those facing poor network conditions and desiring to utilize custom services or plugins.
yindo commented 2026-02-22 18:19:13 -05:00
Author
Owner
Copy Link
Copy Source

@sherry0429 commented on GitHub (Feb 1, 2024):

I am not sure in openai npm official package how to set is, but in python

import openai
openai.api_base = ""   #  custom api base, like https://test.api/v1
openai.api_key = ""     #  custom key
@sherry0429 commented on GitHub (Feb 1, 2024): I am not sure in openai npm official package how to set is, but in python ``` import openai openai.api_base = "" # custom api base, like https://test.api/v1 openai.api_key = "" # custom key ```
yindo commented 2026-02-22 18:19:13 -05:00
Author
Owner
Copy Link
Copy Source

@sherry0429 commented on GitHub (Feb 1, 2024):

Furthermore, it would be beneficial if AnythingLLM could provide a list of the OpenAI APIs it utilizes. This could potentially further ignite enthusiasm among similar middleware or integrators to integrate AnythingLLM.

@sherry0429 commented on GitHub (Feb 1, 2024): Furthermore, it would be beneficial if AnythingLLM could provide a list of the OpenAI APIs it utilizes. This could potentially further ignite enthusiasm among similar middleware or integrators to integrate AnythingLLM.
yindo commented 2026-02-22 18:19:13 -05:00
Author
Owner
Copy Link
Copy Source

@sherry0429 commented on GitHub (Feb 4, 2024):

@timothycarambat could anything llm add this feature in future ? thanks

@sherry0429 commented on GitHub (Feb 4, 2024): @timothycarambat could anything llm add this feature in future ? thanks
yindo commented 2026-02-22 18:19:13 -05:00
Author
Owner
Copy Link
Copy Source

@Jiongguang commented on GitHub (Feb 7, 2024):

The following code, with the addition of the basepath parameter, should be able to specify a proxy, but during testing there were errors that I couldn't figure out where the problem was.

const config = new Configuration({
apiKey: process.env.OPEN_AI_KEY,
basePath: "http://xxx.xxx.xxx.xxx:3000/v1"
});

2024-02-07T08:37:30.839326169Z OpenAI:listModels Request failed with status code 401
2024-02-07T08:37:42.421404929Z [TELEMETRY SENT] {
2024-02-07T08:37:42.421448968Z event: 'api_key_created',
2024-02-07T08:37:42.421457794Z distinctId: 'b988b14e-f0d5-4fe1-a3f4-48ec0fd63266',
2024-02-07T08:37:42.421463778Z properties: { runtime: 'docker' }
2024-02-07T08:37:42.421469076Z }
2024-02-07T08:46:33.544755978Z Error: OpenAI::CreateModeration failed with: Request failed with status code 503
2024-02-07T08:46:33.544802705Z at /app/server/utils/AiProviders/openAi/index.js:106:15
2024-02-07T08:46:33.544810634Z at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-02-07T08:46:33.544815584Z at async OpenAiLLM.isSafe (/app/server/utils/AiProviders/openAi/index.js:95:50)
2024-02-07T08:46:33.544820165Z at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:35:34)
2024-02-07T08:46:33.544824563Z at async /app/server/endpoints/chat.js:94:9

@Jiongguang commented on GitHub (Feb 7, 2024): The following code, with the addition of the basepath parameter, should be able to specify a proxy, but during testing there were errors that I couldn't figure out where the problem was. const config = new Configuration({ apiKey: process.env.OPEN_AI_KEY, basePath: "http://xxx.xxx.xxx.xxx:3000/v1" }); 2024-02-07T08:37:30.839326169Z OpenAI:listModels Request failed with status code 401 2024-02-07T08:37:42.421404929Z [TELEMETRY SENT] { 2024-02-07T08:37:42.421448968Z event: 'api_key_created', 2024-02-07T08:37:42.421457794Z distinctId: 'b988b14e-f0d5-4fe1-a3f4-48ec0fd63266', 2024-02-07T08:37:42.421463778Z properties: { runtime: 'docker' } 2024-02-07T08:37:42.421469076Z } 2024-02-07T08:46:33.544755978Z Error: OpenAI::CreateModeration failed with: Request failed with status code 503 2024-02-07T08:46:33.544802705Z at /app/server/utils/AiProviders/openAi/index.js:106:15 2024-02-07T08:46:33.544810634Z at process.processTicksAndRejections (node:internal/process/task_queues:95:5) 2024-02-07T08:46:33.544815584Z at async OpenAiLLM.isSafe (/app/server/utils/AiProviders/openAi/index.js:95:50) 2024-02-07T08:46:33.544820165Z at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:35:34) 2024-02-07T08:46:33.544824563Z at async /app/server/endpoints/chat.js:94:9
yindo commented 2026-02-22 18:19:13 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 7, 2024):

The error here is because the chats flow through a moderation endpoint that normally exists for openAI, but given the provider URL and API key changed. it is going to 503 since its not a valid key.

We will make a "generic" provider that is basically openAI, but you can modify the baseURL and API key. We dont want to edit the OpenAi method mostly because many people would not need this, but it would be nice to have the generic wrapper that functions in the way you have outlined.

@timothycarambat commented on GitHub (Feb 7, 2024): The error here is because the chats flow through a moderation endpoint that normally exists for openAI, but given the provider URL and API key changed. it is going to 503 since its not a valid key. We will make a "generic" provider that is basically openAI, but you can modify the baseURL and API key. We dont want to edit the OpenAi method mostly because many people would not need this, but it would be nice to have the generic wrapper that functions in the way you have outlined.
yindo commented 2026-02-22 18:19:13 -05:00
Author
Owner
Copy Link
Copy Source

@Jiongguang commented on GitHub (Feb 18, 2024):

Thanks for your reply

@Jiongguang commented on GitHub (Feb 18, 2024): Thanks for your reply
yindo commented 2026-02-22 18:19:14 -05:00
Author
Owner
Copy Link
Copy Source

@kloudtaxi commented on GitHub (Feb 26, 2024):

👋 everyone,
My team is working on the use case mentioned above by @sherry0429. I think a generic OpenAI LLM wrapper is a wonderful feature that will enhance the appeal of anthingLLM

@kloudtaxi commented on GitHub (Feb 26, 2024): 👋 everyone, My team is working on the use case mentioned above by @sherry0429. I think a generic OpenAI LLM wrapper is a wonderful feature that will enhance the appeal of anthingLLM
yindo commented 2026-02-22 18:19:14 -05:00
Author
Owner
Copy Link
Copy Source

@krishna-praveen commented on GitHub (Apr 21, 2024):

Please i need this. Currently this is there in big-agi and i want to switch to anything llm but this option is missing. If there is extra input that can set openai base url that would be great.

@krishna-praveen commented on GitHub (Apr 21, 2024): Please i need this. Currently this is there in big-agi and i want to switch to anything llm but this option is missing. If there is extra input that can set openai base url that would be great.
yindo commented 2026-02-22 18:19:14 -05:00
Author
Owner
Copy Link
Copy Source

@dantosXD commented on GitHub (Apr 22, 2024):

I too am looking for a way to change the Base URL. Any guidance would be appreciated.

@dantosXD commented on GitHub (Apr 22, 2024): I too am looking for a way to change the Base URL. Any guidance would be appreciated.
yindo changed title from [FEAT]: OPENAI BASE params to [GH-ISSUE #662] [FEAT]: OPENAI BASE params 2026-06-05 14:34:59 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#376
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?