Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2749] [FEAT]: Add bearer token to header for Ollama LLM Provider and UI for setting this. #1768

New Issue
Closed
opened 2026-02-22 18:26:26 -05:00 by yindo · 4 comments
yindo commented 2026-02-22 18:26:26 -05:00
Owner
Copy Link
Copy Source

Originally created by @sheneman on GitHub (Dec 2, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2749

Originally assigned to: @shatfield4, @timothycarambat on GitHub.

What would you like to see?

This should be really simple, high-impact, low-hanging fruit for the Desktop and Web App!!

Please add an optional advanced setting for Ollama providers where you can specify a bearer token.

Currently, AnythingLLM does not appear to have a mechanism or user interface for setting the bearer token in the http header for Ollama LLM Providers (Ollama endpoints). Other tools like WebUI and Enchanted provide this, which is really helpful if your central Ollama infrastructure supports Ollama API keys in this way using an authenticating proxy like NGINX, Caddy, etc.

This tiny missing feature is the only thing preventing much larger adoption of AnythingLLM within our organization.

Thank you so much for your consideration!

Originally created by @sheneman on GitHub (Dec 2, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2749 Originally assigned to: @shatfield4, @timothycarambat on GitHub. ### What would you like to see? This should be really simple, high-impact, low-hanging fruit for the Desktop and Web App!! Please add an optional advanced setting for Ollama providers where you can specify a bearer token. Currently, AnythingLLM does not appear to have a mechanism or user interface for setting the bearer token in the http header for Ollama LLM Providers (Ollama endpoints). Other tools like WebUI and Enchanted provide this, which is really helpful if your central Ollama infrastructure supports Ollama API keys in this way using an authenticating proxy like NGINX, Caddy, etc. This tiny missing feature is the only thing preventing much larger adoption of AnythingLLM within our organization. Thank you so much for your consideration!
yindo added the enhancementIntegration Requestfeature request labels 2026-02-22 18:26:26 -05:00
yindo closed this issue 2026-02-22 18:26:27 -05:00
yindo commented 2026-02-22 18:26:27 -05:00
Author
Owner
Copy Link
Copy Source

@ssbodapati commented on GitHub (Jan 22, 2025):

This works for me, @timothycarambat and @sheneman. Since the genericLLM constructor is using openAI constructor, the API key passed/set as env variable will be used for the header construction.

the below settings worked for me,

Base url : {host}/v1/ [ activating Open AI completions, reference.]
API Key : {token} [ Just the token, "Bearer " is appended by the open AI SDK ]
Chat Model Name: llava
ollama log -> [GIN] 200 | 57.044524133s | POST "/v1/chat/completions"

Image Image
@ssbodapati commented on GitHub (Jan 22, 2025): This works for me, @timothycarambat and @sheneman. Since the genericLLM constructor is using openAI constructor, the API key passed/set as env variable will be used for the header construction. the below settings worked for me, Base url : {host}/v1/ [ activating Open AI completions, [reference](https://ollama.com/blog/openai-compatibility).] API Key : {token} [ Just the token, "Bearer " is appended by the open AI SDK ] Chat Model Name: llava ollama log -> [GIN] 200 | 57.044524133s | POST "/v1/chat/completions" <img width="1745" alt="Image" src="https://github.com/user-attachments/assets/dbd6bd32-c873-4622-a4a6-af47e4f45914" /> <img width="1502" alt="Image" src="https://github.com/user-attachments/assets/e1b07f8e-e47f-4e1a-b650-6773c4831715" />
yindo commented 2026-02-22 18:26:27 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 22, 2025):

Where BaseURL is the URL of the Ollama instance, correct?

@timothycarambat commented on GitHub (Jan 22, 2025): Where BaseURL is the URL of the Ollama instance, correct?
yindo commented 2026-02-22 18:26:27 -05:00
Author
Owner
Copy Link
Copy Source

@sheneman commented on GitHub (Jan 22, 2025):

Yes, @timothycarambat and @ssbodapati -- for client use cases that are expecting an OpenAI Chat Completion endpoint, that would work. But I am interested in supporting a bearer token that can be optionally passed via the header for Ollama native endpoints. Some clients do not support OpenAI style endpoints.

The popular Enchanted iOS app is one example.

Image

Here is a relevant article on how to configure an auth proxy for Ollama.

https://www.arsturn.com/blog/understanding-authentication-in-ollama

And here is a relevant repo:

https://github.com/bartolli/ollama-bearer-auth

We are interested in having AnythingLLM support bearer tokens for native Ollama completion endpoints. Among other things, that would allow a more dynamic selection of different models, etc via pulldown menu in Workspace settings, etc. and be more similar to other tools (like Enhanted) that support auth to Ollama via bearer tokens.

@sheneman commented on GitHub (Jan 22, 2025): Yes, @timothycarambat and @ssbodapati -- for client use cases that are expecting an OpenAI Chat Completion endpoint, that would work. But I am interested in supporting a bearer token that can be optionally passed via the header for Ollama native endpoints. Some clients **_do not_** support OpenAI style endpoints. The popular Enchanted iOS app is one example. ![Image](https://github.com/user-attachments/assets/4e7f16a5-28c6-40ab-a98b-237d86403e43) Here is a relevant article on how to configure an auth proxy for Ollama. https://www.arsturn.com/blog/understanding-authentication-in-ollama And here is a relevant repo: https://github.com/bartolli/ollama-bearer-auth We are interested in having AnythingLLM support bearer tokens for native Ollama completion endpoints. Among other things, that would allow a more dynamic selection of different models, etc via pulldown menu in Workspace settings, etc. and be more similar to other tools (like Enhanted) that support auth to Ollama via bearer tokens.
yindo commented 2026-02-22 18:26:27 -05:00
Author
Owner
Copy Link
Copy Source

@ssbodapati commented on GitHub (Jan 23, 2025):

@timothycarambat yes, the base url is Ollama host url
I made a quick draft, here is the PR -> 2749 ollama client auth token #3005

@sheneman I'm able to fetch model list through /api/tags on a secure Ollama server by setting a bearer token.This should work for all other APIs as well.

Ollama LLM Preferences with Auth Token

Image

@ssbodapati commented on GitHub (Jan 23, 2025): @timothycarambat yes, the base url is Ollama host url I made a quick draft, here is the PR -> 2749 ollama client auth token #3005 @sheneman I'm able to fetch model list through /api/tags on a secure Ollama server by setting a bearer token.This should work for all other APIs as well. Ollama LLM Preferences with Auth Token ![Image](https://github.com/user-attachments/assets/9de49d18-92c3-43f3-8a37-fe35ea3dbeda)
yindo changed title from [FEAT]: Add bearer token to header for Ollama LLM Provider and UI for setting this. to [GH-ISSUE #2749] [FEAT]: Add bearer token to header for Ollama LLM Provider and UI for setting this. 2026-06-05 14:42:34 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label Integration Request enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1768
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?