Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5715] Question: How does a fresh AnythingLLM user authenticate Ollama cloud models? #5252

New Issue
Closed
opened 2026-06-05 14:52:56 -04:00 by yindo · 2 comments
yindo commented 2026-06-05 14:52:56 -04:00
Owner
Copy Link
Copy Source

Originally created by @aestheticdoodle on GitHub (May 27, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5715

Hi Mintplex Labs team,

I've been exploring AnythingLLM's built-in Ollama provider and had a question I couldn't find a clear answer to.

I noticed that cloud models (e.g. gemma4:31b-cloud) can be imported through the built-in provider's "ollama run ..." import field. However, authentication for Ollama cloud models relies on the Ed25519 key in the system's ~/.ollama/ folder, which gets linked to an ollama.com account via "ollama signin" in the terminal.

My question is: what is the intended flow for a fresh AnythingLLM user who has never installed system Ollama and has no existing ~/.ollama/id_ed25519 key linked to an account? Specifically:

  1. Does AnythingLLM's bundled llm.exe trigger a browser-based sign-in flow automatically when a cloud model import is attempted without valid credentials?
  2. Or is the expectation that users must separately install system Ollama and run "ollama signin" in the terminal before cloud models can work through AnythingLLM?

I ask because there doesn't appear to be any sign-in button or OAuth flow within AnythingLLM's UI itself, which seems like it could leave fresh users with no clear path to authenticating for cloud models.

Thanks for your time and for building such a great tool.

Best regards.

Originally created by @aestheticdoodle on GitHub (May 27, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5715 Hi Mintplex Labs team, I've been exploring AnythingLLM's built-in Ollama provider and had a question I couldn't find a clear answer to. I noticed that cloud models (e.g. gemma4:31b-cloud) can be imported through the built-in provider's "ollama run ..." import field. However, authentication for Ollama cloud models relies on the Ed25519 key in the system's ~/.ollama/ folder, which gets linked to an ollama.com account via "ollama signin" in the terminal. My question is: what is the intended flow for a fresh AnythingLLM user who has never installed system Ollama and has no existing ~/.ollama/id_ed25519 key linked to an account? Specifically: 1. Does AnythingLLM's bundled llm.exe trigger a browser-based sign-in flow automatically when a cloud model import is attempted without valid credentials? 2. Or is the expectation that users must separately install system Ollama and run "ollama signin" in the terminal before cloud models can work through AnythingLLM? I ask because there doesn't appear to be any sign-in button or OAuth flow within AnythingLLM's UI itself, which seems like it could leave fresh users with no clear path to authenticating for cloud models. Thanks for your time and for building such a great tool. Best regards.
yindo closed this issue 2026-06-05 14:52:56 -04:00
yindo commented 2026-06-05 14:52:57 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 27, 2026):

In general, at least because we have not intentionally designed it, we dont support ollama cloud (apparently?)

Does AnythingLLM's bundled llm.exe trigger a browser-based sign-in flow automatically when a cloud model import is attempted without valid credentials?

We do not - since I am unsure at this moment if there is a way to pre-detect if a model is cloud based or not from the tag. I am sure there is an API though?

Or is the expectation that users must separately install system Ollama and run "ollama signin" in the terminal before cloud models can work through AnythingLLM?

This would not be ideal either!

To maybe reframe this. The "internal" ollama engine we ship with is intended to just be the engine that runs GGUFs that we can pull from a central repository for without having to mirror or host TB of GGUFs. Its not really intended to act as Ollama in its place. In fact, we dont even have the full app - it is just the MIT-Licensed engine part of Ollama.

That being said, if someone was a paying customer of Ollama Cloud and they wanted to use that in AnythingLLM, they should be able to just use the dedicated Ollama provider (not our built-in!) that they use already, which points at that pubkey and already has been signed in via the CLI.

AnythingLLM is designed to be provider and model agnostic, however shipping a desktop app without a way to run a model for someone who has never done so before is a non-starter. Since when we started the desktop app (1.5yr ago i think) llama.cpp was still a lot more clunky to use so we used Ollama since license permitted it.

If i had to restart today, I would just bake in llama.cpp.

Does that explanation make sense and solve the issue?

<!-- gh-comment-id:4558188127 --> @timothycarambat commented on GitHub (May 27, 2026): In general, at least because we have not intentionally designed it, we dont support ollama cloud (apparently?) > Does AnythingLLM's bundled llm.exe trigger a browser-based sign-in flow automatically when a cloud model import is attempted without valid credentials? We do not - since I am unsure at this moment if there is a way to pre-detect if a model is cloud based or not from the tag. I am sure there is an API though? > Or is the expectation that users must separately install system Ollama and run "ollama signin" in the terminal before cloud models can work through AnythingLLM? This would not be ideal either! To maybe reframe this. The "internal" ollama engine we ship with is intended to just be the engine that runs GGUFs that we can pull from a central repository for without having to mirror or host TB of GGUFs. Its not really intended to act as Ollama in its place. In fact, we dont even have the full app - it is just the MIT-Licensed engine part of Ollama. That being said, if someone was a paying customer of Ollama Cloud and they wanted to use that in AnythingLLM, they _should_ be able to just use the dedicated Ollama provider (not our built-in!) that they use already, which points at that pubkey and already has been signed in via the CLI. AnythingLLM is designed to be provider and model agnostic, however shipping a desktop app without a way to run a model for someone who has never done so before is a non-starter. Since when we started the desktop app (1.5yr ago i think) llama.cpp was still a lot more clunky to use so we used Ollama since license permitted it. If i had to restart today, I would just bake in llama.cpp. Does that explanation make sense and solve the issue?
yindo commented 2026-06-05 14:52:57 -04:00
Author
Owner
Copy Link
Copy Source

@aestheticdoodle commented on GitHub (May 28, 2026):

Thank you for the detailed response — that clears things up!

One small suggestion: it might be worth adding a brief note in the official docs clarifying that the built-in AnythingLLM provider is just Ollama's MIT-licensed engine intended for running local GGUFs, and is not a full Ollama replacement. Specifically mentioning that Ollama cloud models are not an intentionally supported feature through the built-in provider could save future users a lot of confusion.

I went down quite a rabbit hole trying to figure out why cloud models accidentally worked through the built-in provider — and the answer wasn't findable anywhere online. A single line in the docs would have saved that entire investigation 😅

Thanks again for taking the time to respond.

<!-- gh-comment-id:4560893084 --> @aestheticdoodle commented on GitHub (May 28, 2026): Thank you for the detailed response — that clears things up! One small suggestion: it might be worth adding a brief note in the official docs clarifying that the built-in AnythingLLM provider is just Ollama's MIT-licensed engine intended for running local GGUFs, and is not a full Ollama replacement. Specifically mentioning that Ollama cloud models are not an intentionally supported feature through the built-in provider could save future users a lot of confusion. I went down quite a rabbit hole trying to figure out why cloud models accidentally worked through the built-in provider — and the answer wasn't findable anywhere online. A single line in the docs would have saved that entire investigation 😅 Thanks again for taking the time to respond.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5252
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?