Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1039] [DOCS]: How to select your model in the program for Windows? #640

New Issue
Closed
opened 2026-02-22 18:20:36 -05:00 by yindo · 5 comments
yindo commented 2026-02-22 18:20:36 -05:00
Owner
Copy Link
Copy Source

Originally created by @intulint on GitHub (Apr 4, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1039

Description

I only found this instruction for docker.
https://github.com/Mintplex-Labs/anything-llm/blob/master/server/storage/models/README.md#text-generation-llm-selection
How to add your model to the program? Creating a downloaded folder in AppData\Roaming\anythingllm-desktop\storage\models\ doesn't work.
Before this, I wanted to connect the model via koboldcpp, but the application does not support this backend.
I need a model with q8, I don’t like ollama, but the worst thing is that it downloads 4q versions of neural networks. They're too stupid.

Originally created by @intulint on GitHub (Apr 4, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1039 ### Description I only found this instruction for docker. https://github.com/Mintplex-Labs/anything-llm/blob/master/server/storage/models/README.md#text-generation-llm-selection How to add your model to the program? Creating a downloaded folder in AppData\Roaming\anythingllm-desktop\storage\models\ doesn't work. Before this, I wanted to connect the model via koboldcpp, but the application does not support this backend. I need a model with q8, I don’t like ollama, but the worst thing is that it downloads 4q versions of neural networks. They're too stupid.
yindo added the documentation label 2026-02-22 18:20:36 -05:00
yindo closed this issue 2026-02-22 18:20:36 -05:00
yindo commented 2026-02-22 18:20:36 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 4, 2024):

If that is the case, then this issue needs to be worked #513 to get full support for you then

@timothycarambat commented on GitHub (Apr 4, 2024): If that is the case, then this issue needs to be worked #513 to get full support for you then
yindo commented 2026-02-22 18:20:36 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 4, 2024):

Otherwise, you may certainly get a sub-par performance. Some people have said that using the LocalAI connector but pointed at your Kobold endpoint works? Oddly enough

@timothycarambat commented on GitHub (Apr 4, 2024): Otherwise, you may certainly get a sub-par performance. Some people have said that using the LocalAI connector but pointed at your Kobold endpoint works? Oddly enough
yindo commented 2026-02-22 18:20:37 -05:00
Author
Owner
Copy Link
Copy Source

@intulint commented on GitHub (Apr 4, 2024):

@timothycarambat I tried every available local menu item, nothing wanted to work with kobaldcpp.
Judging by the fact that that task has been hanging for 4 months, adding another local backend is of no interest to anyone. Which seems very strange. Since there is no llama.cpp server, no kobaldcpp, no text-generation-webui.

But there are several server part models that are cumbersome and not easy to launch, as well as adding the llm model you need with the launch parameters you need. impossible or inconvenient.

@intulint commented on GitHub (Apr 4, 2024): @timothycarambat I tried every available local menu item, nothing wanted to work with kobaldcpp. Judging by the fact that that task has been hanging for 4 months, adding another local backend is of no interest to anyone. Which seems very strange. Since there is no llama.cpp server, no kobaldcpp, no text-generation-webui. But there are several server part models that are cumbersome and not easy to launch, as well as adding the llm model you need with the launch parameters you need. impossible or inconvenient.
yindo commented 2026-02-22 18:20:37 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 4, 2024):

@intulint It's been in the backlog because we don't use it and have only seen the request 2, maybe 3 times now. If we prioritize every single provider that gets added we will never move on past that set of features since there are an unlimited amount of tools for running LLMs. Everyone has a preference and it's unreasonable to expect us to add them instantly if the demand is not high enough. Looking now, that request has 4 votes on it.

That's just the way it is. Also, we are OSS, so if you really need it today, you can add it. This strategy we have taken thus far is not impossible or inconvenient for those who use the tool in its current state, just for you and others who need XYZ integration.

In an ideal world, every LLM provider is supported so nobody has any requests, but we are a small core team and contribution is optional, not required. So at the end of the day, we are responsible for doing most integrations.

Since that is what this issue is about - I'll close this issue and it can be tracked on #513. Until then.

@timothycarambat commented on GitHub (Apr 4, 2024): @intulint It's been in the backlog because we don't use it and have only seen the request 2, maybe 3 times now. If we prioritize every single provider that gets added we will never move on past that set of features since there are an unlimited amount of tools for running LLMs. Everyone has a preference and it's unreasonable to expect us to add them instantly if the demand is not high enough. Looking now, that request has 4 votes on it. That's just the way it is. Also, we are OSS, so if you _really need it today_, you can add it. This strategy we have taken thus far is not impossible or inconvenient for those who use the tool in its current state, just for you and others who need XYZ integration. In an ideal world, every LLM provider is supported so nobody has any requests, but we are a small core team and contribution is optional, not required. So at the end of the day, we are responsible for doing most integrations. Since that is what this issue is about - I'll close this issue and it can be tracked on #513. Until then.
yindo commented 2026-02-22 18:20:37 -05:00
Author
Owner
Copy Link
Copy Source

@intulint commented on GitHub (Apr 4, 2024):

@timothycarambat

But the problem for which I opened this ticket is not in the launch method and it has not yet been resolved. I don't agree with the closure.

I did not find how to add my already downloaded model to the program that is installed by the installer. This is not in the documentation, that's all. There is a mention that you can add your own model in the Docker version, but it is not clear how in the application.

I have a feeling that everyone is trying to make users depend on some servers from which models will be downloaded. Neither here, nor in lmstudio, nor most likely in ollama, you can simply add your pre-downloaded model.
This, guys, already looks like some kind of conspiracy, heh.

@intulint commented on GitHub (Apr 4, 2024): @timothycarambat But the problem for which I opened this ticket is not in the launch method and it has not yet been resolved. I don't agree with the closure. I did not find how to add my already downloaded model to the program that is installed by the installer. This is not in the documentation, that's all. There is a mention that you can add your own model in the Docker version, but it is not clear how in the application. I have a feeling that everyone is trying to make users depend on some servers from which models will be downloaded. Neither here, nor in lmstudio, nor most likely in ollama, you can simply add your pre-downloaded model. This, guys, already looks like some kind of conspiracy, heh.
yindo changed title from [DOCS]: How to select your model in the program for Windows? to [GH-ISSUE #1039] [DOCS]: How to select your model in the program for Windows? 2026-06-05 14:36:26 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label documentation
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#640
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?