Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #712] [DOCS]: AnythingLLM for desktop beta - folder for storing downloaded LLAMA GGUF LLMs from HuggingFace #409

New Issue
Closed
opened 2026-02-22 18:19:21 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:19:21 -05:00
Owner
Copy Link
Copy Source

Originally created by @BharatBlade on GitHub (Feb 12, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/712

Description

I see documentation for where to store gguf models here: https://github.com/Mintplex-Labs/anything-llm/blob/master/server/storage/models/README.md

saying that "/server/storage/models/downloaded is the default location that your model files should be at". However the directory structure for the anythingllm-desktop installation is understandably very different from the source/dev directory structure in GitHub. I'm very confused where I'm supposed to put models for the "Native" LLM Preference. What I'm referring to specifically is neither a dev-build nor a docker build. I'm referring to the very exciting "AnythingLLM for desktop" public beta.

Any help at all is greatly appreciated!

I can't find a similar folder for where to put downloaded gguf models so I just get the message "-- waiting for models --"
image

For reference, this is the directory structure/files I'm looking at in the anythingllm-desktop installation:
image

image

Originally created by @BharatBlade on GitHub (Feb 12, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/712 ### Description I see documentation for where to store gguf models here: https://github.com/Mintplex-Labs/anything-llm/blob/master/server/storage/models/README.md saying that "/server/storage/models/downloaded is the default location that your model files should be at". However the directory structure for the anythingllm-desktop installation is understandably very different from the source/dev directory structure in GitHub. I'm very confused where I'm supposed to put models for the "Native" LLM Preference. What I'm referring to specifically is neither a dev-build nor a docker build. I'm referring to the very exciting "AnythingLLM for desktop" public beta. Any help at all is greatly appreciated! I can't find a similar folder for where to put downloaded gguf models so I just get the message "-- waiting for models --" ![image](https://github.com/Mintplex-Labs/anything-llm/assets/46267284/538fd031-55d5-40e9-bf60-45a2e68b1052) For reference, this is the directory structure/files I'm looking at in the anythingllm-desktop installation: ![image](https://github.com/Mintplex-Labs/anything-llm/assets/46267284/de74cb8f-f675-4640-b9b1-e5f2f1ca1eb3) ![image](https://github.com/Mintplex-Labs/anything-llm/assets/46267284/da209036-ee21-47da-bd32-78e9bf6229ea)
yindo added the documentation label 2026-02-22 18:19:21 -05:00
yindo closed this issue 2026-02-22 18:19:21 -05:00
yindo commented 2026-02-22 18:19:22 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 12, 2024):

We are going to deprecate Native LLM support for AnythingLLM desktop next update because the node-llama-cpp does not pre-build for the underlying operating system which seems to cause problems with inferencing. It should have been removed from the beta but that was overlooked.

We would really recommend using LMStudio, LocalAI or Ollama for LocalLLM support as their build and interface is much more stable 👍

@timothycarambat commented on GitHub (Feb 12, 2024): We are going to deprecate Native LLM support for AnythingLLM desktop next update because the node-llama-cpp does not pre-build for the underlying operating system which seems to cause problems with inferencing. It should have been removed from the beta but that was overlooked. We would really recommend using LMStudio, LocalAI or Ollama for LocalLLM support as their build and interface is much more stable 👍
yindo changed title from [DOCS]: AnythingLLM for desktop beta - folder for storing downloaded LLAMA GGUF LLMs from HuggingFace to [GH-ISSUE #712] [DOCS]: AnythingLLM for desktop beta - folder for storing downloaded LLAMA GGUF LLMs from HuggingFace 2026-06-05 14:35:10 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label documentation
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#409
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?