Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4480] Local set-up quality #2853

New Issue
Closed
opened 2026-02-22 18:31:32 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:31:32 -05:00
Owner
Copy Link
Copy Source

Originally created by @KORSBJERG on GitHub (Oct 3, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4480

My AnythingLLM works well with a cloud-based server with Anthropic/claude-sonnet-4-20250514 and embedding via OpenAI/text-embedding-3-large.
Basically, I have thousands of chunks of text in a work area, and I need accurate answers to searches, before daring to make use of the set-up for “real” AI tasks.
When I move over to an Ollama-based private (which is necessary) installment, I cannot find anything that works just as well. Depending on the model I select, answers are incomplete, nonsense or hallucinations. Some models do not even investigate the work area, even though this is exclusively the very task.
I have 24 GB RAM in the private setup.
What can I do to make it work just as well as in the cloud-based setup?

Please excuse if this is not the place for such support issues. It is my first time here

Originally created by @KORSBJERG on GitHub (Oct 3, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4480 My AnythingLLM works well with a cloud-based server with Anthropic/claude-sonnet-4-20250514 and embedding via OpenAI/text-embedding-3-large. Basically, I have thousands of chunks of text in a work area, and I need accurate answers to searches, before daring to make use of the set-up for “real” AI tasks. When I move over to an Ollama-based private (which is necessary) installment, I cannot find anything that works just as well. Depending on the model I select, answers are incomplete, nonsense or hallucinations. Some models do not even investigate the work area, even though this is exclusively the very task. I have 24 GB RAM in the private setup. What can I do to make it work just as well as in the cloud-based setup? Please excuse if this is not the place for such support issues. It is my first time here
yindo closed this issue 2026-02-22 18:31:32 -05:00
yindo commented 2026-02-22 18:31:32 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Oct 3, 2025):

So this isnt the place for this kind of discussion - however, I can still help in this thread.

Basically, everything comes down to the model. You mention RAM but not if you have a GPU. So what are these models running on? Additionally, what specific model are you using from Ollama.

This could be a quantization + hardware + model + context window issue. Ollama will use a very small context window wrt to your available hardware unless you tell it to use the whole window (which can be slow!). We allow you to do this in the Ollama LLM connection > Advanced Settings > Performance Mode > Max

@timothycarambat commented on GitHub (Oct 3, 2025): So this isnt the place for this kind of discussion - however, I can still help in this thread. Basically, everything comes down to the model. You mention RAM but not if you have a GPU. So what are these models running on? Additionally, what specific model are you using from Ollama. This could be a quantization + hardware + model + context window issue. Ollama will use a very small context window wrt to your available hardware unless you tell it to use the whole window (which can be slow!). We allow you to do this in the Ollama LLM connection > Advanced Settings > Performance Mode > Max
yindo changed title from Local set-up quality to [GH-ISSUE #4480] Local set-up quality 2026-06-05 14:48:53 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2853
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?