Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1011] Use Ollama to reload the model each time #622

New Issue
Closed
opened 2026-02-22 18:20:31 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:20:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @17Reset on GitHub (Apr 2, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1011

Originally assigned to: @timothycarambat on GitHub.

When I use ollama as LLM preference in Anythingllm, every time I ask a question, the model in ollama will be reloaded, which will take a lot of time. But when I use ollama's api alone to make a model call to ollama, ollama can load the model into the GPU's memory, and asking the question at the next time will be very fast.

Originally created by @17Reset on GitHub (Apr 2, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1011 Originally assigned to: @timothycarambat on GitHub. When I use ollama as LLM preference in Anythingllm, every time I ask a question, the model in ollama will be reloaded, which will take a lot of time. But when I use ollama's api alone to make a model call to ollama, ollama can load the model into the GPU's memory, and asking the question at the next time will be very fast.
yindo added the enhancementIntegration Requestcore-team-only labels 2026-02-22 18:20:31 -05:00
yindo closed this issue 2026-02-22 18:20:31 -05:00
yindo commented 2026-02-22 18:20:32 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 2, 2024):

Are you using Ollama via the regular integration or the built-in Ollama on desktop?

Seems like this can be solved by passing mlock during chat requests

@timothycarambat commented on GitHub (Apr 2, 2024): Are you using Ollama via the regular integration or the built-in Ollama on desktop? Seems like this can be solved by passing `mlock` during chat requests
yindo commented 2026-02-22 18:20:32 -05:00
Author
Owner
Copy Link
Copy Source

@17Reset commented on GitHub (Apr 2, 2024):

My solution was to start two ollama api's and put LLM and Embedding in two separate api's so that it doesn't cause memory resident interruptions in LLM.

@17Reset commented on GitHub (Apr 2, 2024): My solution was to start two ollama api's and put LLM and Embedding in two separate api's so that it doesn't cause memory resident interruptions in LLM.
yindo changed title from Use Ollama to reload the model each time to [GH-ISSUE #1011] Use Ollama to reload the model each time 2026-06-05 14:36:20 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label Integration Request core-team-only enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#622
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?