Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5103] [BUG]: Missing GPU Dependencies in V1.11.0 Installation Forces CPU Inference #4916

New Issue
Closed
opened 2026-06-05 14:50:51 -04:00 by yindo · 2 comments
yindo commented 2026-06-05 14:50:51 -04:00
Owner
Copy Link
Copy Source

Originally created by @ouy160 on GitHub (Mar 2, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5103

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

When installing version 1.9.1 on Windows 11 x64, the corresponding GPU dependencies can be installed. However, during the installation of version 1.11.0, there are no GPU dependency items to install, resulting in CPU-based inference, while the goal is to use GPU inference.

Image

Are there known steps to reproduce?

No response

Originally created by @ouy160 on GitHub (Mar 2, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5103 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? When installing version 1.9.1 on Windows 11 x64, the corresponding GPU dependencies can be installed. However, during the installation of version 1.11.0, there are no GPU dependency items to install, resulting in CPU-based inference, while the goal is to use GPU inference. <img width="740" height="217" alt="Image" src="https://github.com/user-attachments/assets/7399756c-3bac-43ee-a3db-fd2ce1ec2f50" /> ### Are there known steps to reproduce? _No response_
yindo added the possible bug label 2026-06-05 14:50:51 -04:00
yindo closed this issue 2026-06-05 14:50:51 -04:00
yindo commented 2026-06-05 14:50:52 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 2, 2026):

This is not the case, we download the CUDA support at runtime, not pre-pack into the app and it is possible the download or unzip failed. You can check the storage in the engines/ollama and everything should be in there.

You are on a 3090, which is certainly supported. You should check that directory to ensure the file were downloaded at install time, but they should be there. should be in engines/ollama/lib/ollama and then it should cuda_xx and all that.

Can confirm this is still working

<!-- gh-comment-id:3985127351 --> @timothycarambat commented on GitHub (Mar 2, 2026): This is not the case, we download the CUDA support at runtime, not pre-pack into the app and it is possible the download or unzip failed. You can check the [storage](https://docs.anythingllm.com/installation-desktop/storage) in the `engines/ollama` and everything should be in there. You are on a 3090, which is certainly supported. You should check that directory to ensure the file were downloaded at install time, but they should be there. should be in `engines/ollama/lib/ollama` and then it should `cuda_xx` and all that. Can confirm this is still working
yindo commented 2026-06-05 14:50:52 -04:00
Author
Owner
Copy Link
Copy Source

@ouy160 commented on GitHub (Mar 2, 2026):

This is not the case, we download the CUDA support at runtime, not pre-pack into the app and it is possible the download or unzip failed. You can check the storage in the and everything should be in there.engines/ollama

You are on a 3090, which is certainly supported. You should check that directory to ensure the file were downloaded at install time, but they should be there. should be in and then it should and all that.engines/ollama/lib/ollama``cuda_xx

Can confirm this is still working

But the issue is that it did not download the CUDA support at runtime.
This folder is empty：engines/ollama/lib/ollama``cuda_xx,It does not include the CUDA support libraries.

<!-- gh-comment-id:3988297219 --> @ouy160 commented on GitHub (Mar 2, 2026): > This is not the case, we download the CUDA support at runtime, not pre-pack into the app and it is possible the download or unzip failed. You can check the [storage](https://docs.anythingllm.com/installation-desktop/storage) in the and everything should be in there.`engines/ollama` > > You are on a 3090, which is certainly supported. You should check that directory to ensure the file were downloaded at install time, but they should be there. should be in and then it should and all that.`engines/ollama/lib/ollama``cuda_xx` > > Can confirm this is still working But the issue is that it did not download the CUDA support at runtime. This folder is empty：`engines/ollama/lib/ollama``cuda_xx`,It does not include the CUDA support libraries.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#4916
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?