Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1899] Old GPU (Quadro K6000) is not used #1234

New Issue
Closed
opened 2026-02-22 18:23:52 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:23:52 -05:00
Owner
Copy Link
Copy Source

Originally created by @LLMuser on GitHub (Jul 19, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1899

How are you running AnythingLLM?

Docker (local)

What happened?

Running lastest AnythingLLM on Docker in Linux Ubuntu 20.4 Server
Needed to use Lance_revert image though to prevent crashes
NVIDIA Driver Version: 470.256.02
CUDA Version: 11.4

nvidia-smi gives no errors but no processes are found = AnythingLLM is not using the GPU :-(

I know it´s an outdated GPU but maybe there is a workaround?

Are there known steps to reproduce?

docker pull mintplexlabs/anythingllm:lancedb_revert

export STORAGE_LOCATION=/mnt &&
mkdir -p $STORAGE_LOCATION &&
touch "$STORAGE_LOCATION/.env" &&
docker run -d -p 3001:3001
--cap-add SYS_ADMIN
--gpus all
--name anythingllm
-v ${STORAGE_LOCATION}:/app/server/storage
-v ${STORAGE_LOCATION}/.env:/app/server/.env
-e STORAGE_DIR="/app/server/storage"
mintplexlabs/anythingllm:lancedb_revert

Originally created by @LLMuser on GitHub (Jul 19, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1899 ### How are you running AnythingLLM? Docker (local) ### What happened? Running lastest AnythingLLM on Docker in Linux Ubuntu 20.4 Server Needed to use Lance_revert image though to prevent crashes NVIDIA Driver Version: 470.256.02 CUDA Version: 11.4 nvidia-smi gives no errors but no processes are found = AnythingLLM is not using the GPU :-( I know it´s an outdated GPU but maybe there is a workaround? ### Are there known steps to reproduce? docker pull mintplexlabs/anythingllm:lancedb_revert export STORAGE_LOCATION=/mnt && \ mkdir -p $STORAGE_LOCATION && \ touch "$STORAGE_LOCATION/.env" && \ docker run -d -p 3001:3001 \ --cap-add SYS_ADMIN \ --gpus all \ --name anythingllm \ -v ${STORAGE_LOCATION}:/app/server/storage \ -v ${STORAGE_LOCATION}/.env:/app/server/.env \ -e STORAGE_DIR="/app/server/storage" \ mintplexlabs/anythingllm:lancedb_revert
yindo added the question label 2026-02-22 18:23:52 -05:00
yindo closed this issue 2026-02-22 18:23:52 -05:00
yindo commented 2026-02-22 18:23:52 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 19, 2024):

This would be entirely dependent on the LLM provider you are using - are you loading a model manually into /models or using something external like Ollama/LMstudio/LocalAI?

There is really nothing in AnythingLLM that would use the GPU unless you are using the "native" llama-cpp LLM, which is going to be deprecated soon because of these kinds of issues with CUDA detection and needing to rebuild the binary for it

@timothycarambat commented on GitHub (Jul 19, 2024): This would be entirely dependent on the LLM provider you are using - are you loading a model manually into `/models` or using something external like Ollama/LMstudio/LocalAI? There is really nothing in AnythingLLM that would use the GPU unless you are using the "native" llama-cpp LLM, which is going to be deprecated soon because of these kinds of issues with CUDA detection and needing to rebuild the binary for it
yindo commented 2026-02-22 18:23:52 -05:00
Author
Owner
Copy Link
Copy Source

@LLMuser commented on GitHub (Jul 22, 2024):

Thank you @timothycarambat. I tried both, native and external LLMs. But good to know that if I have an external LLM running, AnythingLLM does not need GPU support...

@LLMuser commented on GitHub (Jul 22, 2024): Thank you @timothycarambat. I tried both, native and external LLMs. But good to know that if I have an external LLM running, AnythingLLM does not need GPU support...
yindo changed title from Old GPU (Quadro K6000) is not used to [GH-ISSUE #1899] Old GPU (Quadro K6000) is not used 2026-06-05 14:39:40 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1234
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?