Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3304] [BUG]: Returning existing instance for model: deepseek-r1:14b llama runner process has terminated: CUDA error #2124

New Issue
Closed
opened 2026-02-22 18:28:14 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:28:14 -05:00
Owner
Copy Link
Copy Source

Originally created by @lizhichao999 on GitHub (Feb 21, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3304

How are you running AnythingLLM?

Docker (local)

What happened?

2025-02-21 14:55:08 [backend] info: [NativeEmbedder] Embedded Chunk 1 of 1
2025-02-21 14:55:08 [backend] info: [TokenManager] Returning existing instance for model: deepseek-r1:14b
2025-02-21 14:55:21 [backend] error: ResponseError: llama runner process has terminated: CUDA error
2025-02-21 14:55:21 at checkOk (/app/server/node_modules/ollama/dist/browser.cjs:74:9)
2025-02-21 14:55:21 at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2025-02-21 14:55:21 at async post (/app/server/node_modules/ollama/dist/browser.cjs:120:3)
2025-02-21 14:55:21 at async Ollama.processStreamableRequest (/app/server/node_modules/ollama/dist/browser.cjs:231:25)
2025-02-21 14:55:21 at async LLMPerformanceMonitor.measureStream (/app/server/utils/helpers/chat/LLMPerformanceMonitor.js:67:20)
2025-02-21 14:55:21 at async OllamaAILLM.streamGetChatCompletion (/app/server/utils/AiProviders/ollama/index.js:186:35)
2025-02-21 14:55:21 at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:248:20)

Qwen2.5 can runing
2025-02-21 14:55:47 [backend] info: [NativeEmbedder] Embedded Chunk 1 of 1
2025-02-21 14:55:47 [backend] info: [TokenManager] Initialized new TokenManager instance for model: qwen2.5:latest
2025-02-21 14:55:53 [backend] info: [TELEMETRY SENT] {"event":"sent_chat","distinctId":"9ebb97ae-0a42-4c33-812c-aed12413bcc2","properties":{"multiUserMode":true,"LLMSelection":"ollama","Embedder":"native","VectorDbSelection":"lancedb","multiModal":false,"TTSSelection":"native","runtime":"docker"}}
2025-02-21 14:55:53 [backend] info: [Event Logged] - sent_chat
2025-02-21 14:56:18 [backend] info: [NativeEmbedder] Initialized
2025-02-21 14:56:18 [backend] info: [Ollama] OllamaAILLM initialized with
2025-02-21 14:56:18 model: qwen2.5:latest

You are running deepseek-r1:14b locally using ollama, and you can get inference results in the ollama client, but an error is reported in anythingLLM.

what can i do

Are there known steps to reproduce?

You are running deepseek-r1:14b locally using ollama, and you can get inference results in the ollama client, but an error is reported in anythingLLM.

Originally created by @lizhichao999 on GitHub (Feb 21, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3304 ### How are you running AnythingLLM? Docker (local) ### What happened? 2025-02-21 14:55:08 [backend] info: [NativeEmbedder] Embedded Chunk 1 of 1 2025-02-21 14:55:08 [backend] info: [TokenManager] Returning existing instance for model: deepseek-r1:14b 2025-02-21 14:55:21 [backend] error: ResponseError: llama runner process has terminated: CUDA error 2025-02-21 14:55:21 at checkOk (/app/server/node_modules/ollama/dist/browser.cjs:74:9) 2025-02-21 14:55:21 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) 2025-02-21 14:55:21 at async post (/app/server/node_modules/ollama/dist/browser.cjs:120:3) 2025-02-21 14:55:21 at async Ollama.processStreamableRequest (/app/server/node_modules/ollama/dist/browser.cjs:231:25) 2025-02-21 14:55:21 at async LLMPerformanceMonitor.measureStream (/app/server/utils/helpers/chat/LLMPerformanceMonitor.js:67:20) 2025-02-21 14:55:21 at async OllamaAILLM.streamGetChatCompletion (/app/server/utils/AiProviders/ollama/index.js:186:35) 2025-02-21 14:55:21 at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:248:20) Qwen2.5 can runing 2025-02-21 14:55:47 [backend] info: [NativeEmbedder] Embedded Chunk 1 of 1 2025-02-21 14:55:47 [backend] info: [TokenManager] Initialized new TokenManager instance for model: qwen2.5:latest 2025-02-21 14:55:53 [backend] info: [TELEMETRY SENT] {"event":"sent_chat","distinctId":"9ebb97ae-0a42-4c33-812c-aed12413bcc2","properties":{"multiUserMode":true,"LLMSelection":"ollama","Embedder":"native","VectorDbSelection":"lancedb","multiModal":false,"TTSSelection":"native","runtime":"docker"}} 2025-02-21 14:55:53 [backend] info: [Event Logged] - sent_chat 2025-02-21 14:56:18 [backend] info: [NativeEmbedder] Initialized 2025-02-21 14:56:18 [backend] info: [Ollama] OllamaAILLM initialized with 2025-02-21 14:56:18 model: qwen2.5:latest You are running deepseek-r1:14b locally using ollama, and you can get inference results in the ollama client, but an error is reported in anythingLLM. what can i do ### Are there known steps to reproduce? You are running deepseek-r1:14b locally using ollama, and you can get inference results in the ollama client, but an error is reported in anythingLLM.
yindo added the possible bug label 2026-02-22 18:28:14 -05:00
yindo closed this issue 2026-02-22 18:28:14 -05:00
yindo commented 2026-02-22 18:28:14 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 21, 2025):

You should check the ollama server logs for more information on this since the error is coming from ollama and we are just sending the API request - which should not result in a CUDA error

@timothycarambat commented on GitHub (Feb 21, 2025): You should check the ollama server logs for more information on this since the error is coming from ollama and we are just sending the API request - which should not result in a CUDA error
yindo changed title from [BUG]: Returning existing instance for model: deepseek-r1:14b llama runner process has terminated: CUDA error to [GH-ISSUE #3304] [BUG]: Returning existing instance for model: deepseek-r1:14b llama runner process has terminated: CUDA error 2026-06-05 14:44:41 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2124
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?