Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4596] [BUG]: Cannot load Qwen3VL via Ollama #2924

New Issue
Closed
opened 2026-02-22 18:31:52 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:31:52 -05:00
Owner
Copy Link
Copy Source

Originally created by @98h398hrpohpoai on GitHub (Oct 30, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4596

How are you running AnythingLLM?

Docker (local)

What happened?

Trying to run any of the qwen3vl 30b or 32b models results in the following using ollama 0.12.7:
Could not respond to message.
do load request: Post "http://127.0.0.1:45241/load": EOF

Coder 30b works fine, and the qwen3vl models work fine in the command line and with open-webui.

Ollama CLI

ollama run huihui_ai/qwen3-vl-abliterated:32b-thinking-fp16 hello
Thinking...
Okay, the user just said "hello". I need to respond in a friendly and helpful way. Since it's a greeting, I should acknowledge it and offer assistance.

First, I'll say "Hello! How can I help you today?" That's a common and polite response. It's open-ended so the user can state their need. I should keep it simple and not add too much
extra info since they just said hello. No need for additional questions yet. Just a warm, welcoming reply to encourage them to explain what they need.
...done thinking.

Hello! How can I help you today? 😊

Logs

2025-10-30T19:39:42.866409-04:00 emacx ollama[3211413]: time=2025-10-30T19:39:42.866-04:00 level=INFO source=sched.go:446 msg="Load failed" model=/usr/share/ollama/.ollama/models/blobs/sha256-4f8e37dcef3efa37ed2a33aa80b1baf8ca665d8919e0b921123b67a39fd51002 error="do load request: Post "http://127.0.0.1:45241/load": EOF"
2025-10-30T19:39:42.922416-04:00 emacx ollama[3211413]: time=2025-10-30T19:39:42.922-04:00 level=ERROR source=server.go:273 msg="llama runner terminated" error="exit status 2"
2025-10-30T19:39:42.922540-04:00 emacx ollama[3211413]: [GIN] 2025/10/30 - 19:39:42 | 500 | 3.984587308s | 172.17.0.3 | POST "/api/chat"
2025-10-30T19:40:35.248232-04:00 emacx ollama[3211413]: time=2025-10-30T19:40:35.247-04:00 level=INFO source=download.go:295 msg="f7f402c51373 part 41 attempt 0 failed: unexpected EOF, retrying in 1s"

Originally created by @98h398hrpohpoai on GitHub (Oct 30, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4596 ### How are you running AnythingLLM? Docker (local) ### What happened? Trying to run any of the qwen3vl 30b or 32b models results in the following using ollama 0.12.7: Could not respond to message. do load request: Post "http://127.0.0.1:45241/load": EOF Coder 30b works fine, and the qwen3vl models work fine in the command line and with open-webui. ### Ollama CLI ollama run huihui_ai/qwen3-vl-abliterated:32b-thinking-fp16 hello Thinking... Okay, the user just said "hello". I need to respond in a friendly and helpful way. Since it's a greeting, I should acknowledge it and offer assistance. First, I'll say "Hello! How can I help you today?" That's a common and polite response. It's open-ended so the user can state their need. I should keep it simple and not add too much extra info since they just said hello. No need for additional questions yet. Just a warm, welcoming reply to encourage them to explain what they need. ...done thinking. Hello! How can I help you today? 😊 ### Logs 2025-10-30T19:39:42.866409-04:00 emacx ollama[3211413]: time=2025-10-30T19:39:42.866-04:00 level=INFO source=sched.go:446 msg="Load failed" model=/usr/share/ollama/.ollama/models/blobs/sha256-4f8e37dcef3efa37ed2a33aa80b1baf8ca665d8919e0b921123b67a39fd51002 error="do load request: Post \"http://127.0.0.1:45241/load\": EOF" 2025-10-30T19:39:42.922416-04:00 emacx ollama[3211413]: time=2025-10-30T19:39:42.922-04:00 level=ERROR source=server.go:273 msg="llama runner terminated" error="exit status 2" 2025-10-30T19:39:42.922540-04:00 emacx ollama[3211413]: [GIN] 2025/10/30 - 19:39:42 | 500 | 3.984587308s | 172.17.0.3 | POST "/api/chat" 2025-10-30T19:40:35.248232-04:00 emacx ollama[3211413]: time=2025-10-30T19:40:35.247-04:00 level=INFO source=download.go:295 msg="f7f402c51373 part 41 attempt 0 failed: unexpected EOF, retrying in 1s"
yindo added the possible bug label 2026-02-22 18:31:52 -05:00
yindo closed this issue 2026-02-22 18:31:52 -05:00
yindo commented 2026-02-22 18:31:52 -05:00
Author
Owner
Copy Link
Copy Source

@98h398hrpohpoai commented on GitHub (Oct 31, 2025):

This was due to llm being configured in performance mode. Reducing the context window or switching to balance resolved this.

@98h398hrpohpoai commented on GitHub (Oct 31, 2025): This was due to llm being configured in performance mode. Reducing the context window or switching to balance resolved this.
yindo changed title from [BUG]: Cannot load Qwen3VL via Ollama to [GH-ISSUE #4596] [BUG]: Cannot load Qwen3VL via Ollama 2026-06-05 14:49:18 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2924
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?