Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4407] [BUG]: 500 image input is not supported #2806

New Issue
Closed
opened 2026-02-22 18:31:21 -05:00 by yindo · 8 comments
yindo commented 2026-02-22 18:31:21 -05:00
Owner
Copy Link
Copy Source

Originally created by @RafaelLSa on GitHub (Sep 18, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4407

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

I was setting a docker to use and was everything fine, consuming the AI published by the Docker Model Runner as Local AI. But when I tried to feed image file to get a description got this message.

[backend] info: [TokenManager] Returning existing instance for model: ai/gemma3:4B-Q4_K_M
[backend] error: Error: 500 image input is not supported - hint: if this is unexpected, you may need to provide the mmproj
at APIError.generate (/app/server/node_modules/openai/error.js:66:20)
at OpenAI.makeStatusError (/app/server/node_modules/openai/core.js:302:33)
at OpenAI.makeRequest (/app/server/node_modules/openai/core.js:346:30)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async LLMPerformanceMonitor.measureStream (/app/server/utils/helpers/chat/LLMPerformanceMonitor.js:67:20)
at async LocalAiLLM.streamGetChatCompletion (/app/server/utils/AiProviders/localAi/index.js:158:35)
at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:265:20)
at async /app/server/endpoints/chat.js:63:9

The chat don't recognize the uploaded image, and halucinates when I ask about it.

Are there known steps to reproduce?

  1. Publish AnythingLLM on a container.
  2. Publish a Vision Model on Docker Model Runner
  3. Set AnythingLLM to consume the Model through Local AI engine
  4. Drag an image to AnythingLLM chat and ask it to describe
Originally created by @RafaelLSa on GitHub (Sep 18, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4407 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? I was setting a docker to use and was everything fine, consuming the AI published by the Docker Model Runner as Local AI. But when I tried to feed image file to get a description got this message. [backend] info: [TokenManager] Returning existing instance for model: ai/gemma3:4B-Q4_K_M [backend] error: Error: 500 image input is not supported - hint: if this is unexpected, you may need to provide the mmproj at APIError.generate (/app/server/node_modules/openai/error.js:66:20) at OpenAI.makeStatusError (/app/server/node_modules/openai/core.js:302:33) at OpenAI.makeRequest (/app/server/node_modules/openai/core.js:346:30) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async LLMPerformanceMonitor.measureStream (/app/server/utils/helpers/chat/LLMPerformanceMonitor.js:67:20) at async LocalAiLLM.streamGetChatCompletion (/app/server/utils/AiProviders/localAi/index.js:158:35) at async streamChatWithWorkspace (/app/server/utils/chats/stream.js:265:20) at async /app/server/endpoints/chat.js:63:9 The chat don't recognize the uploaded image, and halucinates when I ask about it. ### Are there known steps to reproduce? 1. Publish AnythingLLM on a container. 2. Publish a Vision Model on Docker Model Runner 3. Set AnythingLLM to consume the Model through Local AI engine 4. Drag an image to AnythingLLM chat and ask it to describe
yindo added the possible buginvestigating labels 2026-02-22 18:31:21 -05:00
yindo closed this issue 2026-02-22 18:31:21 -05:00
yindo commented 2026-02-22 18:31:21 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Sep 18, 2025):

consuming the AI published by the Docker Model Runner as Local AI

Which provider are you using? We dont have a dedicated Docker Model Runner inference yet (#4391), but it is OpenAI spec - so it should work with image inputs.

Are you using the Local AI provider? This provider is actually the name of a project also called LocalAI, not "local ai" as a concept.

In this case, you may want to use the "Generic OpenAI" provider found at the end of the provider list

@timothycarambat commented on GitHub (Sep 18, 2025): > consuming the AI published by the Docker Model Runner as Local AI Which provider are you using? We dont have a dedicated Docker Model Runner inference yet (#4391), but it is OpenAI spec - so it should work with image inputs. Are you using the `Local AI` provider? This provider is actually the name of [a project also called LocalAI](https://localai.io/), not "local ai" as a concept. In this case, you may want to use the "Generic OpenAI" provider found at the end of the provider list
yindo commented 2026-02-22 18:31:21 -05:00
Author
Owner
Copy Link
Copy Source

@RafaelLSa commented on GitHub (Sep 18, 2025):

I was using the Local AI provider, sorry for the mistake. 😅
Changed provider for the Generic OpenAI, but got the same error.
The Docker volume is set to /app/server/storage as shown in the tutorial too.

@RafaelLSa commented on GitHub (Sep 18, 2025): I was using the Local AI provider, sorry for the mistake. 😅 Changed provider for the Generic OpenAI, but got the same error. The Docker volume is set to /app/server/storage as shown in the tutorial too.
yindo commented 2026-02-22 18:31:21 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Sep 19, 2025):

After doing some digging to figure out how we can solve this, it actually seems like this is just an incompatibility with the model you are using or an incompatibility with docker model runner.

I was able to find an issue someone opened on the docker model runner repo: https://github.com/docker/model-runner/issues/122#issuecomment-3191809244 and they said that the issue is resolved but from my testing, I still am unable to get multimodal chats to work with this.

The only other place I was able to find this same error was here and it seems like if you're able to load an mmproj model using docker model runner but I was unable to get this working myself.

For the time being, I'll open another issue mentioning this on the docker model runner repo and will close this since this is an issue with model runner, not AnythingLLM.

@shatfield4 commented on GitHub (Sep 19, 2025): After doing some digging to figure out how we can solve this, it actually seems like this is just an incompatibility with the model you are using or an incompatibility with docker model runner. I was able to find an issue someone opened on the docker model runner repo: https://github.com/docker/model-runner/issues/122#issuecomment-3191809244 and they said that the issue is resolved but from my testing, I still am unable to get multimodal chats to work with this. The only other place I was able to find this same error was [here](https://huggingface.co/QuixiAI/Devstral-Vision-Small-2507/discussions/1) and it seems like if you're able to load an mmproj model using docker model runner but I was unable to get this working myself. For the time being, I'll open another issue mentioning this on the docker model runner repo and will close this since this is an issue with model runner, not AnythingLLM.
yindo commented 2026-02-22 18:31:21 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Sep 19, 2025):

Can confirm, at least the gemma3 multimodal is currently broken. The internal llama.cpp engine they use needs to leverage it - might be that gemma image is out of date and needs to be updated? Was last pushed ~6 months ago

Edit: tested gemma3n and still go this :/

@timothycarambat commented on GitHub (Sep 19, 2025): Can confirm, at least the gemma3 multimodal is currently broken. The internal llama.cpp engine they use needs to leverage it - might be that gemma image is out of date and needs to be updated? Was last pushed ~6 months ago Edit: tested `gemma3n` and still go this :/
yindo commented 2026-02-22 18:31:21 -05:00
Author
Owner
Copy Link
Copy Source

@RafaelLSa commented on GitHub (Sep 19, 2025):

Need to say I'm impressed! You guys are working hard!
😃
TY for all the effort!

@RafaelLSa commented on GitHub (Sep 19, 2025): Need to say I'm impressed! You guys are working hard! 😃 TY for all the effort!
yindo commented 2026-02-22 18:31:22 -05:00
Author
Owner
Copy Link
Copy Source

@ericcurtin commented on GitHub (Sep 20, 2025):

This is coming soon to docker model runner, it's next priority

@ericcurtin commented on GitHub (Sep 20, 2025): This is coming soon to docker model runner, it's next priority
yindo commented 2026-02-22 18:31:22 -05:00
Author
Owner
Copy Link
Copy Source

@RafaelLSa commented on GitHub (Sep 21, 2025):

I don't know if it was the new Gemma image or the last Docker update. But it's working again. Ty! Now I know where to cry next time. 😎

@RafaelLSa commented on GitHub (Sep 21, 2025): I don't know if it was the new Gemma image or the last Docker update. But it's working again. Ty! Now I know where to cry next time. 😎
yindo commented 2026-02-22 18:31:22 -05:00
Author
Owner
Copy Link
Copy Source

@ericcurtin commented on GitHub (Sep 22, 2025):

@RafaelLSa ha yes, thank @ilopezluna he's working on this

@ericcurtin commented on GitHub (Sep 22, 2025): @RafaelLSa ha yes, thank @ilopezluna he's working on this
yindo changed title from [BUG]: 500 image input is not supported to [GH-ISSUE #4407] [BUG]: 500 image input is not supported 2026-06-05 14:48:40 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2806
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?