Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3395] [FEAT]: Vision Models and embedding Documents #2188

New Issue
Closed
opened 2026-02-22 18:28:32 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:28:32 -05:00
Owner
Copy Link
Copy Source

Originally created by @kalfbz on GitHub (Mar 4, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3395

Originally assigned to: @shatfield4, @timothycarambat on GitHub.

What would you like to see?

When using a vision model like gemini-2.0-flash, it can extract information from images. However, the current issue is that when an image or document is uploaded in the chatbox, it gets embedded into the workspace instead of being sent to the model for interpretation.

Would it be possible to modify this behavior so that uploading an image or document in the chatbox sends it to the model for interpretation, while files intended for embedding into the workspace should be uploaded via the upload button in the left-side workspace?

Originally created by @kalfbz on GitHub (Mar 4, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3395 Originally assigned to: @shatfield4, @timothycarambat on GitHub. ### What would you like to see? When using a vision model like gemini-2.0-flash, it can extract information from images. However, the current issue is that when an image or document is uploaded in the chatbox, it gets embedded into the workspace instead of being sent to the model for interpretation. Would it be possible to modify this behavior so that uploading an image or document in the chatbox sends it to the model for interpretation, while files intended for embedding into the workspace should be uploaded via the upload button in the left-side workspace?
yindo added the enhancementfeature request labels 2026-02-22 18:28:32 -05:00
yindo closed this issue 2026-02-22 18:28:32 -05:00
yindo commented 2026-02-22 18:28:32 -05:00
Author
Owner
Copy Link
Copy Source

@th3f001 commented on GitHub (Mar 7, 2025):

...or maybe decouple the functionality from the workspace embedder (that I would assume is meant to be used with text-based files to be organized in chunks before being sent to the model)...and make a Custom Skill out of it?

So at that point the user would have 2 separate flows:
-> standard files to RAG upon -> workspace file embedder
-> image files to be passed to a Vision-enabled model -> Agent with dedicated Skill

Following the same approach we could also have a 3rd flow:
-> PDFs and other mixed-contents files to be passed to an OCR-enabled model (like latest Mistral-OCR) -> Agent with dedicated skill

@th3f001 commented on GitHub (Mar 7, 2025): ...or maybe decouple the functionality from the workspace embedder (that I would assume is meant to be used with text-based files to be organized in chunks before being sent to the model)...and make a Custom Skill out of it? So at that point the user would have 2 separate flows: -> standard files to RAG upon -> workspace file embedder -> image files to be passed to a Vision-enabled model -> Agent with dedicated Skill Following the same approach we could also have a 3rd flow: -> PDFs and other mixed-contents files to be passed to an OCR-enabled model (like latest Mistral-OCR) -> Agent with dedicated skill
yindo commented 2026-02-22 18:28:32 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Oct 1, 2025):

This is an old issue, but is now the case as of the writing of this comment

@timothycarambat commented on GitHub (Oct 1, 2025): This is an old issue, but is now the case as of the writing of this comment
yindo changed title from [FEAT]: Vision Models and embedding Documents to [GH-ISSUE #3395] [FEAT]: Vision Models and embedding Documents 2026-06-05 14:45:05 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2188
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?