langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 04:25:29 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Set the Vision model as a tool #1145

New Issue
Closed
opened 2026-02-21 17:30:12 -05:00 by yindo · 2 comments
yindo commented 2026-02-21 17:30:12 -05:00
Owner
Copy Link
Copy Source

Originally created by @richards199999 on GitHub (Mar 2, 2024).

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • Pleas do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing?

Only the model that have vision capability can receive images (i.e. gpt-4-vision-preview), however, they can't use Function Calling natively.

2. Describe the feature you'd like to see

The possible implementation:
When the user send an image, the image can be sent with the original prompt to the Vision model, or set a pre-prompt for this tool. e.g.:

You are an image-to-text model./nYou MUST describe the image *in great details*./nYou have to capture *every little details*./nIf the image contain text, then you should transcribe them originally.

Then the model can receive the detailed description.

3. How will this feature improve your workflow or experience?

This can let us send images to the models that do not natively support it.

4. Additional context or comments

No response

5. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @richards199999 on GitHub (Mar 2, 2024). ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] Pleas do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Only the model that have vision capability can receive images (i.e. gpt-4-vision-preview), however, they can't use Function Calling natively. ### 2. Describe the feature you'd like to see The possible implementation: When the user send an image, the image can be sent with the original prompt to the Vision model, or set a pre-prompt for this tool. e.g.: ```plain text You are an image-to-text model./nYou MUST describe the image *in great details*./nYou have to capture *every little details*./nIf the image contain text, then you should transcribe them originally. ``` Then the model can receive the detailed description. ### 3. How will this feature improve your workflow or experience? This can let us send images to the models that do not natively support it. ### 4. Additional context or comments _No response_ ### 5. Can you help us with this feature? - [X] I am interested in contributing to this feature.
yindo added the 💪 enhancement🤖 feat:agent labels 2026-02-21 17:30:12 -05:00
yindo closed this issue 2026-02-21 17:30:12 -05:00
yindo commented 2026-02-21 17:30:13 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Mar 2, 2024):

Thank you for your interest in contributing to this feature! Your proposed implementation and description of how it will improve the workflow are very helpful. We appreciate your willingness to contribute, and we look forward to reviewing your pull request.

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

@dosubot[bot] commented on GitHub (Mar 2, 2024): <!-- Answer --> Thank you for your interest in contributing to this feature! Your proposed implementation and description of how it will improve the workflow are very helpful. We appreciate your willingness to contribute, and we look forward to reviewing your pull request. <!-- Dosu Comment Footer --> <details open> <summary> #### About Dosu </summary> This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot. </details>
yindo commented 2026-02-21 17:30:13 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Mar 17, 2024):

Close due to it's no longer active, if you have any questions, you can reopen it.

@github-actions[bot] commented on GitHub (Mar 17, 2024): Close due to it's no longer active, if you have any questions, you can reopen it.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 💪 enhancement 🤖 feat:agent
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#1145
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?