Vision works in the preview of the Dify workflow, but not in the public URL (Dify versions 1.1.1 to 1.1.3 and GPT-4o Search Preview). #12344

Closed
opened 2026-02-21 19:07:00 -05:00 by yindo · 11 comments
Owner

Originally created by @CTD-Networks-CO-LTD on GitHub (Mar 29, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

v1.1.1, v1.1.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

After confirming that Vision was working as expected in the preview,
I tried asking the same question about the same image in the chat at the public URL.

  • LLM Model
    GPT-4o Search Preview1
  • image file
    Image

  • Questions to ask for LLM
    What is this illustration?
    (In fact, I was typing in Japanese like this. -> このイラストは何ですか?)

✔️ Expected Behavior

I'd like Vision to work for chats with public URLs, not just previews.

Actual Behavior

As you can see, Vision works in the preview, but
when chatting with a public URL, Vision does not work and Search functions instead.

https://github.com/user-attachments/assets/47b81e51-b3bb-443a-87a7-e58e090be30c


  1. In the LLM block, the "dify-plugin-miibo-agent (v0.0.2)" is used, and on the miibo side, the "GPT-4o-Search Preview" model is selected. ↩︎

Originally created by @CTD-Networks-CO-LTD on GitHub (Mar 29, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version v1.1.1, v1.1.3 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce After confirming that Vision was working as expected in the preview, I tried asking the same question about the same image in the chat at the public URL. - LLM Model `GPT-4o Search Preview`[^1] [^1]:In the LLM block, the "[dify-plugin-miibo-agent](https://github.com/cshara1/dify-plugin-miibo-agent) (v0.0.2)" is used, and on the [miibo](https://miibo.ai/) side, the "GPT-4o-Search Preview" model is selected. - image file ![Image](https://github.com/user-attachments/assets/7d43f5f2-e8ab-4300-979c-9e6d2fd6b8f4) - Questions to ask for LLM `What is this illustration?` (In fact, I was typing in Japanese like this. -> `このイラストは何ですか?`) ### ✔️ Expected Behavior I'd like Vision to work for chats with public URLs, not just previews. ### ❌ Actual Behavior As you can see, Vision works in the preview, but when chatting with a public URL, Vision does not work and Search functions instead. https://github.com/user-attachments/assets/47b81e51-b3bb-443a-87a7-e58e090be30c
yindo added the 🐞 bug label 2026-02-21 19:07:00 -05:00
yindo closed this issue 2026-02-21 19:07:00 -05:00
Author
Owner

@dosubot[bot] commented on GitHub (Mar 29, 2025):

@CTD-Networks-CO-LTD Hello there! I'm here to help you with your bug report. I'm a bot that can assist with bug fixes, answer questions, and guide you on becoming a contributor. Let's take a look at the issue you're facing.

To address the issue where Vision works in the preview but not in the public URL environment, you can follow these troubleshooting steps:

  1. Verify Vision Configuration: Ensure that the 'vision' configuration is correctly declared in your YAML file. The presence of an eye icon in the interface indicates that the vision capability is enabled. Without this configuration, the system will not process images [1].

  2. Check Image Upload Method: Make sure that images are uploaded using the upload button near the send message button, not through the workflow step configuration. Uploading images through the workflow step configuration will result in the files array being empty, and the LLM will not reference the image [1].

  3. Ensure URL Accessibility: If using the URL format for image uploads, ensure that FILES_URL is configured with an externally accessible address. This is crucial for the public URL environment to access and process the images [2].

  4. Model Configuration: If you are using a specific model like GPT-4o Search Preview, ensure that the model is correctly configured to support Vision. This might involve additional settings or configurations specific to the model or the plugin you are using.

  5. Update to Latest Version: Ensure you are using the latest version of Dify, as updates may include fixes for known issues related to Vision functionality [2].

By following these steps, you should be able to resolve the issue of Vision not functioning in the public URL environment.

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Mar 29, 2025): <!-- Greeting --> @CTD-Networks-CO-LTD Hello there! I'm here to help you with your bug report. I'm a bot that can assist with bug fixes, answer questions, and guide you on becoming a contributor. Let's take a look at the issue you're facing. <!-- Answer --> To address the issue where Vision works in the preview but not in the public URL environment, you can follow these troubleshooting steps: 1. **Verify Vision Configuration**: Ensure that the 'vision' configuration is correctly declared in your YAML file. The presence of an eye icon in the interface indicates that the vision capability is enabled. Without this configuration, the system will not process images <sup>[[1]](https://github.com/langgenius/dify/issues/16008)</sup>. 2. **Check Image Upload Method**: Make sure that images are uploaded using the upload button near the send message button, not through the workflow step configuration. Uploading images through the workflow step configuration will result in the files array being empty, and the LLM will not reference the image <sup>[[1]](https://github.com/langgenius/dify/issues/16008)</sup>. 3. **Ensure URL Accessibility**: If using the URL format for image uploads, ensure that `FILES_URL` is configured with an externally accessible address. This is crucial for the public URL environment to access and process the images <sup>[[2]](https://github.com/langgenius/dify/issues/16057)</sup>. 4. **Model Configuration**: If you are using a specific model like GPT-4o Search Preview, ensure that the model is correctly configured to support Vision. This might involve additional settings or configurations specific to the model or the plugin you are using. 5. **Update to Latest Version**: Ensure you are using the latest version of Dify, as updates may include fixes for known issues related to Vision functionality <sup>[[2]](https://github.com/langgenius/dify/issues/16057)</sup>. By following these steps, you should be able to resolve the issue of Vision not functioning in the public URL environment. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/8e61a0bc-3d48-4207-a1c4-e5951e792e9b?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/8e61a0bc-3d48-4207-a1c4-e5951e792e9b?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/8e61a0bc-3d48-4207-a1c4-e5951e792e9b?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/8e61a0bc-3d48-4207-a1c4-e5951e792e9b?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/8e61a0bc-3d48-4207-a1c4-e5951e792e9b?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/8e61a0bc-3d48-4207-a1c4-e5951e792e9b?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/8e61a0bc-3d48-4207-a1c4-e5951e792e9b?feedback_type=other)</sup>
Author
Owner

@CTD-Networks-CO-LTD commented on GitHub (Mar 29, 2025):

Hi, @dosu,

  1. Verify Vision Configuration:

-> This is correctly set to Vision because it works correctly in the preview.

  1. Check Image Upload Method:

-> This is also set correctly. The upload button works in both the preview and the chat with the public URL.

  1. Ensure URL Accessibility:

-> This issue is with local uploads, so URL accessibility is not affected.

  1. Model Configuration:

-> GPT-4o Search Preview works with Vision in the preview, so Vision is supported.

  1. Update to Latest Version:

-> I updated Dify to the latest version at the moment, 1.1.3, but the same problem occurred.

@CTD-Networks-CO-LTD commented on GitHub (Mar 29, 2025): Hi, @dosu, > 1. Verify Vision Configuration: -> This is correctly set to Vision because it works correctly in the preview. <br> > 2. Check Image Upload Method: -> This is also set correctly. The upload button works in both the preview and the chat with the public URL. <br> > 3. Ensure URL Accessibility: -> This issue is with local uploads, so URL accessibility is not affected. <br> > 4. Model Configuration: -> `GPT-4o Search Preview` works with Vision in the preview, so Vision is supported. <br> > 5. Update to Latest Version: -> I updated Dify to the latest version at the moment, 1.1.3, but the same problem occurred. <br>
Author
Owner

@dosubot[bot] commented on GitHub (Mar 29, 2025):

The issue with Vision not working in the public URL environment, despite correct configuration and updates, is likely due to the fact that Dify's Vision LLM can only process images that are uploaded using the upload button near the send message button. Images uploaded through workflow steps are not supported, which is the intended behavior of the system. Therefore, to use vision capabilities successfully, you should include the image directly in the user prompt rather than attempting to upload it through workflow steps [1].

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Mar 29, 2025): <!-- Answer --> The issue with Vision not working in the public URL environment, despite correct configuration and updates, is likely due to the fact that Dify's Vision LLM can only process images that are uploaded using the upload button near the send message button. Images uploaded through workflow steps are not supported, which is the intended behavior of the system. Therefore, to use vision capabilities successfully, you should include the image directly in the user prompt rather than attempting to upload it through workflow steps <sup>[[1]](https://github.com/langgenius/dify/issues/16008)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/093dd0bc-1647-4aa9-8bf1-0dd686c02c71?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/093dd0bc-1647-4aa9-8bf1-0dd686c02c71?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/093dd0bc-1647-4aa9-8bf1-0dd686c02c71?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/093dd0bc-1647-4aa9-8bf1-0dd686c02c71?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/093dd0bc-1647-4aa9-8bf1-0dd686c02c71?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/093dd0bc-1647-4aa9-8bf1-0dd686c02c71?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/093dd0bc-1647-4aa9-8bf1-0dd686c02c71?feedback_type=other)</sup>
Author
Owner

@CTD-Networks-CO-LTD commented on GitHub (Mar 29, 2025):

Hi, @dosu,

to use vision capabilities successfully, you should include the image directly in the user prompt rather than attempting to upload it through workflow steps ( https://github.com/langgenius/dify/issues/16008 ) .

I see, indeed, the sys.files.array function has not been added to the user prompt.
I'll try adding it.

Image

PS: ...The results were different.
https://github.com/langgenius/dify/issues/17121#issuecomment-2764381522

@CTD-Networks-CO-LTD commented on GitHub (Mar 29, 2025): Hi, @dosu, > to use vision capabilities successfully, you should include the image directly in the user prompt rather than attempting to upload it through workflow steps ( https://github.com/langgenius/dify/issues/16008 ) . I see, indeed, the sys.files.array function has not been added to the user prompt. I'll try adding it. ![Image](https://github.com/user-attachments/assets/1b350cbc-eb7e-4c02-b1b8-74c87722e9d9) PS: ...The results were different. https://github.com/langgenius/dify/issues/17121#issuecomment-2764381522
Author
Owner

@dosubot[bot] commented on GitHub (Mar 29, 2025):

Hey @CTD-Networks-CO-LTD! It looks like you're diving into some configuration details. While I can help with bugs and technical questions, this specific request is a bit outside my wheelhouse. If you think I should be able to tackle this, feel free to reach out to the Dosu Team. They'll be happy to hear your thoughts!

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Mar 29, 2025): <!-- Answer --> Hey @CTD-Networks-CO-LTD! It looks like you're diving into some configuration details. While I can help with bugs and technical questions, this specific request is a bit outside my wheelhouse. If you think I should be able to tackle this, feel free to reach out to the [Dosu Team](mailto:customersuccess@dosu.dev). They'll be happy to hear your thoughts! <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/8518c4d9-822e-4fd1-8fa1-144af0d9ffd3?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/8518c4d9-822e-4fd1-8fa1-144af0d9ffd3?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/8518c4d9-822e-4fd1-8fa1-144af0d9ffd3?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/8518c4d9-822e-4fd1-8fa1-144af0d9ffd3?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/8518c4d9-822e-4fd1-8fa1-144af0d9ffd3?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/8518c4d9-822e-4fd1-8fa1-144af0d9ffd3?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/8518c4d9-822e-4fd1-8fa1-144af0d9ffd3?feedback_type=other)</sup>
Author
Owner

@CTD-Networks-CO-LTD commented on GitHub (Mar 30, 2025):

Dify workflow_Vision fails with GPT4oSearchPreview_GPT4o is stable

(Dify1.1.3_miiboPlugin0.0.2_Changed from GPT4oSearchPreview to GPT4o)

https://github.com/user-attachments/assets/f9405077-a54d-41b9-a80b-89f617641150

Things I learned while trying it out :

  • (1) Vision will not run stably unless you use "GPT-4o".
  • (2) There is no need to add the "sys.files" function to the USER prompt. @dosu,
    • The "sys.files" function in the Vision item was enough. (Please watch the video.)
      Image

      Reverted. (Only Vision Setup {sys.files})
      Image

So, is this ultimately a problem with "GPT-4o Search Preview" ?
... Well, it is a preview version after all.

By the way, after I added the sys.files variable to the USER prompt as advised by @dosu bot,
I was never able to get Vision to respond with "GPT-4o-Search Preview" again, even if I removed it again...

@CTD-Networks-CO-LTD commented on GitHub (Mar 30, 2025): ## Dify workflow_Vision fails with GPT4oSearchPreview_GPT4o is stable (Dify1.1.3_miiboPlugin0.0.2_Changed from GPT4oSearchPreview to GPT4o) https://github.com/user-attachments/assets/f9405077-a54d-41b9-a80b-89f617641150 ### Things I learned while trying it out : - **(1) Vision will not run stably unless you use "GPT-4o".** - **(2) There is no need to add the "sys.files" function to the USER prompt.** @dosu, - **The "sys.files" function in the Vision item was enough. (Please watch the video.)** ![Image](https://github.com/user-attachments/assets/7e188f02-7056-4e5f-a8ce-8d3fae33457b) ↓ Reverted. (Only Vision Setup {sys.files}) ![Image](https://github.com/user-attachments/assets/6d406a33-5a49-46ef-87a1-3bf6a3fdc46e) So, is this ultimately a problem with "GPT-4o Search Preview" ? ... Well, it is a preview version after all. By the way, after I added the sys.files variable to the USER prompt as advised by @dosu bot, I was never able to get Vision to respond with "GPT-4o-Search Preview" again, even if I removed it again...
Author
Owner

@dosubot[bot] commented on GitHub (Mar 30, 2025):

Hey @CTD-Networks-CO-LTD! It sounds like you're exploring some interesting findings with Dify and GPT-4o configurations. While I can help with bugs and technical questions, diving into the specifics of preview versions and their stability is a bit outside my expertise. If you think I should be able to help with this, feel free to reach out to the Dosu Team. They'll be happy to hear your thoughts!

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Mar 30, 2025): <!-- Answer --> Hey @CTD-Networks-CO-LTD! It sounds like you're exploring some interesting findings with Dify and GPT-4o configurations. While I can help with bugs and technical questions, diving into the specifics of preview versions and their stability is a bit outside my expertise. If you think I should be able to help with this, feel free to reach out to the [Dosu Team](mailto:customersuccess@dosu.dev). They'll be happy to hear your thoughts! <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/e9f85a23-41de-4706-8b23-ea2d1ef17dd4?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/e9f85a23-41de-4706-8b23-ea2d1ef17dd4?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/e9f85a23-41de-4706-8b23-ea2d1ef17dd4?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/e9f85a23-41de-4706-8b23-ea2d1ef17dd4?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/e9f85a23-41de-4706-8b23-ea2d1ef17dd4?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/e9f85a23-41de-4706-8b23-ea2d1ef17dd4?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/e9f85a23-41de-4706-8b23-ea2d1ef17dd4?feedback_type=other)</sup>
Author
Owner

@crazywoola commented on GitHub (Mar 30, 2025):

When you publish the application, the configuration only applies for the new conversation. In this case, you need to press the "New conversation" button and start chatting.

@crazywoola commented on GitHub (Mar 30, 2025): When you publish the application, the configuration only applies for the new conversation. In this case, you need to press the "New conversation" button and start chatting.
Author
Owner

@CTD-Networks-CO-LTD commented on GitHub (Mar 30, 2025):

Hi, @crazywoola
Thanks for the reply.

When you publish the application, the configuration only applies for the new conversation. In this case, you need to press the "New conversation" button and start chatting.

In my testing videos, all of the public URL chat started from "New Conversation."

My conclusion so far is that "GPT-4o-Search Preview" is unstable, but what do you think?
Is there a possibility that Dify can improve this?

@CTD-Networks-CO-LTD commented on GitHub (Mar 30, 2025): Hi, @crazywoola Thanks for the reply. > When you publish the application, the configuration only applies for the new conversation. In this case, you need to press the "New conversation" button and start chatting. In my testing videos, all of the public URL chat started from "New Conversation." My conclusion so far is that "GPT-4o-Search Preview" is unstable, but what do you think? Is there a possibility that Dify can improve this?
Author
Owner

@crazywoola commented on GitHub (Mar 30, 2025):

We will take a look at the plugin later.

@crazywoola commented on GitHub (Mar 30, 2025): We will take a look at the plugin later.
Author
Owner

@CTD-Networks-CO-LTD commented on GitHub (Mar 30, 2025):

Dify workflow: We tried to make it compatible by dividing the LLM of the conditional branch into "GPT-4o" and "GPT-4o Search Preview"

https://github.com/user-attachments/assets/b9b71cf9-278f-402d-8b35-1b58ac434f7d

In the Dify conditional branch, we decided to operate it separately into an LLM that handles "images" (select GPT-4o, enable Vision) and another LLM (GPT-4o Search Preview).

Image

When using "dify-plugin-miibo-agent", we set it up as follows for miibo.

  • (1) Set the base LLM model to "GPT-4o Search Preview".
    Image

  • (2) Create an "LLM for vision" in the "scenario" (when the conversation condition includes: "image, picture, illustration, jpg, png, capture") and set the LLM model to "GPT-4o".
    Image

@CTD-Networks-CO-LTD commented on GitHub (Mar 30, 2025): ## Dify workflow: We tried to make it compatible by dividing the LLM of the conditional branch into "GPT-4o" and "GPT-4o Search Preview" https://github.com/user-attachments/assets/b9b71cf9-278f-402d-8b35-1b58ac434f7d In the Dify conditional branch, we decided to operate it separately into an LLM that handles "images" (select GPT-4o, enable Vision) and another LLM (GPT-4o Search Preview). ![Image](https://github.com/user-attachments/assets/55be5c6c-c52e-41b8-8fa3-db1c42d49e28) ### When using "[dify-plugin-miibo-agent](https://github.com/cshara1/dify-plugin-miibo-agent)", we set it up as follows for [miibo](https://miibo.ai/). - (1) Set the base LLM model to "GPT-4o Search Preview". ![Image](https://github.com/user-attachments/assets/42d4f1ea-0e2b-4a7d-a6c5-0415a0213e89) - (2) Create an "LLM for vision" in the "scenario" (when the conversation condition includes: "image, picture, illustration, jpg, png, capture") and set the LLM model to "GPT-4o". ![Image](https://github.com/user-attachments/assets/ecfa6443-c8c2-430f-8d97-8955ac59a67c)
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#12344