Multimodal LLM analyze image errors. #21686

Open
opened 2026-02-21 20:13:51 -05:00 by yindo · 11 comments
Owner

Originally created by @DEMOUSL on GitHub (Jan 15, 2026).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.11.4

Cloud or Self Hosted

Cloud

Steps to reproduce

Image When I tried to analyze images or videos, an error occurred. I tried different ways of entering the parameters, and it only worked with plain text. Errors occurred when using multimodal input.

✔️ Expected Behavior

The analysis results, including images or videos, can be output normally.

Actual Behavior

No response

Originally created by @DEMOUSL on GitHub (Jan 15, 2026). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.11.4 ### Cloud or Self Hosted Cloud ### Steps to reproduce <img width="3298" height="1715" alt="Image" src="https://github.com/user-attachments/assets/d07365fe-4017-4dc2-a5ba-2d5fd01e283e" /> When I tried to analyze images or videos, an error occurred. I tried different ways of entering the parameters, and it only worked with plain text. Errors occurred when using multimodal input. ### ✔️ Expected Behavior The analysis results, including images or videos, can be output normally. ### ❌ Actual Behavior _No response_
yindo added the 🐞 bugstalecloud labels 2026-02-21 20:13:51 -05:00
Author
Owner

@dosubot[bot] commented on GitHub (Jan 15, 2026):

Hi @DEMOUSL! I'm Dosu and I’m helping the dify team.

This is a known bug in Dify v1.11.4 (cloud) affecting multimodal LLM workflows, especially with Gemini 3 Pro Preview. The error happens because the system can't generate or retrieve the required file upload URL for images or videos during plugin invocation, which breaks multimodal analysis—plain text works since it doesn't need file upload handling. The root cause is a recent change that deleted the /datasets/{dataset_id}/documents/{document_id}/upload-file API endpoint, which previously provided upload file URLs for plugins and workflows. Without this, the backend can't validate or pass files to the model, resulting in no output or a JSON error about missing upload URLs source.

There’s no official fix yet. Workarounds include switching to models that don’t require strict file upload handling (like Qwen-Max or DeepSeek), converting videos to images or audio for analysis, or using plugin versions that pin older, compatible SDKs if available source. For video, converting to images or audio is the most reliable option right now source.

You can track progress or updates in the open issue here. If your scenario differs or you find a new edge case, sharing details in the issue may help the maintainers.

To reply, just mention @dosu.


How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Chat with Dosu Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Jan 15, 2026): <!-- Greeting --> Hi @DEMOUSL! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the dify team. <!-- Answer --> This is a known bug in Dify v1.11.4 (cloud) affecting multimodal LLM workflows, especially with Gemini 3 Pro Preview. The error happens because the system can't generate or retrieve the required file upload URL for images or videos during plugin invocation, which breaks multimodal analysis—plain text works since it doesn't need file upload handling. The root cause is a recent change that deleted the `/datasets/{dataset_id}/documents/{document_id}/upload-file` API endpoint, which previously provided upload file URLs for plugins and workflows. Without this, the backend can't validate or pass files to the model, resulting in no output or a JSON error about missing upload URLs [source](https://github.com/langgenius/dify/pull/25543). There’s no official fix yet. Workarounds include switching to models that don’t require strict file upload handling (like Qwen-Max or DeepSeek), converting videos to images or audio for analysis, or using plugin versions that pin older, compatible SDKs if available [source](https://github.com/langgenius/dify/issues/16353). For video, converting to images or audio is the most reliable option right now [source](https://github.com/langgenius/dify/issues/29215). You can track progress or updates in the open issue [here](https://github.com/langgenius/dify/issues/1542). If your scenario differs or you find a new edge case, sharing details in the issue may help the maintainers. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/875ad09a-c8bd-4773-bc4e-34505d9f2758?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/875ad09a-c8bd-4773-bc4e-34505d9f2758?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/875ad09a-c8bd-4773-bc4e-34505d9f2758?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/875ad09a-c8bd-4773-bc4e-34505d9f2758?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/875ad09a-c8bd-4773-bc4e-34505d9f2758?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/875ad09a-c8bd-4773-bc4e-34505d9f2758?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/875ad09a-c8bd-4773-bc4e-34505d9f2758?feedback_type=other)</sup> [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=This%20is%20a%20known%20bug%20in%20Dify%20v1.11.4%20%28cloud%29%20affecting%20multimodal%20LLM%20workflows%2C%20especially%20with%20Gemini%203%20Pro%20Preview.%20The%20error%20happens%20because%20the%20system%20can%27t%20generate%20or%20retrieve%20the%20required%20file%20upload%20URL%20for%20images%20or%20videos%20during%20plugin%20invocation%2C%20which%20breaks%20multimodal%20analysis%E2%80%94plain%20text%20works%20since%20it%20doesn%27t%20need%20file%20upload%20handling.%20The%20root%20cause%20is%20a%20recent%20change%20that%20deleted%20the%20%60/datasets/%7Bdataset_id%7D/documents/%7Bdocument_id%7D/upload-file%60%20API%20endpoint%2C%20which%20previously%20provided%20upload%20file%20URLs%20for%20plugins%20and%20workflows.%20Without%20this%2C%20the%20backend%20can%27t%20validate%20or%20pass%20files%20to%20the%20model%2C%20resulting%20in%20no%20output%20or%20a%20JSON%20error%20about%20missing%20upload%20URLs%20%5Bsource%5D%28https%3A//github.com/langgenius/dify/pull/25543%29.%0A%0AThere%E2%80%99s%20no%20official%20fix%20yet.%20Workarounds%20include%20switching%20to%20models%20that%20don%E2%80%99t%20require%20strict%20file%20upload%20handling%20%28like%20Qwen-Max%20or%20DeepSeek%29%2C%20converting%20videos%20to%20images%20or%20audio%20for%20analysis%2C%20or%20using%20plugin%20versions%20that%20pin%20older%2C%20compatible%20SDKs%20if%20available%20%5Bsource%5D%28https%3A//github.com/langgenius/dify/issues/16353%29.%20For%20video%2C%20converting%20to%20images%20or%20audio%20is%20the%20most%20reliable%20option%20right%20now%20%5Bsource%5D%28https%3A//github.com/langgenius/dify/issues/29215%29.%0A%0AYou%20can%20track%20progress%20or%20updates%20in%20the%20open%20issue%20%5Bhere%5D%28https%3A//github.com/langgenius/dify/issues/1542%29.%20If%20your%20scenario%20differs%20or%20you%20find%20a%20new%20edge%20case%2C%20sharing%20details%20in%20the%20issue%20may%20help%20the%20maintainers.)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/31082)
Author
Owner

@QuantumGhost commented on GitHub (Jan 15, 2026):

@DEMOUSL Could you please provide a minimal reproducible example to help us troubleshooting this problem?

@QuantumGhost commented on GitHub (Jan 15, 2026): @DEMOUSL Could you please provide a [minimal reproducible example](https://en.wikipedia.org/wiki/Minimal_reproducible_example) to help us troubleshooting this problem?
Author
Owner

@DEMOUSL commented on GitHub (Jan 15, 2026):

image analysis.yml
This is my most simplified method for generating multimodal output.

@DEMOUSL commented on GitHub (Jan 15, 2026): [image analysis.yml](https://github.com/user-attachments/files/24658346/image.analysis.yml) This is my most simplified method for generating multimodal output.
Author
Owner

@fatelei commented on GitHub (Jan 15, 2026):

Self Checks

  • I have read the Contributing Guide and Language Policy.[x] This is only for bug report, if you would like to ask a question, please head to Discussions.[x] I have searched for existing issues search for existing issues, including closed ones.[x] I confirm that I am using English to submit this report, otherwise it will be closed.[x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)[ ] Please do not modify this template :) and fill in all the required fields.

Dify version

1.11.4

Cloud or Self Hosted

Cloud

Steps to reproduce

Image When I tried to analyze images or videos, an error occurred. I tried different ways of entering the parameters, and it only worked with plain text. Errors occurred when using multimodal input.

✔️ Expected Behavior

The analysis results, including images or videos, can be output normally.

Actual Behavior

No response

need upgrade the plugin, after plugin updated, there is parameter use_inline_file, enable this parameter, it will not upload file to google

@fatelei commented on GitHub (Jan 15, 2026): > ### Self Checks > * [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).[x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).[x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.[x] I confirm that I am using English to submit this report, otherwise it will be closed.[x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)[ ] Please do not modify this template :) and fill in all the required fields. > > ### Dify version > 1.11.4 > > ### Cloud or Self Hosted > Cloud > > ### Steps to reproduce > [<img alt="Image" width="2000" height="1715" src="https://private-user-images.githubusercontent.com/58839535/536532030-d07365fe-4017-4dc2-a5ba-2d5fd01e283e.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Njg1MzQ0NzIsIm5iZiI6MTc2ODUzNDE3MiwicGF0aCI6Ii81ODgzOTUzNS81MzY1MzIwMzAtZDA3MzY1ZmUtNDAxNy00ZGMyLWE1YmEtMmQ1ZmQwMWUyODNlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjAxMTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwMTE2VDAzMjkzMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTRlOTM5MmU0NWE0ZDA4OWQ3ZWM4ZjRkZDQ1ZThlODFiMTQ5MDY3OGVlZWFjNWIzYTExMDg3NTA2ODEyYjAyMzMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.OgW_7K1ZanAzh1FNpWfF-ulodZHg4Eerz7XkiYEytcc">](https://private-user-images.githubusercontent.com/58839535/536532030-d07365fe-4017-4dc2-a5ba-2d5fd01e283e.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Njg1MzMzMjYsIm5iZiI6MTc2ODUzMzAyNiwicGF0aCI6Ii81ODgzOTUzNS81MzY1MzIwMzAtZDA3MzY1ZmUtNDAxNy00ZGMyLWE1YmEtMmQ1ZmQwMWUyODNlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjAxMTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwMTE2VDAzMTAyNlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTAzYjAwOTAxOGM4OThlMjkzZTI0YWEwNDE1MTk3YzI1NGQ0ZDk2MjRlYTNhOTRhZDJkYjdlZjM5MjlhYzdmYzQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.n_31iU8zPxlAEqI-DmQ6UtHQl_Ed8WJrINNlKxRFDzo) When I tried to analyze images or videos, an error occurred. I tried different ways of entering the parameters, and it only worked with plain text. Errors occurred when using multimodal input. > ### ✔️ Expected Behavior > The analysis results, including images or videos, can be output normally. > > ### ❌ Actual Behavior > _No response_ need upgrade the plugin, after plugin updated, there is parameter `use_inline_file`, enable this parameter, it will not upload file to google
Author
Owner

@fatelei commented on GitHub (Jan 15, 2026):

this error is upload file to google, it failed

@fatelei commented on GitHub (Jan 15, 2026): this error is upload file to google, it failed
Author
Owner

@QuantumGhost commented on GitHub (Jan 15, 2026):

Gemini plugin 0.7.4 works for me.

@DEMOUSL Could you please upgrade you gemini plugin to 0.7.4 and check if the problem still exists?

@QuantumGhost commented on GitHub (Jan 15, 2026): Gemini plugin 0.7.4 works for me. @DEMOUSL Could you please upgrade you gemini plugin to `0.7.4` and check if the problem still exists?
Author
Owner

@DEMOUSL commented on GitHub (Jan 16, 2026):

@fatelei @QuantumGhost I have updated the plugin to version 0.7.4, but the error persists, and I couldn't find the new use_inline_file parameter in the settings. Is there anything else I need to modify?

Image Image
@DEMOUSL commented on GitHub (Jan 16, 2026): @fatelei @QuantumGhost I have updated the plugin to version 0.7.4, but the error persists, and I couldn't find the new `use_inline_file` parameter in the settings. Is there anything else I need to modify? <img width="442" height="199" alt="Image" src="https://github.com/user-attachments/assets/f70e859f-671e-44a0-bea1-3adc9a995533" /> <img width="3408" height="1672" alt="Image" src="https://github.com/user-attachments/assets/75cea587-0634-4d17-903a-ef9e33032998" />
Author
Owner

@fatelei commented on GitHub (Jan 16, 2026):

@fatelei @QuantumGhost I have updated the plugin to version 0.7.4, but the error persists, and I couldn't find the new use_inline_file parameter in the settings. Is there anything else I need to modify?

Image Image

currently only model gemini-3-flash-preview support this parameter

@fatelei commented on GitHub (Jan 16, 2026): > [@fatelei](https://github.com/fatelei) [@QuantumGhost](https://github.com/QuantumGhost) I have updated the plugin to version 0.7.4, but the error persists, and I couldn't find the new `use_inline_file` parameter in the settings. Is there anything else I need to modify? > > <img alt="Image" width="442" height="199" src="https://private-user-images.githubusercontent.com/58839535/536616245-f70e859f-671e-44a0-bea1-3adc9a995533.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Njg1NTA5NzQsIm5iZiI6MTc2ODU1MDY3NCwicGF0aCI6Ii81ODgzOTUzNS81MzY2MTYyNDUtZjcwZTg1OWYtNjcxZS00NGEwLWJlYTEtM2FkYzlhOTk1NTMzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjAxMTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwMTE2VDA4MDQzNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWE0ZDc2NzI1YTVmNDRhZWRkNjk1YThkM2I4OThjOGMyM2ZjNTBiNzQwYmFmYzJiNzllNmY2NzE1NzIyNjBjZDYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.GHShXrJ4TwPz9za9Hg58ucrnpHWg8qE2YzEELLJn8aQ"> <img alt="Image" width="2000" height="1672" src="https://private-user-images.githubusercontent.com/58839535/536616926-75cea587-0634-4d17-903a-ef9e33032998.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Njg1NTA5NzQsIm5iZiI6MTc2ODU1MDY3NCwicGF0aCI6Ii81ODgzOTUzNS81MzY2MTY5MjYtNzVjZWE1ODctMDYzNC00ZDE3LTkwM2EtZWY5ZTMzMDMyOTk4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjAxMTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwMTE2VDA4MDQzNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTliODRlMzgwMzI5MzkzMjdiODgyYzEzOWM5MmYzZGY1NDBkZGQzZWY0ZWMxNmFiMmFiNjA4NmUwOWVjMmU3OWImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.RKVonIvosN4IcQdpEfyxKAF_aFNXwuCvizJXNkfarCQ"> currently only model gemini-3-flash-preview support this parameter
Author
Owner

@DEMOUSL commented on GitHub (Jan 16, 2026):

@fatelei I tried gemini-3-flash-preview and it worked, but the results seemed slightly worse than with the pro version. Could you please add the same parameters to gemini-3-pro-preview later?

@DEMOUSL commented on GitHub (Jan 16, 2026): @fatelei I tried gemini-3-flash-preview and it worked, but the results seemed slightly worse than with the pro version. Could you please add the same parameters to gemini-3-pro-preview later?
Author
Owner

@fatelei commented on GitHub (Jan 16, 2026):

@fatelei I tried gemini-3-flash-preview and it worked, but the results seemed slightly worse than with the pro version. Could you please add the same parameters to gemini-3-pro-preview later?

already submit a pr langgenius/dify-official-plugins#2439

@fatelei commented on GitHub (Jan 16, 2026): > [@fatelei](https://github.com/fatelei) I tried gemini-3-flash-preview and it worked, but the results seemed slightly worse than with the pro version. Could you please add the same parameters to gemini-3-pro-preview later? already submit a pr langgenius/dify-official-plugins#2439
Author
Owner

@dosubot[bot] commented on GitHub (Feb 17, 2026):

Hi, @DEMOUSL. I'm Dosu, and I'm helping the Dify team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You reported errors in Dify v1.11.4 Cloud when analyzing multimodal inputs, while plain text works fine.
  • The root cause is a removed API endpoint causing file upload URL failures, breaking multimodal workflows.
  • Workarounds include using models like Qwen-Max, DeepSeek, or converting videos to images/audio.
  • You updated the Gemini plugin to 0.7.4 but still lacked the use_inline_file parameter, which is currently only supported in the gemini-3-flash-preview model.
  • The gemini-3-flash-preview model works for you but with slightly worse results than pro; a PR has been submitted to add this parameter to gemini-3-pro-preview.

Next Steps:

  • Please confirm if this issue is still relevant with the latest version of Dify and the updated models.
  • If it is, you can keep the discussion open by commenting here; otherwise, I will automatically close this issue in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Feb 17, 2026): Hi, @DEMOUSL. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog and am marking this issue as stale. **Issue Summary:** - You reported errors in Dify v1.11.4 Cloud when analyzing multimodal inputs, while plain text works fine. - The root cause is a removed API endpoint causing file upload URL failures, breaking multimodal workflows. - Workarounds include using models like Qwen-Max, DeepSeek, or converting videos to images/audio. - You updated the Gemini plugin to 0.7.4 but still lacked the `use_inline_file` parameter, which is currently only supported in the gemini-3-flash-preview model. - The gemini-3-flash-preview model works for you but with slightly worse results than pro; a PR has been submitted to add this parameter to gemini-3-pro-preview. **Next Steps:** - Please confirm if this issue is still relevant with the latest version of Dify and the updated models. - If it is, you can keep the discussion open by commenting here; otherwise, I will automatically close this issue in 15 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#21686