langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Files are not being input into LLMs #20268

New Issue
Closed
opened 2026-02-21 20:06:35 -05:00 by yindo · 18 comments
yindo commented 2026-02-21 20:06:35 -05:00
Owner
Copy Link
Copy Source

Originally created by @jaias92 on GitHub (Nov 11, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.9.1

Cloud or Self Hosted

Cloud

Steps to reproduce

Currently the LLM is not taking in any Files.
{ "uploads": [ { "dify_model_identity": "__dify__file__", "extension": ".docx", "filename": "Version 2.docx", "id": null, "mime_type": "application/vnd.openxmlformats-officedocument.wordprocessingml.document", "related_id": "6b35048e-3632-4b22-9965-d5a8c83e5d22", "remote_url": "https://upload.dify.ai/files/6b35048e-3632-4b22-9965-d5a8c83e5d22/file-preview?timestamp=1762919663&nonce=2db1c83c278e0ce03921290034424fbd&sign=Lco1yvVYktQ3AY_5BpAcU9FC2bTj1H_Aq1kfsI_c2Cc%3D", "size": 20015, "tenant_id": "7c903e1f-33ab-4735-8c85-3fff71098944", "transfer_method": "local_file", "type": "document", "url": "https://upload.dify.ai/files/6b35048e-3632-4b22-9965-d5a8c83e5d22/file-preview?timestamp=1762919668&nonce=13e84d2ac5913e5daa41f729d2b56869&sign=vanc7Eh_8RLGmqIVuaW4n87v1J1InG1Tdgj5I0GK1zE%3D" } ], "sys.files": [], "sys.user_id": "de68b25c-5d3a-4dd3-bb8f-fc5c6a5c0131", "sys.app_id": "5bc37f87-5343-4969-928b-1fb3a440dff5", "sys.workflow_id": "c18259a2-6a4a-4ea1-a89f-60f80930a028", "sys.workflow_run_id": "9a1a4178-b11d-4809-86f7-4f7c84170b19", "sys.query": "whats in the file", "sys.conversation_id": "2491fa6c-11f9-44d9-8e6d-d92964deef7b", "sys.dialogue_count": 1 }

The data processed by the LLM is as follows

{ "model_mode": "chat", "prompts": [ { "files": [], "role": "system", "text": "is a file" }, { "files": [], "role": "system", "text": "" }, { "files": [], "role": "user", "text": "whats in the file\n\n" }, { "files": [], "role": "user", "text": "" } ], "usage": { "completion_price": "0.01379", "completion_price_unit": "0.000001", "completion_tokens": 1379, "completion_unit_price": "10", "currency": "USD", "latency": 13.873042586999873, "prompt_price": "0.0000125", "prompt_price_unit": "0.000001", "prompt_tokens": 10, "prompt_unit_price": "1.25", "total_price": "0.0138025", "total_tokens": 1389 }, "finish_reason": "STOP", "model_provider": "langgenius/gemini/google", "model_name": "gemini-2.5-pro" }

Image

below is the response,

"I'm sorry, but I can't see or access any files on your computer. For your security and privacy, I don't have the ability to interact with your local file system."

✔️ Expected Behavior

I am expecting the LLM to be able to process the file.

❌ Actual Behavior

The file upload is not received by the LLM Module

Originally created by @jaias92 on GitHub (Nov 11, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.9.1 ### Cloud or Self Hosted Cloud ### Steps to reproduce Currently the LLM is not taking in any Files. `{ "uploads": [ { "dify_model_identity": "__dify__file__", "extension": ".docx", "filename": "Version 2.docx", "id": null, "mime_type": "application/vnd.openxmlformats-officedocument.wordprocessingml.document", "related_id": "6b35048e-3632-4b22-9965-d5a8c83e5d22", "remote_url": "https://upload.dify.ai/files/6b35048e-3632-4b22-9965-d5a8c83e5d22/file-preview?timestamp=1762919663&nonce=2db1c83c278e0ce03921290034424fbd&sign=Lco1yvVYktQ3AY_5BpAcU9FC2bTj1H_Aq1kfsI_c2Cc%3D", "size": 20015, "tenant_id": "7c903e1f-33ab-4735-8c85-3fff71098944", "transfer_method": "local_file", "type": "document", "url": "https://upload.dify.ai/files/6b35048e-3632-4b22-9965-d5a8c83e5d22/file-preview?timestamp=1762919668&nonce=13e84d2ac5913e5daa41f729d2b56869&sign=vanc7Eh_8RLGmqIVuaW4n87v1J1InG1Tdgj5I0GK1zE%3D" } ], "sys.files": [], "sys.user_id": "de68b25c-5d3a-4dd3-bb8f-fc5c6a5c0131", "sys.app_id": "5bc37f87-5343-4969-928b-1fb3a440dff5", "sys.workflow_id": "c18259a2-6a4a-4ea1-a89f-60f80930a028", "sys.workflow_run_id": "9a1a4178-b11d-4809-86f7-4f7c84170b19", "sys.query": "whats in the file", "sys.conversation_id": "2491fa6c-11f9-44d9-8e6d-d92964deef7b", "sys.dialogue_count": 1 }` The data processed by the LLM is as follows `{ "model_mode": "chat", "prompts": [ { "files": [], "role": "system", "text": "is a file" }, { "files": [], "role": "system", "text": "" }, { "files": [], "role": "user", "text": "whats in the file\n\n" }, { "files": [], "role": "user", "text": "" } ], "usage": { "completion_price": "0.01379", "completion_price_unit": "0.000001", "completion_tokens": 1379, "completion_unit_price": "10", "currency": "USD", "latency": 13.873042586999873, "prompt_price": "0.0000125", "prompt_price_unit": "0.000001", "prompt_tokens": 10, "prompt_unit_price": "1.25", "total_price": "0.0138025", "total_tokens": 1389 }, "finish_reason": "STOP", "model_provider": "langgenius/gemini/google", "model_name": "gemini-2.5-pro" }` <img width="374" height="562" alt="Image" src="https://github.com/user-attachments/assets/eeac6c39-e473-4a2c-b650-2f758a943dea" /> below is the response, "I'm sorry, but I can't see or access any files on your computer. For your security and privacy, I don't have the ability to interact with your local file system." ### ✔️ Expected Behavior I am expecting the LLM to be able to process the file. ### ❌ Actual Behavior The file upload is not received by the LLM Module
yindo added the 🐞 bugcloud labels 2026-02-21 20:06:35 -05:00
yindo closed this issue 2026-02-21 20:06:35 -05:00
yindo commented 2026-02-21 20:06:35 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Nov 11, 2025):

Hi @jaias92! I'm Dosu and I’m helping the dify team.

The root cause is that, while your files are uploaded and metadata appears in the uploads field, they aren't being mapped into sys.files or injected into the LLM prompt—so the LLM receives empty file arrays and can't process your files. Dify relies on prompt templates or workflow configuration to explicitly reference uploaded files (like using /sys.files or /uploads in the prompt setup). If these aren't set up correctly, the files won't reach the LLM, even though the upload succeeds.

To fix this, check your workflow or prompt template and make sure it references the uploaded files variable (for example, sys.files). If you use a Document Extractor node, connect its output to the LLM node's prompt so the file content is actually passed to the model. For Gemini models, ensure the payload includes both the file and a required text field, as Gemini may reject requests that only contain file links or metadata without actual content or text context reference.

If you see empty arrays in the prompt, review your template and workflow configuration—this is usually where the mapping breaks down reference.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

@dosubot[bot] commented on GitHub (Nov 11, 2025): <!-- Greeting --> Hi @jaias92! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the dify team. <!-- Answer --> The root cause is that, while your files are uploaded and metadata appears in the `uploads` field, they aren't being mapped into `sys.files` or injected into the LLM prompt—so the LLM receives empty file arrays and can't process your files. Dify relies on prompt templates or workflow configuration to explicitly reference uploaded files (like using `/sys.files` or `/uploads` in the prompt setup). If these aren't set up correctly, the files won't reach the LLM, even though the upload succeeds. To fix this, check your workflow or prompt template and make sure it references the uploaded files variable (for example, `sys.files`). If you use a Document Extractor node, connect its output to the LLM node's prompt so the file content is actually passed to the model. For Gemini models, ensure the payload includes both the file and a required `text` field, as Gemini may reject requests that only contain file links or metadata without actual content or text context [reference](https://github.com/langgenius/dify-official-plugins/issues/1403). If you see empty arrays in the prompt, review your template and workflow configuration—this is usually where the mapping breaks down [reference](https://github.com/langgenius/dify-official-plugins/issues/1544). <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/a04fa177-9183-475f-a743-e04d7866c449?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/a04fa177-9183-475f-a743-e04d7866c449?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/a04fa177-9183-475f-a743-e04d7866c449?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/a04fa177-9183-475f-a743-e04d7866c449?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/a04fa177-9183-475f-a743-e04d7866c449?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/a04fa177-9183-475f-a743-e04d7866c449?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/a04fa177-9183-475f-a743-e04d7866c449?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/28105)
yindo commented 2026-02-21 20:06:35 -05:00
Author
Owner
Copy Link
Copy Source

@jaias92 commented on GitHub (Nov 11, 2025):

@dosu i have handled the file variables as required and included it inside the LLM module . it is still not seeing the file

@jaias92 commented on GitHub (Nov 11, 2025): @dosu i have handled the file variables as required and included it inside the LLM module . it is still not seeing the file
yindo commented 2026-02-21 20:06:35 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Nov 11, 2025):

If the LLM module still isn't seeing your file after including the file variables, the most common cause (especially for DOCX and other document types) is that Dify does not pass document files directly to the LLM node. Instead, you need to add a Document Extractor node to your workflow: connect your uploaded file variable (like sys.files) to the extractor, then pass the extractor's output (usually a text variable) into the LLM node's prompt. The LLM node only processes text (or images if vision is enabled), not raw document files—this is by design in both the code and documentation. If you skip the extractor, the LLM will always see an empty file array, even if the file variable is present in the workflow. For Gemini and similar models, this step is required for DOCX, PDF, and other document types to be processed as prompt content citation.

If you already have a Document Extractor node, double-check that its output is actually connected to the LLM node's input, and that the variable you reference in the prompt is the extractor's output (not just the original file variable). If you only need to process images, make sure the vision feature is enabled in the LLM node and that the file type is supported citation.

Recent updates also improved file input handling and validation, so if you're on an older version or using custom plugins, make sure your workflow matches the latest requirements for file mapping and input formatting citation.

If you want a minimal working example, the correct setup is: file upload → Document Extractor node → LLM node (with the extractor's output as input). This should resolve the issue for document files.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

@dosubot[bot] commented on GitHub (Nov 11, 2025): <!-- Answer --> If the LLM module still isn't seeing your file after including the file variables, the most common cause (especially for DOCX and other document types) is that Dify does not pass document files directly to the LLM node. Instead, you need to add a Document Extractor node to your workflow: connect your uploaded file variable (like `sys.files`) to the extractor, then pass the extractor's output (usually a `text` variable) into the LLM node's prompt. The LLM node only processes text (or images if vision is enabled), not raw document files—this is by design in both the code and documentation. If you skip the extractor, the LLM will always see an empty file array, even if the file variable is present in the workflow. For Gemini and similar models, this step is required for DOCX, PDF, and other document types to be processed as prompt content [citation](https://github.com/langgenius/dify/issues/21375). If you already have a Document Extractor node, double-check that its output is actually connected to the LLM node's input, and that the variable you reference in the prompt is the extractor's output (not just the original file variable). If you only need to process images, make sure the vision feature is enabled in the LLM node and that the file type is supported [citation](https://github.com/langgenius/dify/issues/11813). Recent updates also improved file input handling and validation, so if you're on an older version or using custom plugins, make sure your workflow matches the latest requirements for file mapping and input formatting [citation](https://github.com/langgenius/dify/pull/26848). If you want a minimal working example, the correct setup is: file upload → Document Extractor node → LLM node (with the extractor's output as input). This should resolve the issue for document files. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/cff0cb5e-728c-476d-9caf-6afe9a846076?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/cff0cb5e-728c-476d-9caf-6afe9a846076?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/cff0cb5e-728c-476d-9caf-6afe9a846076?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/cff0cb5e-728c-476d-9caf-6afe9a846076?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/cff0cb5e-728c-476d-9caf-6afe9a846076?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/cff0cb5e-728c-476d-9caf-6afe9a846076?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/cff0cb5e-728c-476d-9caf-6afe9a846076?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/28105)
yindo commented 2026-02-21 20:06:35 -05:00
Author
Owner
Copy Link
Copy Source

@jaias92 commented on GitHub (Nov 12, 2025):

@dosu the models are supposed to accept the file directly without the need of preprocessing.

@jaias92 commented on GitHub (Nov 12, 2025): @dosu the models are supposed to accept the file directly without the need of preprocessing.
yindo commented 2026-02-21 20:06:35 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Nov 12, 2025):

Dify's LLM nodes—including Gemini 2.5 Pro—do not support direct ingestion of document files like DOCX or PDF. The platform is designed to require preprocessing (for example, using a Document Extractor node) to convert these files into text before passing them to the LLM. This is enforced in both the workflow logic and the backend code, which expects only preprocessed content (not raw files) to be sent to the model. Direct file input is only supported for certain image types if the vision feature is enabled; for documents, preprocessing is required by design to meet model and API requirements and to ensure consistent behavior across models. There is no evidence in the codebase or recent updates that direct document file ingestion is supported or planned at this time.

If you need the LLM to process document content, the recommended workflow is: file upload → Document Extractor → LLM node, with the extractor's output (text) as the LLM input. For more details, see the recent improvements and validation in file input handling in Dify's workflow logic citation.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

@dosubot[bot] commented on GitHub (Nov 12, 2025): <!-- Answer --> Dify's LLM nodes—including Gemini 2.5 Pro—do not support direct ingestion of document files like DOCX or PDF. The platform is designed to require preprocessing (for example, using a Document Extractor node) to convert these files into text before passing them to the LLM. This is enforced in both the workflow logic and the backend code, which expects only preprocessed content (not raw files) to be sent to the model. Direct file input is only supported for certain image types if the vision feature is enabled; for documents, preprocessing is required by design to meet model and API requirements and to ensure consistent behavior across models. There is no evidence in the codebase or recent updates that direct document file ingestion is supported or planned at this time. If you need the LLM to process document content, the recommended workflow is: file upload → Document Extractor → LLM node, with the extractor's output (text) as the LLM input. For more details, see the recent improvements and validation in file input handling in Dify's workflow logic [citation](https://github.com/langgenius/dify/pull/26848). <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/9f7d16b0-2beb-408d-9c3d-e2ad934b03c6?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/9f7d16b0-2beb-408d-9c3d-e2ad934b03c6?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/9f7d16b0-2beb-408d-9c3d-e2ad934b03c6?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/9f7d16b0-2beb-408d-9c3d-e2ad934b03c6?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/9f7d16b0-2beb-408d-9c3d-e2ad934b03c6?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/9f7d16b0-2beb-408d-9c3d-e2ad934b03c6?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/9f7d16b0-2beb-408d-9c3d-e2ad934b03c6?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/28105)
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@issac2e commented on GitHub (Nov 21, 2025):

Hello, I understand the issue you're facing. Indeed, the Gemini latest model itself supports file parsing, but currently, there may be some limitations in the Dify platform when handling file input. Based on your suggestion, we could consider adding a toggle switch named "File Parsing," similar to the "Visual" feature. When the model supports file parsing, this switch will appear on the interface, and users can turn it on to upload files, which can be in the form of a URL or base64 encoding. Subsequently, the Dify platform can process these files by invoking the large model plugin package through the underlying plugin-daemon. Such an improvement will significantly enhance user experience and make file processing more flexible and powerful. I look forward to the Dify team considering this suggestion and implementing it as soon as possible. Thank you!

Image
@issac2e commented on GitHub (Nov 21, 2025): Hello, I understand the issue you're facing. Indeed, the Gemini latest model itself supports file parsing, but currently, there may be some limitations in the Dify platform when handling file input. Based on your suggestion, we could consider adding a toggle switch named "File Parsing," similar to the "Visual" feature. When the model supports file parsing, this switch will appear on the interface, and users can turn it on to upload files, which can be in the form of a URL or base64 encoding. Subsequently, the Dify platform can process these files by invoking the large model plugin package through the underlying plugin-daemon. Such an improvement will significantly enhance user experience and make file processing more flexible and powerful. I look forward to the Dify team considering this suggestion and implementing it as soon as possible. Thank you! <img width="443" height="857" alt="Image" src="https://github.com/user-attachments/assets/ad019b70-fce2-4b44-831f-01cfcc87ce46" />
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@QIN2DIM commented on GitHub (Nov 21, 2025):

@issac2e

To fully leverage Gemini's multimodal capabilities, it is recommended to preprocess files before uploading. For files containing complex structural information and diagrams, upload them as PDFs whenever possible; otherwise, upload them as txt files.

You can use Dify's built-in file processing tool to pre-process files.

https://ai.google.dev/gemini-api/docs/document-processing?hl=zh-cn#document-types

https://github.com/langgenius/dify-official-plugins/blob/main/models/gemini/models/llm/utils.py#L61

@QIN2DIM commented on GitHub (Nov 21, 2025): @issac2e To fully leverage Gemini's multimodal capabilities, it is recommended to preprocess files before uploading. For files containing complex structural information and diagrams, upload them as PDFs whenever possible; otherwise, upload them as txt files. You can use Dify's built-in file processing tool to pre-process files. https://ai.google.dev/gemini-api/docs/document-processing?hl=zh-cn#document-types https://github.com/langgenius/dify-official-plugins/blob/main/models/gemini/models/llm/utils.py#L61
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@issac2e commented on GitHub (Nov 21, 2025):

@QIN2DIM I think this is not good. Since the model has built-in text parsing capabilities, I don't think users should do their own preprocessing. This increases the complexity of the workflow, and the results may not necessarily be good. You don't know what format the preprocessing should be in to help the model understand it, and in places with a mix of text and images, doing too much engineering work can easily lead to errors. We should trust large models to handle text (mixed understanding of text, images, and formulas).

@issac2e commented on GitHub (Nov 21, 2025): @QIN2DIM I think this is not good. Since the model has built-in text parsing capabilities, I don't think users should do their own preprocessing. This increases the complexity of the workflow, and the results may not necessarily be good. You don't know what format the preprocessing should be in to help the model understand it, and in places with a mix of text and images, doing too much engineering work can easily lead to errors. We should trust large models to handle text (mixed understanding of text, images, and formulas).
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@QIN2DIM commented on GitHub (Nov 21, 2025):

@issac2e

That's not what I'm saying. In the current dify gemini plugin implementation, files that the model interface supports get uploaded directly to GCP through the file API. The plugin filters out unsupported file types internally to prevent exceptions from being thrown.

@QIN2DIM commented on GitHub (Nov 21, 2025): @issac2e That's not what I'm saying. In the current dify gemini plugin implementation, files that the **model interface** supports get uploaded directly to GCP through the file API. The plugin filters out unsupported file types internally to prevent exceptions from being thrown.
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@QIN2DIM commented on GitHub (Nov 21, 2025):

The dify gemini plugin simply implements application-layer transmission according to protocol specifications—there's no 'experience-based' file interception. You can test this in AI Studio: drag in a .docx file and the frontend will block it.

Image
@QIN2DIM commented on GitHub (Nov 21, 2025): The dify gemini plugin simply implements application-layer transmission according to protocol specifications—there's no 'experience-based' file interception. You can test this in AI Studio: drag in a .docx file and the frontend will block it. <img width="1247" height="292" alt="Image" src="https://github.com/user-attachments/assets/4120705e-a0e6-49be-9f7f-a7209c1de669" />
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@QIN2DIM commented on GitHub (Nov 21, 2025):

Also, you're conflating the concepts of 'file,' and 'modal tokens'. As one of the most powerful multimodal models out there, Gemini can handle very long texts and numerous images in one go. But the model still cannot directly process 'files'.

@QIN2DIM commented on GitHub (Nov 21, 2025): Also, you're conflating the concepts of 'file,' and 'modal tokens'. As one of the most powerful multimodal models out there, Gemini can handle very long texts and numerous images in one go. But the model still cannot directly process 'files'.
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@rty813 commented on GitHub (Nov 22, 2025):

I want the openai-api-compatible plugin to support sending files (not image) to the LLM, but it seems this is currently not supported. Are there any plans to support this?

@rty813 commented on GitHub (Nov 22, 2025): I want the `openai-api-compatible` plugin to support sending files (not image) to the LLM, but it seems this is currently not supported. Are there any plans to support this?
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@rty813 commented on GitHub (Nov 22, 2025):

Image

According to the official documentation, Dify should support inputting files directly in the prompt, but in actual use, it just became empty here.

@rty813 commented on GitHub (Nov 22, 2025): <img width="3598" height="2188" alt="Image" src="https://github.com/user-attachments/assets/4f6eb0e7-2c79-461f-8760-0237bace3639" /> According to the [official documentation](https://docs.dify.ai/zh-hans/guides/workflow/file-upload#%E6%96%B9%E6%B3%95%E4%B8%80%EF%BC%9A%E4%BD%BF%E7%94%A8%E5%85%B7%E5%A4%87%E6%96%87%E4%BB%B6%E5%A4%84%E7%90%86%E8%83%BD%E5%8A%9B%E7%9A%84-llm), Dify should support inputting files directly in the prompt, but in actual use, it just became empty here.
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@QIN2DIM commented on GitHub (Nov 22, 2025):

@rty813 This part of the Gemini plugin has been optimized to facilitate file transfer in accordance with the characteristics of the Gemini model. You can refer to the following demo: a PDF file containing both text and images, which is submitted in a chatflow with only the most basic nodes. The model can "see" the complete content of the PDF file, including images. Dify does not intercept or discard it.

However, openai-api-compatible is somewhat special; I have carefully reviewed the code in this section. While it might be relatively easy to implement the ability to upload files, users must clearly understand that they are using file types supported by the upstream provider through the OpenAI-compatible interface.

In other words, users must know themselves which file formats can be transferred and which cannot, otherwise they will receive 4xx error responses from upstream.

Typically, PDF files are a mandatory supported file type.

testfile:

测试文本.pdf

demo:
https://github.com/user-attachments/assets/9f29558b-2bac-4a54-9ee1-46a64e0b2854

@QIN2DIM commented on GitHub (Nov 22, 2025): @rty813 This part of the Gemini plugin has been optimized to facilitate file transfer in accordance with the characteristics of the Gemini model. You can refer to the following demo: a PDF file containing both text and images, which is submitted in a chatflow with only the most basic nodes. The model can "see" the complete content of the PDF file, including images. Dify does not intercept or discard it. However, `openai-api-compatible` is somewhat special; I have carefully reviewed the code in this section. While it might be relatively easy to implement the ability to upload files, users must clearly understand that they are using file types supported by the upstream provider through the OpenAI-compatible interface. In other words, users must know themselves which file formats can be transferred and which cannot, otherwise they will receive 4xx error responses from upstream. Typically, PDF files are a mandatory supported file type. testfile: [测试文本.pdf](https://github.com/user-attachments/files/23688301/default.pdf) demo: https://github.com/user-attachments/assets/9f29558b-2bac-4a54-9ee1-46a64e0b2854
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@issac2e commented on GitHub (Nov 22, 2025):

Thank you for your patient explanation. Through the demo, I understand how to use LLM to parse files. It turns out that the gemini plugin performs these conversion operations. I also reviewed the code for dify-official-plugins, and indeed, after splitting the conversation, it passes the files via the upload file API or embedded method (function to build parts from content). We are also using an openai-api-compatible plugin, and the downstream LLM gateway (litellm) actually handles the entire operation of uploading files to various suppliers for us. In reality, we only need to modify the openai-api-compatible plugin to parse the file content within the content (at PromptMessageContentType.DOCUMENT) and construct appropriate request parameters to be given to the LLM gateway.

However, I still believe that if the "upload file" feature of large models can become a future standard, similar to "parse images," I think it would be beneficial to add an upload file entry on the LLM node. This would make it easier for users to understand the input. If so, the plugin layer can directly obtain this parameter without needing to split from content

Image
@issac2e commented on GitHub (Nov 22, 2025): Thank you for your patient explanation. Through the demo, I understand how to use LLM to parse files. It turns out that the gemini plugin performs these conversion operations. I also reviewed the code for `dify-official-plugins`, and indeed, after splitting the conversation, it passes the files via the upload file API or embedded method (function to build parts from content). We are also using an `openai-api-compatible` plugin, and the downstream LLM gateway (litellm) actually handles the entire operation of uploading files to various suppliers for us. In reality, we only need to modify the `openai-api-compatible` plugin to parse the file content within the `content` (at PromptMessageContentType.DOCUMENT) and construct appropriate request parameters to be given to the LLM gateway. However, I still believe that if the "upload file" feature of large models can become a future standard, similar to "parse images," I think it would be beneficial to add an upload file entry on the LLM node. This would make it easier for users to understand the input. If so, the plugin layer can directly obtain this parameter without needing to split from content <img width="427" height="597" alt="Image" src="https://github.com/user-attachments/assets/c33dc191-a115-4fb7-8bc9-25376296357e" />
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@QIN2DIM commented on GitHub (Nov 22, 2025):

@issac2e

The design of 'passing images via visual variables' is likely legacy and doesn't align with current best practices. In reality, dify sys.files already includes all file types (images, documents, audio, video), so you don't need to use that 'visual variable' entry point.

Also, I'm not quite sure what you mean by 'split.' The purpose of checking the files variable in the plugin is mainly to retain file types that the provider supports receiving—like pdf, markdown, txt—while excluding unsupported types such as .doc and .xlsx.

Furthermore, taking multimodal large models like gemini and glm as examples, we typically need to apply special optimization to the user_prompt. The user content is a variable of type str | List[Part]. We place non-textual modality inputs—such as files, images, audio, video—along with quoted long-context at the beginning of the context, while positioning the user's actual question instructions and tasks later in the context. This approach enhances the model's context retrieval capability.

In dify, long-context is part of the text, and within the plugin it has already been merged with the user's actual question or other textual content. Only Parts from non-textual modalities are extracted and placed at the beginning of the context.

Image
@QIN2DIM commented on GitHub (Nov 22, 2025): @issac2e The design of 'passing images via visual variables' is likely legacy and doesn't align with current best practices. In reality, dify sys.files already includes all file types (images, documents, audio, video), so you don't need to use that 'visual variable' entry point. Also, I'm not quite sure what you mean by 'split.' The purpose of checking the files variable in the plugin is mainly to retain file types that the provider supports receiving—like pdf, markdown, txt—while excluding unsupported types such as .doc and .xlsx. Furthermore, taking multimodal large models like gemini and glm as examples, we typically need to apply special optimization to the user_prompt. The user content is a variable of type `str | List[Part]`. We place non-textual modality inputs—such as files, images, audio, video—along with quoted long-context at the beginning of the context, while positioning the user's actual question instructions and tasks later in the context. This approach enhances the model's context retrieval capability. In dify, long-context is part of the text, and within the plugin it has already been merged with the user's actual question or other textual content. Only Parts from non-textual modalities are extracted and placed at the beginning of the context. <img width="1590" height="1491" alt="Image" src="https://github.com/user-attachments/assets/49670462-6d81-4414-b93d-4c386cff4c42" />
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@issac2e commented on GitHub (Nov 22, 2025):

@QIN2DIM I understand the design of the LLM node in dify, where user prompt words can transmit almost all types of files, and then the underlying plugin layer performs personalized splitting and management of content for different models. It's a good choice.

@issac2e commented on GitHub (Nov 22, 2025): @QIN2DIM I understand the design of the LLM node in dify, where user prompt words can transmit almost all types of files, and then the underlying plugin layer performs personalized splitting and management of content for different models. It's a good choice.
yindo commented 2026-02-21 20:06:36 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 25, 2026):

Hi, @jaias92. I'm Dosu, and I'm helping the Dify team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You reported that uploaded document files like .docx were not processed by the LLM in version 1.9.1 on cloud deployment.
  • It was clarified that Dify requires files to be preprocessed via a Document Extractor node before passing text content to the LLM.
  • Direct ingestion of document files by the LLM is not supported by design.
  • Discussion included potential future improvements such as a "File Parsing" toggle to leverage Gemini’s multimodal capabilities.
  • Currently, file handling depends on plugin-level processing and explicit workflow configuration.

Next Steps:

  • Please confirm if this issue is still relevant with the latest version of Dify; if so, you can keep the discussion open by commenting here.
  • Otherwise, I will automatically close this issue in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Jan 25, 2026): Hi, @jaias92. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog and am marking this issue as stale. **Issue Summary:** - You reported that uploaded document files like .docx were not processed by the LLM in version 1.9.1 on cloud deployment. - It was clarified that Dify requires files to be preprocessed via a Document Extractor node before passing text content to the LLM. - Direct ingestion of document files by the LLM is not supported by design. - Discussion included potential future improvements such as a "File Parsing" toggle to leverage Gemini’s multimodal capabilities. - Currently, file handling depends on plugin-level processing and explicit workflow configuration. **Next Steps:** - Please confirm if this issue is still relevant with the latest version of Dify; if so, you can keep the discussion open by commenting here. - Otherwise, I will automatically close this issue in 15 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label cloud 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#20268
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?