langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 03:55:25 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

How to correctly operate the input and output of images in chatflow #5164

New Issue
Closed
opened 2026-02-21 18:09:42 -05:00 by yindo · 6 comments
yindo commented 2026-02-21 18:09:42 -05:00
Owner
Copy Link
Copy Source

Originally created by @401557122 on GitHub (Aug 22, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.7.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

image
How to correctly operate the input and output of images in chatflow

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

Originally created by @401557122 on GitHub (Aug 22, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version 0.7.0 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce ![image](https://github.com/user-attachments/assets/1a2e022c-f1c1-4360-aae0-eaea9bfe4604) How to correctly operate the input and output of images in chatflow ### ✔️ Expected Behavior _No response_ ### ❌ Actual Behavior _No response_
yindo closed this issue 2026-02-21 18:09:42 -05:00
yindo commented 2026-02-21 18:09:42 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 22, 2024):

To correctly operate the input and output of images in chatflow, you can use the "直接返信" (Direct Reply) node to define the response content. This node allows you to output images, text, or a combination of both. Here are the steps:

  1. Output Pure Text:

    • You can define a fixed text content or use variables from previous steps.

  2. Output Image + LLM Reply:

    • You can combine images with text generated by the LLM (Language Model).

Example of outputting an image with LLM reply:

**例2：** 画像+LLMの返信を出力

<figure><img src="path_to_your_image.png" alt=""><figcaption>Your LLM generated text here</figcaption></figure>

This configuration supports a live-editing mode, allowing you to visually arrange and mix text and images as needed. The Direct Reply node can be seamlessly integrated at any point to dynamically deliver content into the dialogue responses.

Additionally, the Direct Reply node can be used to stream intermediate results within the process, not just as a final output node. This flexibility allows for dynamic and interactive chatflows.

To handle image inputs, you can use the files parameter in the request body. This parameter supports inputting files (images) combined with text understanding and answering questions, available only when the model supports Vision capability. The files parameter should include:

  • type (string): Supported type is image.
  • transfer_method (string): Transfer method, either remote_url for image URL or local_file for file upload.
  • url (string): Image URL (when the transfer method is remote_url).
  • upload_file_id (string): Uploaded file ID, which must be obtained by uploading through the File Upload API in advance (when the transfer method is local_file).

Example of a request to upload a file:

curl -X POST '${props.appDetail.api_base_url}/files/upload' \
--header 'Authorization: Bearer {api_key}' \
--form 'file=@localfile;type=image/[png|jpeg|jpg|webp|gif]' \
--form 'user=abc-123'

By following these steps, you can effectively manage the input and output of images in your chatflow, ensuring a seamless and interactive user experience [1][2][3][4][5].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Aug 22, 2024): <!-- Answer --> To correctly operate the input and output of images in chatflow, you can use the "直接返信" (Direct Reply) node to define the response content. This node allows you to output images, text, or a combination of both. Here are the steps: 1. **Output Pure Text:** - You can define a fixed text content or use variables from previous steps. 2. **Output Image + LLM Reply:** - You can combine images with text generated by the LLM (Language Model). Example of outputting an image with LLM reply: ```md **例2：** 画像+LLMの返信を出力 <figure><img src="path_to_your_image.png" alt=""><figcaption>Your LLM generated text here</figcaption></figure> ``` This configuration supports a live-editing mode, allowing you to visually arrange and mix text and images as needed. The Direct Reply node can be seamlessly integrated at any point to dynamically deliver content into the dialogue responses. Additionally, the Direct Reply node can be used to stream intermediate results within the process, not just as a final output node. This flexibility allows for dynamic and interactive chatflows. To handle image inputs, you can use the `files` parameter in the request body. This parameter supports inputting files (images) combined with text understanding and answering questions, available only when the model supports Vision capability. The `files` parameter should include: - `type` (string): Supported type is `image`. - `transfer_method` (string): Transfer method, either `remote_url` for image URL or `local_file` for file upload. - `url` (string): Image URL (when the transfer method is `remote_url`). - `upload_file_id` (string): Uploaded file ID, which must be obtained by uploading through the File Upload API in advance (when the transfer method is `local_file`). Example of a request to upload a file: ```bash curl -X POST '${props.appDetail.api_base_url}/files/upload' \ --header 'Authorization: Bearer {api_key}' \ --form 'file=@localfile;type=image/[png|jpeg|jpg|webp|gif]' \ --form 'user=abc-123' ``` By following these steps, you can effectively manage the input and output of images in your chatflow, ensuring a seamless and interactive user experience <sup>[[1]](https://github.com/langgenius/dify-docs/blob/main/zh_CN/guides/workflow/node/answer.md)</sup><sup>[[2]](https://github.com/langgenius/dify-docs/blob/main/en/guides/workflow/node/answer.md)</sup><sup>[[3]](https://github.com/langgenius/dify-docs/blob/main/jp/guides/workflow/node/answer.md)</sup><sup>[[4]](https://github.com/langgenius/dify/blob/main/web/app/components/develop/template/template_advanced_chat.en.mdx)</sup><sup>[[5]](https://github.com/langgenius/dify-docs/blob/main/en/guides/workflow/key_concepts.md)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:09:42 -05:00
Author
Owner
Copy Link
Copy Source

@401557122 commented on GitHub (Aug 22, 2024):

When the QR code tool is connected to the direct reply, entering the file object generated by the QR code in the direct reply will result in an error message: Run failed: Node. Directly reply with 2 run failed: 'FileVar' object has no attribute 'markdown'@dosu.

@401557122 commented on GitHub (Aug 22, 2024): When the QR code tool is connected to the direct reply, entering the file object generated by the QR code in the direct reply will result in an error message: Run failed: Node. Directly reply with 2 run failed: 'FileVar' object has no attribute 'markdown'@dosu.
yindo commented 2026-02-21 18:09:42 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 22, 2024):

The error message you're encountering, 'FileVar' object has no attribute 'markdown', indicates that the FileVar object does not have a markdown attribute. Instead, you should use the to_markdown method to convert the FileVar object to a markdown format.

Here's how you can modify your code to use the to_markdown method:

# Assuming file_var is your FileVar object
markdown_content = file_var.to_markdown()

This will convert the FileVar object to a markdown string that can be used in your Direct Reply node.

If you need to handle the QR code file specifically, ensure that the FileVar object is correctly instantiated and that the to_markdown method is called on it before passing it to the Direct Reply node.

For example:

# Example of creating a FileVar object for a QR code
file_var = FileVar(
    tenant_id="your_tenant_id",
    type=FileType.IMAGE,  # Assuming the QR code is an image
    transfer_method=FileTransferMethod.LOCAL_FILE,
    filename="qrcode.png",
    url="path_to_your_qrcode_image"
)

# Convert to markdown
markdown_content = file_var.to_markdown()

# Use markdown_content in your Direct Reply node

This should resolve the error and allow you to correctly operate the input and output of images, including QR codes, in your chatflow [1].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Aug 22, 2024): <!-- Answer --> The error message you're encountering, `'FileVar' object has no attribute 'markdown'`, indicates that the `FileVar` object does not have a `markdown` attribute. Instead, you should use the `to_markdown` method to convert the `FileVar` object to a markdown format. Here's how you can modify your code to use the `to_markdown` method: ```python # Assuming file_var is your FileVar object markdown_content = file_var.to_markdown() ``` This will convert the `FileVar` object to a markdown string that can be used in your Direct Reply node. If you need to handle the QR code file specifically, ensure that the `FileVar` object is correctly instantiated and that the `to_markdown` method is called on it before passing it to the Direct Reply node. For example: ```python # Example of creating a FileVar object for a QR code file_var = FileVar( tenant_id="your_tenant_id", type=FileType.IMAGE, # Assuming the QR code is an image transfer_method=FileTransferMethod.LOCAL_FILE, filename="qrcode.png", url="path_to_your_qrcode_image" ) # Convert to markdown markdown_content = file_var.to_markdown() # Use markdown_content in your Direct Reply node ``` This should resolve the error and allow you to correctly operate the input and output of images, including QR codes, in your chatflow <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/core/file/file_obj.py)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:09:42 -05:00
Author
Owner
Copy Link
Copy Source

@401557122 commented on GitHub (Aug 22, 2024):

Is this a bug because only direct replies to nodes can use files as input, even if file upload is enabled, files are not an optional input for nodes such as LLM. However, using files in direct replies still generates an error message@dosu.

@401557122 commented on GitHub (Aug 22, 2024): Is this a bug because only direct replies to nodes can use files as input, even if file upload is enabled, files are not an optional input for nodes such as LLM. However, using files in direct replies still generates an error message@dosu.
yindo commented 2026-02-21 18:09:43 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 22, 2024):

Yes, it appears to be a bug. The LLMNode class has a method _fetch_files to fetch files from the variable pool, but the usage of files as input is not fully integrated or supported in the same way as direct reply nodes. Additionally, the _invoke_llm method does not handle file inputs, which could lead to errors when files are used as input.

Here is the relevant part of the code that fetches files:

def _fetch_files(self, node_data: LLMNodeData, variable_pool: VariablePool) -> list["FileVar"]:
    if not node_data.vision.enabled:
        return []

    files = variable_pool.get_any(['sys', SystemVariableKey.FILES.value])
    if not files:
        return []

    return files

However, there is no corresponding handling in the _invoke_llm method to process these files, which could lead to errors. This indicates that while the mechanism to fetch files exists, the complete support for using files as input in nodes other than direct reply nodes is not implemented, leading to the observed issues.

Additionally, the WorkflowConverter class shows that file upload is considered in the conversion process, but it does not seem to be fully integrated into the LLM node handling. Specifically, the _convert_to_llm_node method includes a file_upload parameter, but there is no detailed implementation for processing these files within the LLM node.

def _convert_to_llm_node(
    self,
    original_app_mode: AppMode,
    new_app_mode: AppMode,
    graph: dict,
    model_config: ModelConfigEntity,
    prompt_template: PromptTemplateEntity,
    file_upload: Optional[FileExtraConfig] = None,
    external_data_variable_node_mapping: dict[str, str] | None = None,
) -> dict:
    # Implementation details...

This further supports the conclusion that the integration of file inputs for nodes such as LLM is incomplete, leading to the observed errors and limitations [1][2].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Aug 22, 2024): <!-- Answer --> Yes, it appears to be a bug. The `LLMNode` class has a method `_fetch_files` to fetch files from the variable pool, but the usage of files as input is not fully integrated or supported in the same way as direct reply nodes. Additionally, the `_invoke_llm` method does not handle file inputs, which could lead to errors when files are used as input. Here is the relevant part of the code that fetches files: ```python def _fetch_files(self, node_data: LLMNodeData, variable_pool: VariablePool) -> list["FileVar"]: if not node_data.vision.enabled: return [] files = variable_pool.get_any(['sys', SystemVariableKey.FILES.value]) if not files: return [] return files ``` However, there is no corresponding handling in the `_invoke_llm` method to process these files, which could lead to errors. This indicates that while the mechanism to fetch files exists, the complete support for using files as input in nodes other than direct reply nodes is not implemented, leading to the observed issues. Additionally, the `WorkflowConverter` class shows that file upload is considered in the conversion process, but it does not seem to be fully integrated into the LLM node handling. Specifically, the `_convert_to_llm_node` method includes a `file_upload` parameter, but there is no detailed implementation for processing these files within the LLM node. ```python def _convert_to_llm_node( self, original_app_mode: AppMode, new_app_mode: AppMode, graph: dict, model_config: ModelConfigEntity, prompt_template: PromptTemplateEntity, file_upload: Optional[FileExtraConfig] = None, external_data_variable_node_mapping: dict[str, str] | None = None, ) -> dict: # Implementation details... ``` This further supports the conclusion that the integration of file inputs for nodes such as LLM is incomplete, leading to the observed errors and limitations <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/core/workflow/nodes/llm/llm_node.py)</sup><sup>[[2]](https://github.com/langgenius/dify/blob/main/api/services/workflow/workflow_converter.py)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:09:43 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Aug 22, 2024):

This is resolved in v0.7.1 also this is a duplicated question.

@crazywoola commented on GitHub (Aug 22, 2024): This is resolved in v0.7.1 also this is a duplicated question.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#5164
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?