langgenius/dify-plugin-sdks
1
0
Fork 0
mirror of https://github.com/langgenius/dify-plugin-sdks.git synced 2026-07-22 02:15:22 -04:00
Code Issues 58 Packages Projects Releases 10 Wiki Activity

How to get the file url in LargeLanguageModel? #65

New Issue
Closed
opened 2026-02-15 21:15:31 -05:00 by yindo · 1 comment
yindo commented 2026-02-15 21:15:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @sweetwxh on GitHub (Dec 4, 2025).

When passing a video file into an LLM node, the input is as follows:
{
"#files#": [
{
"dify_model_identity": "dify__file",
"id": null,
"tenant_id": "ebfc4fce-084a-4fea-8f62-ff0464f4ca79",
"type": "video",
"transfer_method": "local_file",
"remote_url": "",
"related_id": "16c33ddf-d0bd-4dec-90d6-a39856fe4037",
"filename": "1.mp4",
"extension": ".mp4",
"mime_type": "video/mp4",
"size": 1471368,
"url": ""
}
]
}
Within a custom Model provider, can the parameter prompt_messages inside LargeLanguageModel._invoke obtain the URL of the input file (VideoPromptMessageContent)?

Originally created by @sweetwxh on GitHub (Dec 4, 2025). When passing a video file into an LLM node, the input is as follows: { "#files#": [ { "dify_model_identity": "__dify__file__", "id": null, "tenant_id": "ebfc4fce-084a-4fea-8f62-ff0464f4ca79", "type": "video", "transfer_method": "local_file", "remote_url": "", "related_id": "16c33ddf-d0bd-4dec-90d6-a39856fe4037", "filename": "1.mp4", "extension": ".mp4", "mime_type": "video/mp4", "size": 1471368, "url": "" } ] } Within a custom Model provider, can the parameter prompt_messages inside LargeLanguageModel._invoke obtain the URL of the input file (VideoPromptMessageContent)?
yindo added the question label 2026-02-15 21:15:31 -05:00
yindo closed this issue 2026-02-15 21:15:31 -05:00
yindo commented 2026-02-15 21:15:32 -05:00
Author
Owner
Copy Link
Copy Source

@Mairuis commented on GitHub (Dec 8, 2025):

Hi @sweetwxh,

Thanks for raising this question. Based on the SDK and Dify core code, here's how you can get the file URL in your custom LLM provider:

How it works

When files are passed to an LLM node, Dify converts them to PromptMessageContent objects (like VideoPromptMessageContent). The MultiModalPromptMessageContent base class has both url and base64_data fields:

class MultiModalPromptMessageContent(PromptMessageContent):
    format: str = Field(default=..., description="the format of multi-modal file")
    base64_data: str = Field(default="", description="the base64 data of multi-modal file")
    url: str = Field(default="", description="the url of multi-modal file")
    mime_type: str = Field(default=..., description="the mime type of multi-modal file")
    filename: str = Field(default="", description="the filename of multi-modal file")

    @property
    def data(self):
        return self.url or f"data:{self.mime_type};base64,{self.base64_data}"

Getting the URL in your LargeLanguageModel._invoke

In your _invoke method, you can access the file content like this:

def _invoke(self, model: str, credentials: dict, prompt_messages: list[PromptMessage], ...) -> Generator:
    for message in prompt_messages:
        if isinstance(message.content, list):
            for content in message.content:
                if isinstance(content, VideoPromptMessageContent):
                    # Use .data property which returns URL if available, or base64 data URI
                    file_data = content.data
                    
                    # Or access fields directly:
                    url = content.url  # Will be populated if MULTIMODAL_SEND_FORMAT=url
                    base64_data = content.base64_data  # Will be populated if MULTIMODAL_SEND_FORMAT=base64
                    mime_type = content.mime_type
                    filename = content.filename

Important: MULTIMODAL_SEND_FORMAT configuration

Whether you get a URL or base64 data depends on the MULTIMODAL_SEND_FORMAT environment variable in Dify:

  • MULTIMODAL_SEND_FORMAT=base64 (default): Files are sent as base64-encoded data
  • MULTIMODAL_SEND_FORMAT=url: Files are sent as signed URLs

If you need URLs, you'll need to set MULTIMODAL_SEND_FORMAT=url in your Dify deployment's environment configuration.

Hope this helps! Let me know if you have any further questions.

@Mairuis commented on GitHub (Dec 8, 2025): Hi @sweetwxh, Thanks for raising this question. Based on the SDK and Dify core code, here's how you can get the file URL in your custom LLM provider: ## How it works When files are passed to an LLM node, Dify converts them to `PromptMessageContent` objects (like `VideoPromptMessageContent`). The `MultiModalPromptMessageContent` base class has both `url` and `base64_data` fields: ```python class MultiModalPromptMessageContent(PromptMessageContent): format: str = Field(default=..., description="the format of multi-modal file") base64_data: str = Field(default="", description="the base64 data of multi-modal file") url: str = Field(default="", description="the url of multi-modal file") mime_type: str = Field(default=..., description="the mime type of multi-modal file") filename: str = Field(default="", description="the filename of multi-modal file") @property def data(self): return self.url or f"data:{self.mime_type};base64,{self.base64_data}" ``` ## Getting the URL in your LargeLanguageModel._invoke In your `_invoke` method, you can access the file content like this: ```python def _invoke(self, model: str, credentials: dict, prompt_messages: list[PromptMessage], ...) -> Generator: for message in prompt_messages: if isinstance(message.content, list): for content in message.content: if isinstance(content, VideoPromptMessageContent): # Use .data property which returns URL if available, or base64 data URI file_data = content.data # Or access fields directly: url = content.url # Will be populated if MULTIMODAL_SEND_FORMAT=url base64_data = content.base64_data # Will be populated if MULTIMODAL_SEND_FORMAT=base64 mime_type = content.mime_type filename = content.filename ``` ## Important: MULTIMODAL_SEND_FORMAT configuration Whether you get a URL or base64 data depends on the `MULTIMODAL_SEND_FORMAT` environment variable in Dify: - `MULTIMODAL_SEND_FORMAT=base64` (default): Files are sent as base64-encoded data - `MULTIMODAL_SEND_FORMAT=url`: Files are sent as signed URLs If you need URLs, you'll need to set `MULTIMODAL_SEND_FORMAT=url` in your Dify deployment's environment configuration. Hope this helps! Let me know if you have any further questions.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
documentation
enhancement
pull-request
Mirrored from GitHub Pull Request
 question
No Label question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-plugin-sdks#65
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?