langgenius/dify-plugin-sdks
1
0
Fork 0
mirror of https://github.com/langgenius/dify-plugin-sdks.git synced 2026-07-22 02:15:22 -04:00
Code Issues 58 Packages Projects Releases 10 Wiki Activity

[Feature]: OAICompatLargeLanguageModel: support video modal type #66

New Issue
Closed
opened 2026-02-15 21:15:31 -05:00 by yindo · 2 comments
yindo commented 2026-02-15 21:15:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @AkiraVoid on GitHub (Dec 1, 2025).

This issue is originally posted at https://github.com/langgenius/dify-official-plugins/issues/2127

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

I am using OpenAI API Compatible 0.0.25 and tried to upload file through Vision field and User Prompt, non of them being properly analyzed.

2. Additional context or comments

The current function of converting abstract PromptMessage into OpenAI API payload is defined as below:

https://github.com/langgenius/dify-plugin-sdks/blob/f1d201d9a88054cd70f08aaa12f8c13f8a6af6ed/python/dify_plugin/interfaces/model/openai_compatible/llm.py#L713-L781

which did not handle the video/audio/document modal properly (it just ignores them all).

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @AkiraVoid on GitHub (Dec 1, 2025). *This issue is originally posted at https://github.com/langgenius/dify-official-plugins/issues/2127* ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify-official-plugins/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. I am using OpenAI API Compatible 0.0.25 and tried to upload file through Vision field and User Prompt, non of them being properly analyzed. ### 2. Additional context or comments The current function of converting abstract PromptMessage into OpenAI API payload is defined as below: https://github.com/langgenius/dify-plugin-sdks/blob/f1d201d9a88054cd70f08aaa12f8c13f8a6af6ed/python/dify_plugin/interfaces/model/openai_compatible/llm.py#L713-L781 which did not handle the video/audio/document modal properly (it just ignores them all). ### 3. Can you help us with this feature? - [ ] I am interested in contributing to this feature.
yindo closed this issue 2026-02-15 21:15:31 -05:00
yindo commented 2026-02-15 21:15:32 -05:00
Author
Owner
Copy Link
Copy Source

@sweetwxh commented on GitHub (Dec 4, 2025):

Can you pass the video-type prompt to _invoke with the built-in URL if the video feature is enabled? Make sure this URL is publicly accessible; right now it’s a base64-encoded string. If the URL is passed directly, the operation should be much easier. Besides, I’d like to write my own video-processing logic inside the openai_api_compatible plugin, but at the moment I have to upload the file to a CDN first, which is a waste of time.

@sweetwxh commented on GitHub (Dec 4, 2025): Can you pass the video-type prompt to _invoke with the built-in URL if the video feature is enabled? Make sure this URL is publicly accessible; right now it’s a base64-encoded string. If the URL is passed directly, the operation should be much easier. Besides, I’d like to write my own video-processing logic inside the openai_api_compatible plugin, but at the moment I have to upload the file to a CDN first, which is a waste of time.
yindo commented 2026-02-15 21:15:32 -05:00
Author
Owner
Copy Link
Copy Source

@AkiraVoid commented on GitHub (Dec 4, 2025):

@sweetwxh Bro I am developing a custom plugin to solve this issue, and I can say that if your model provider supports video transferring in it's API (which is not supported by OpenAI), you can just add your own logic to handle this. I will close this issue as it's by design to not support video modal, because OpenAI does not support it, which means it's not a standard to allow video transferring in OpenAI Compatible API.

@AkiraVoid commented on GitHub (Dec 4, 2025): @sweetwxh Bro I am developing a custom plugin to solve this issue, and I can say that if your model provider supports video transferring in it's API (which is not supported by OpenAI), you can just add your own logic to handle this. I will close this issue as it's by design to not support video modal, because OpenAI does not support it, which means it's not a standard to allow video transferring in OpenAI Compatible API.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
documentation
enhancement
pull-request
Mirrored from GitHub Pull Request
 question
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-plugin-sdks#66
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?