langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #9828] feat: support LLM understand video #26579

New Issue
Closed
opened 2026-02-21 20:39:51 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:39:51 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/9828

State: closed
Merged: Yes

Checklist:

Important

Please review the checklist below before submitting your pull request.

  • Please open an issue before creating a PR or link to an existing issue
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

Description

change list

  • support transfer a video file to a VideoPromptMessageContent
  • support qwen model understand video
  • support zhipu model understand video
  • when create a new agent app and open the vision feature, default allow video format files upload

restrictions

  • the qwen model requires users to submit a work order for application, and it only support url send mode
  • the exist agent app, its model config already saved in the database, so can't upload video format files

preview

local_video_file

remote_video_file

workflow_node

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update, included: Dify Document
  • Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement
  • Dependency upgrade

Testing Instructions

test locally

  • Test local video file send with base64 format
  • Test remote video file send with base64 format
  • Test remote video file send with url format
  • Test workflow's llm node

test link: https://cloud.video.taobao.com/vod/S8T54f_w1rkdfLdYjL3S5zKN9CrhkzuhRwOhF313tIQ.mp4
test file: https://github.com/user-attachments/assets/f5264e1f-e360-4b07-a201-da1216b0310a

**Original Pull Request:** https://github.com/langgenius/dify/pull/9828 **State:** closed **Merged:** Yes --- # Checklist: > [!IMPORTANT] > Please review the checklist below before submitting your pull request. - [ ] Please open an issue before creating a PR or link to an existing issue - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I ran `dev/reformat`(backend) and `cd web && npx lint-staged`(frontend) to appease the lint gods # Description ### change list - support transfer a video file to a `VideoPromptMessageContent` - support qwen model understand video - support zhipu model understand video - when create a new agent app and open the vision feature, default allow video format files upload ### restrictions - the qwen model requires users to submit a work order for application, and it only support `url` send mode - the exist agent app, its model config already saved in the database, so can't upload video format files ### preview ![local_video_file](https://github.com/user-attachments/assets/ca54f56d-32f8-423d-9414-ae71b9f0140f) ![remote_video_file](https://github.com/user-attachments/assets/fd07af52-eda7-41bf-ba67-a0d5a07a3ddf) ![workflow_node](https://github.com/user-attachments/assets/f866b8a2-2a24-4577-9373-8d08ad8c3243) ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs) - [ ] Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement - [ ] Dependency upgrade # Testing Instructions test locally - [x] Test local video file send with base64 format - [x] Test remote video file send with base64 format - [x] Test remote video file send with url format - [x] Test workflow's llm node test link: https://cloud.video.taobao.com/vod/S8T54f_w1rkdfLdYjL3S5zKN9CrhkzuhRwOhF313tIQ.mp4 test file: https://github.com/user-attachments/assets/f5264e1f-e360-4b07-a201-da1216b0310a
yindo added the pull-request label 2026-02-21 20:39:51 -05:00
yindo closed this issue 2026-02-21 20:39:51 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#26579
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?