langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-21 17:45:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

GPU OOM using Xinferences Qwen2.5-VL-32B via Difys xinference plugin with fewer images than direct Xinference API #249

New Issue
Closed
opened 2026-02-16 10:18:35 -05:00 by yindo · 4 comments
yindo commented 2026-02-16 10:18:35 -05:00
Owner
Copy Link
Copy Source

Originally created by @moshilangzi on GitHub (Apr 30, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues Dify issues & Dify Official Plugins, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.3.1

Plugin version

0.0.3

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

When using Dify to interact with a qwen2.5-vl-32B-instruct model deployed on Xinference for multi-modal tasks, uploading and processing 5 images results in a GPU Out-of-Memory (OOM) error on the Xinference side.
However, directly calling the Xinference API for the same model and task with 10 images works correctly and returns the results quickly without any memory issues. This suggests there might be a difference in how Dify handles multi-modal data or interacts with Xinference compared to a direct API call that leads to higher memory consumption for fewer images.

`

  1. Deploy qwen2.5-vl-32B-instruct model on a Xinference instance with 4*24G(4090 GPU), use VLLM engine .
  2. Configure Dify to use this Xinference instance and model as a custom model.
  3. Create/Use a Dify application that calls this multi-modal model.
  4. In the Dify application interface, upload 5 images and send a query to the model, XInference log shows GPU OOM.
  5. Observe the process/logs on the Xinference server where the model is running.`

✔️ Error log

GPU OOM

Originally created by @moshilangzi on GitHub (Apr 30, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [Dify issues](https://github.com/langgenius/dify/issues) & [Dify Official Plugins](https://github.com/langgenius/dify-official-plugins/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.3.1 ### Plugin version 0.0.3 ### Cloud or Self Hosted Self Hosted (Source) ### Steps to reproduce When using Dify to interact with a qwen2.5-vl-32B-instruct model deployed on Xinference for multi-modal tasks, uploading and processing 5 images results in a GPU Out-of-Memory (OOM) error on the Xinference side. However, directly calling the Xinference API for the same model and task with 10 images works correctly and returns the results quickly without any memory issues. This suggests there might be a difference in how Dify handles multi-modal data or interacts with Xinference compared to a direct API call that leads to higher memory consumption for fewer images. ` 1. Deploy qwen2.5-vl-32B-instruct model on a Xinference instance with 4*24G(4090 GPU), use VLLM engine . 2. Configure Dify to use this Xinference instance and model as a custom model. 3. Create/Use a Dify application that calls this multi-modal model. 4. In the Dify application interface, upload 5 images and send a query to the model, XInference log shows GPU OOM. 5. Observe the process/logs on the Xinference server where the model is running.` ### ✔️ Error log GPU OOM
yindo added the bug label 2026-02-16 10:18:35 -05:00
yindo closed this issue 2026-02-16 10:18:35 -05:00
yindo commented 2026-02-16 10:18:35 -05:00
Author
Owner
Copy Link
Copy Source

@Yeuoly commented on GitHub (May 6, 2025):

GPU OOM should not be a issue related to Dify side, as for the bad performance, maybe resolved in next release

@Yeuoly commented on GitHub (May 6, 2025): GPU OOM should not be a issue related to Dify side, as for the bad performance, maybe resolved in next release
yindo commented 2026-02-16 10:18:35 -05:00
Author
Owner
Copy Link
Copy Source

@moshilangzi commented on GitHub (May 7, 2025):

GPU OOM 应该不是 Dify 方面的问题，至于性能不佳，可能在下一个版本中得到解决

Is this the problem with dify-plugin-daemon?

@moshilangzi commented on GitHub (May 7, 2025): > GPU OOM 应该不是 Dify 方面的问题，至于性能不佳，可能在下一个版本中得到解决 Is this the problem with dify-plugin-daemon?
yindo commented 2026-02-16 10:18:35 -05:00
Author
Owner
Copy Link
Copy Source

@Yeuoly commented on GitHub (May 7, 2025):

Absolutely not.

@Yeuoly commented on GitHub (May 7, 2025): Absolutely not.
yindo commented 2026-02-16 10:18:35 -05:00
Author
Owner
Copy Link
Copy Source

@moshilangzi commented on GitHub (May 7, 2025):

绝对不是。

The VLLM plugin and Xinference plugin exhibit the same problem. Directly calling the VLLM and Xinference APIs to upload multiple images and perform multi-image question answering works correctly. However, when uploading multiple images using the VLLM and Xinference plugins on the Dify platform, it results in GPU memory overflow.

@moshilangzi commented on GitHub (May 7, 2025): > 绝对不是。 The VLLM plugin and Xinference plugin exhibit the same problem. Directly calling the VLLM and Xinference APIs to upload multiple images and perform multi-image question answering works correctly. However, when uploading multiple images using the VLLM and Xinference plugins on the Dify platform, it results in GPU memory overflow.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#249
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?