langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 04:25:29 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

When uploading a picture to a large model, the API node will have 100% CPU usage. #12752

New Issue
Closed
opened 2026-02-21 19:08:50 -05:00 by yindo · 22 comments
yindo commented 2026-02-21 19:08:50 -05:00
Owner
Copy Link
Copy Source

Originally created by @jarvisaoieong on GitHub (Apr 10, 2025).

Originally assigned to: @Yeuoly on GitHub.

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.2.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

When uploading an image of about 1MB size and using any large model that supports Vision to return the meaning of the image, the API node in Docker will be stuck at 100% for a long time, the data received in Network IO will continue to rise, and the token output per second will drop to almost 1.

Image

✔️ Expected Behavior

CPU load returns to normal

❌ Actual Behavior

CPU 100%

Originally created by @jarvisaoieong on GitHub (Apr 10, 2025). Originally assigned to: @Yeuoly on GitHub. ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.2.0 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce When uploading an image of about 1MB size and using any large model that supports Vision to return the meaning of the image, the API node in Docker will be stuck at 100% for a long time, the data received in Network IO will continue to rise, and the token output per second will drop to almost 1. ![Image](https://github.com/user-attachments/assets/31182fb4-0e9a-46ca-b4a0-a1cd16f4f2cb) ### ✔️ Expected Behavior CPU load returns to normal ### ❌ Actual Behavior CPU 100%
yindo added the 🐞 bug label 2026-02-21 19:08:50 -05:00
yindo closed this issue 2026-02-21 19:08:51 -05:00
yindo commented 2026-02-21 19:08:51 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Apr 10, 2025):

Hey @jarvisaoieong! I'm Dosu, your friendly bot here to help you tackle bugs, answer questions, and make you a contributor while we wait for a human maintainer. Let's get this sorted! 🚀

The high CPU usage when uploading a 1MB image to a large model using the Dify API in a self-hosted Docker environment is likely due to the langfuse_client not automatically shutting down after API calls and event sending. To resolve this issue, you should explicitly call the shutdown method on the langfuse_client after each interface call completes. This approach has been shown to significantly improve CPU usage during stress testing [1].

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Apr 10, 2025): <!-- Greeting --> Hey @jarvisaoieong! I'm [Dosu](https://dosu.dev), your friendly bot here to help you tackle bugs, answer questions, and make you a contributor while we wait for a human maintainer. Let's get this sorted! 🚀 <!-- Answer --> The high CPU usage when uploading a 1MB image to a large model using the Dify API in a self-hosted Docker environment is likely due to the `langfuse_client` not automatically shutting down after API calls and event sending. To resolve this issue, you should explicitly call the shutdown method on the `langfuse_client` after each interface call completes. This approach has been shown to significantly improve CPU usage during stress testing <sup>[[1]](https://github.com/langgenius/dify/issues/12496)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/f64fc260-e905-47f8-a775-471678cb299c?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/f64fc260-e905-47f8-a775-471678cb299c?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/f64fc260-e905-47f8-a775-471678cb299c?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/f64fc260-e905-47f8-a775-471678cb299c?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/f64fc260-e905-47f8-a775-471678cb299c?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/f64fc260-e905-47f8-a775-471678cb299c?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/f64fc260-e905-47f8-a775-471678cb299c?feedback_type=other)</sup>
yindo commented 2026-02-21 19:08:51 -05:00
Author
Owner
Copy Link
Copy Source

@kurokobo commented on GitHub (Apr 10, 2025):

This might be related to this: https://github.com/langgenius/dify-official-plugins/issues/648

@kurokobo commented on GitHub (Apr 10, 2025): This might be related to this: https://github.com/langgenius/dify-official-plugins/issues/648
yindo commented 2026-02-21 19:08:51 -05:00
Author
Owner
Copy Link
Copy Source

@kurokobo commented on GitHub (Apr 10, 2025):

@crazywoola @laipz8200 @Yeuoly
I encountered a similar problem, so I conducted some investigation.

The current implementation has an issue where sending a large (in MBytes) prompt containing images or a large amount of text to a model that supports streaming leads to CPU and network wastage, significantly degrading the performance.

This can potentially lead to serious load not only for the free community users but also on the inter-cluster network in the Enterprise version running on multi-node Kubernetes, so I'm tagging you.

Image

The plugin daemon returns objects of the type LLMResultChunk when responding with chunks to the API. Although only LLMResultChunkDelta is truly necessary for the response, LLMResultChunk includes prompt_messages.

https://github.com/langgenius/dify-plugin-sdks/blob/8310e775335a655bc5ff356b23d37bcb28616d13/python/dify_plugin/entities/model/llm.py#L105-L116

In other words, the plugin daemon sends the complete prompt_messages to the API every time it returns single chunk. As a result, the API receives prompt_messages repeatedly for each chunk.

Since prompt_messages can include text extracted from files and images encoded in Base64, their size can easily exceed several megabytes.

As a result, the API continues to receive and process several megabytes of data in a short period of time for the number of chunks, leading to increased CPU usage and higher network load.

Tested with the DSL and image which is attached in https://github.com/langgenius/dify-official-plugins/issues/648 with gpt-4o-mini, and I can confirm followings:

  • Sending 2.6 MB image to gpt-4o-mini takes 8 minutes to complete renderding super short text with 79 tokens.
    Image
  • By capturing all packets inter-container network during generation, I can confirm the API (172.18.0.9) sends 4 MB to daemon (172.18.0.7) in total, but daemon responds 277 MB to API in total
    Image
  • Perhaps this means that the 2.6 MB image encoded in Base64 became 3.5 MB, and when the prompt_messages were sent back to the API in 79 chunks, the size was almost the same (277 MB = 3.5 MB * 79).
@kurokobo commented on GitHub (Apr 10, 2025): @crazywoola @laipz8200 @Yeuoly I encountered a similar problem, so I conducted some investigation. The current implementation has an issue where sending a large (in MBytes) prompt containing images or a large amount of text to a model that supports streaming leads to CPU and network wastage, significantly degrading the performance. This can potentially lead to serious load not only for the free community users but also on the inter-cluster network in the Enterprise version running on multi-node Kubernetes, so I'm tagging you. ![Image](https://github.com/user-attachments/assets/a8a921df-048e-418c-98bd-763561bcde83) The plugin daemon returns objects of the type `LLMResultChunk` when responding with chunks to the API. Although only `LLMResultChunkDelta` is truly necessary for the response, **`LLMResultChunk` includes `prompt_messages`**. https://github.com/langgenius/dify-plugin-sdks/blob/8310e775335a655bc5ff356b23d37bcb28616d13/python/dify_plugin/entities/model/llm.py#L105-L116 In other words, the plugin daemon sends the complete `prompt_messages` to the API every time it returns **single chunk**. As a result, **the API receives `prompt_messages` repeatedly for each chunk**. Since `prompt_messages` can include text extracted from files and images encoded in Base64, their size can easily exceed several megabytes. As a result, the API continues to receive and process several megabytes of data in a short period of time for the number of chunks, leading to increased CPU usage and higher network load. Tested with the DSL and image which is attached in https://github.com/langgenius/dify-official-plugins/issues/648 with gpt-4o-mini, and I can confirm followings: - Sending 2.6 MB image to gpt-4o-mini takes **8 minutes to complete** renderding super short text with 79 tokens. ![Image](https://github.com/user-attachments/assets/c0a60a9f-6cd2-44cb-a52b-a7996a70bc8f) - By capturing all packets inter-container network during generation, I can confirm the API (172.18.0.9) sends 4 MB to daemon (172.18.0.7) in total, but **daemon responds 277 MB to API in total** ![Image](https://github.com/user-attachments/assets/68f7b456-2abf-49fe-9ce9-4b99d039fc7d) - Perhaps this means that the 2.6 MB image encoded in Base64 became 3.5 MB, and when the `prompt_messages` were sent back to the API in 79 chunks, the size was almost the same (277 MB = 3.5 MB * 79).
yindo commented 2026-02-21 19:08:51 -05:00
Author
Owner
Copy Link
Copy Source

@kurokobo commented on GitHub (Apr 10, 2025):

@hjlarry @Hzao @linqinguang
I would like to share my comment above as it might partially explain your issue.

@kurokobo commented on GitHub (Apr 10, 2025): @hjlarry @Hzao @linqinguang I would like to share my comment above as it might partially explain your issue.
yindo commented 2026-02-21 19:08:51 -05:00
Author
Owner
Copy Link
Copy Source

@Hzao commented on GitHub (Apr 10, 2025):

@hjlarry @Hzao @linqinguang
I would like to share my comment above as it might partially explain your issue.

Thanks for the thorough investigation! That's an awesome job!

I thought it's only an issue in Gemini plugin at first. Looks like it could have a pretty big impact across multiple models in cloud/community/enterprise environments. 🤔

@Hzao commented on GitHub (Apr 10, 2025): > @hjlarry @Hzao @linqinguang > I would like to share my comment above as it might partially explain your issue. Thanks for the thorough investigation! That's an awesome job! I thought it's only an issue in Gemini plugin at first. Looks like it could have a pretty big impact across multiple models in cloud/community/enterprise environments. 🤔
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@hjlarry commented on GitHub (Apr 10, 2025):

@hjlarry @Hzao @linqinguang I would like to share my comment above as it might partially explain your issue.

Cool ! Great work!

I've also added some logs to track the sending of a 5MB image from plugin-sdk to the plugin-daemon. It generates 130MB of logs just for a 5MB image, and lots of repeated content in the log, which supports your point.

plugin sdk:
Image

plugin daemon:
Image

@hjlarry commented on GitHub (Apr 10, 2025): > [@hjlarry](https://github.com/hjlarry) [@Hzao](https://github.com/Hzao) [@linqinguang](https://github.com/linqinguang) I would like to share my comment above as it might partially explain your issue. Cool ! Great work! I've also added some logs to track the sending of a 5MB image from `plugin-sdk` to the `plugin-daemon`. It generates 130MB of logs just for a 5MB image, and lots of repeated content in the log, which supports your point. plugin sdk: ![Image](https://github.com/user-attachments/assets/43113b52-4e67-4e42-ba42-a0228f128a39) plugin daemon: ![Image](https://github.com/user-attachments/assets/30e9ec78-cfc8-4b9b-ba72-75ae2cee732f)
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@hejuntt1014 commented on GitHub (Apr 11, 2025):

fix in https://github.com/langgenius/dify/commit/d3157b46eea7de9a4beef00bcd7dbbbb9dd8f544 @jarvisaoieong @kurokobo @hjlarry

@hejuntt1014 commented on GitHub (Apr 11, 2025): fix in https://github.com/langgenius/dify/commit/d3157b46eea7de9a4beef00bcd7dbbbb9dd8f544 @jarvisaoieong @kurokobo @hjlarry
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@kurokobo commented on GitHub (Apr 11, 2025):

@hejuntt1014

fix in https://github.com/langgenius/dify/commit/d3157b46eea7de9a4beef00bcd7dbbbb9dd8f544 @jarvisaoieong @kurokobo @hjlarry

I don't think so; I believe it is a different issue from the token count.

@kurokobo commented on GitHub (Apr 11, 2025): @hejuntt1014 > fix in https://github.com/langgenius/dify/commit/d3157b46eea7de9a4beef00bcd7dbbbb9dd8f544 @jarvisaoieong @kurokobo @hjlarry I don't think so; I believe it is a different issue from the token count.
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@hejuntt1014 commented on GitHub (Apr 11, 2025):

@hejuntt1014

D3157B4 中的修复

我不这么认为;我相信这与令牌计数是不同的问题。

It seems that this issue is the same as the one I'm experiencing.
https://github.com/langgenius/dify-plugin-sdks/pull/81#issuecomment-2792380131

@hejuntt1014 commented on GitHub (Apr 11, 2025): > [@hejuntt1014](https://github.com/hejuntt1014) > > > [D3157B4](https://github.com/langgenius/dify/commit/d3157b46eea7de9a4beef00bcd7dbbbb9dd8f544) 中的修复 > > 我不这么认为;我相信这与令牌计数是不同的问题。 It seems that this issue is the same as the one I'm experiencing. https://github.com/langgenius/dify-plugin-sdks/pull/81#issuecomment-2792380131
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@kurokobo commented on GitHub (Apr 11, 2025):

Related PRs:

@kurokobo commented on GitHub (Apr 11, 2025): Related PRs: - SDK - https://github.com/langgenius/dify-plugin-sdks/pull/81 - https://github.com/langgenius/dify-plugin-sdks/pull/88 - ... - Daemon - https://github.com/langgenius/dify-plugin-daemon/pull/194 - ... - Dify - https://github.com/langgenius/dify/pull/17867 - https://github.com/langgenius/dify/pull/17883 - ...
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@kurokobo commented on GitHub (Apr 11, 2025):

@hejuntt1014
Thanks! I have read your PR and its related issue. It seems that there was indeed a problem with increased CPU usage related to the token calculations and response decoding processes.
My comments mainly concern the fact that the Daemon generates a large amount of network traffic, which will not be improved even if the token calculations are no longer performed.

However as listed in my comment above, your PR has been merged, and thanks to @Yeuoly it seems the solutions are actively implemented. Let's hope everything gets improved in the next release :)

That said, the cause of the error when sending large images in langgenius/dify-official-plugins#648 has not been thoroughly investigated yet...

@kurokobo commented on GitHub (Apr 11, 2025): @hejuntt1014 Thanks! I have read your PR and its related issue. It seems that there was indeed a problem with increased CPU usage related to the token calculations and response decoding processes. My comments mainly concern the fact that the Daemon generates a large amount of network traffic, which will not be improved even if the token calculations are no longer performed. However as listed in my comment above, your PR has been merged, and thanks to @Yeuoly it seems the solutions are actively implemented. Let's hope everything gets improved in the next release :) That said, the cause of the error when sending large images in langgenius/dify-official-plugins#648 has not been thoroughly investigated yet...
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Apr 11, 2025):

Related issues:

@crazywoola commented on GitHub (Apr 11, 2025): Related issues: - https://github.com/langgenius/dify/issues/17673 - https://github.com/langgenius/dify/issues/17451 - https://github.com/langgenius/dify/issues/17442
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@kurokobo commented on GitHub (Apr 12, 2025):

@hjlarry @Hzao
If you are interested in this, we can test the latest (which includes fixes for our issue) images and SDKs locally.

Place this docker-compose.override.yaml on the next to your docker-compose.yaml:

services:
  api:
    image: langgenius/dify-api:main

  worker:
    image: langgenius/dify-api:main

  web:
    image: langgenius/dify-web:main

  sandbox:
    image: langgenius/dify-sandbox:main

  plugin_daemon:
    image: langgenius/dify-plugin-daemon:main-local
    environment:
      FORCE_VERIFYING_SIGNATURE: false
    command: /bin/bash -c "apt-get update && apt-get install -y git && /app/entrypoint.sh"

Then in dify-official-plugins, update requirements.txt for your provider with following line to make the difypkg to use the latest SDK from GitHub:

dify_plugin @ git+https://github.com/langgenius/dify-plugin-sdks@main#subdirectory=python

Finally build the difypkg and install it.

After a quick test on my home lab, the query with IMG_Larger.png took a bit of time, but at least it responded without any errors.

@kurokobo commented on GitHub (Apr 12, 2025): @hjlarry @Hzao If you are interested in this, we can test the latest (which includes fixes for our issue) images and SDKs locally. Place this `docker-compose.override.yaml` on the next to your `docker-compose.yaml`: ```yaml services: api: image: langgenius/dify-api:main worker: image: langgenius/dify-api:main web: image: langgenius/dify-web:main sandbox: image: langgenius/dify-sandbox:main plugin_daemon: image: langgenius/dify-plugin-daemon:main-local environment: FORCE_VERIFYING_SIGNATURE: false command: /bin/bash -c "apt-get update && apt-get install -y git && /app/entrypoint.sh" ``` Then in `dify-official-plugins`, update `requirements.txt` for your provider with following line to make the difypkg to use the latest SDK from GitHub: ```text dify_plugin @ git+https://github.com/langgenius/dify-plugin-sdks@main#subdirectory=python ``` Finally build the difypkg and install it. After a quick test on my home lab, the query with `IMG_Larger.png` took a bit of time, but at least it responded without any errors.
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@hjlarry commented on GitHub (Apr 12, 2025):

@hjlarry @Hzao If you are interested in this, we can test the latest (which includes fixes for our issue) images and SDKs locally.

Place this docker-compose.override.yaml on the next to your docker-compose.yaml:

services:
api:
image: langgenius/dify-api:main

worker:
image: langgenius/dify-api:main

web:
image: langgenius/dify-web:main

sandbox:
image: langgenius/dify-sandbox:main

plugin_daemon:
image: langgenius/dify-plugin-daemon:main-local
environment:
FORCE_VERIFYING_SIGNATURE: false
command: /bin/bash -c "apt-get update && apt-get install -y git && /app/entrypoint.sh"
Then in dify-official-plugins, update requirements.txt for your provider with following line to make the difypkg to use the latest SDK from GitHub:

dify_plugin @ git+https://github.com/langgenius/dify-plugin-sdks@main#subdirectory=python

Finally build the difypkg and install it.

After a quick test on my home lab, the query with IMG_Larger.png took a bit of time, but at least it responded without any errors.

It works. the IMG_Larger.png now takes 10s

@hjlarry commented on GitHub (Apr 12, 2025): > [@hjlarry](https://github.com/hjlarry) [@Hzao](https://github.com/Hzao) If you are interested in this, we can test the latest (which includes fixes for our issue) images and SDKs locally. > > Place this `docker-compose.override.yaml` on the next to your `docker-compose.yaml`: > > services: > api: > image: langgenius/dify-api:main > > worker: > image: langgenius/dify-api:main > > web: > image: langgenius/dify-web:main > > sandbox: > image: langgenius/dify-sandbox:main > > plugin_daemon: > image: langgenius/dify-plugin-daemon:main-local > environment: > FORCE_VERIFYING_SIGNATURE: false > command: /bin/bash -c "apt-get update && apt-get install -y git && /app/entrypoint.sh" > Then in `dify-official-plugins`, update `requirements.txt` for your provider with following line to make the difypkg to use the latest SDK from GitHub: > > ``` > dify_plugin @ git+https://github.com/langgenius/dify-plugin-sdks@main#subdirectory=python > ``` > > Finally build the difypkg and install it. > > After a quick test on my home lab, the query with `IMG_Larger.png` took a bit of time, but at least it responded without any errors. It works. the `IMG_Larger.png` now takes 10s
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@Hzao commented on GitHub (Apr 12, 2025):

@hjlarry @Hzao If you are interested in this, we can test the latest (which includes fixes for our issue) images and SDKs locally.

Place this docker-compose.override.yaml on the next to your docker-compose.yaml:

services:
api:
image: langgenius/dify-api:main

worker:
image: langgenius/dify-api:main

web:
image: langgenius/dify-web:main

sandbox:
image: langgenius/dify-sandbox:main

plugin_daemon:
image: langgenius/dify-plugin-daemon:main-local
environment:
FORCE_VERIFYING_SIGNATURE: false
command: /bin/bash -c "apt-get update && apt-get install -y git && /app/entrypoint.sh"
Then in dify-official-plugins, update requirements.txt for your provider with following line to make the difypkg to use the latest SDK from GitHub:

dify_plugin @ git+https://github.com/langgenius/dify-plugin-sdks@main#subdirectory=python

Finally build the difypkg and install it.

After a quick test on my home lab, the query with IMG_Larger.png took a bit of time, but at least it responded without any errors.

I confirm it has been resolved!

@Hzao commented on GitHub (Apr 12, 2025): > [@hjlarry](https://github.com/hjlarry) [@Hzao](https://github.com/Hzao) If you are interested in this, we can test the latest (which includes fixes for our issue) images and SDKs locally. > > Place this `docker-compose.override.yaml` on the next to your `docker-compose.yaml`: > > services: > api: > image: langgenius/dify-api:main > > worker: > image: langgenius/dify-api:main > > web: > image: langgenius/dify-web:main > > sandbox: > image: langgenius/dify-sandbox:main > > plugin_daemon: > image: langgenius/dify-plugin-daemon:main-local > environment: > FORCE_VERIFYING_SIGNATURE: false > command: /bin/bash -c "apt-get update && apt-get install -y git && /app/entrypoint.sh" > Then in `dify-official-plugins`, update `requirements.txt` for your provider with following line to make the difypkg to use the latest SDK from GitHub: > > ``` > dify_plugin @ git+https://github.com/langgenius/dify-plugin-sdks@main#subdirectory=python > ``` > > Finally build the difypkg and install it. > > After a quick test on my home lab, the query with `IMG_Larger.png` took a bit of time, but at least it responded without any errors. I confirm it has been resolved!
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@ChnMig commented on GitHub (Apr 14, 2025):

@hjlarry @Hzao If you are interested in this, we can test the latest (which includes fixes for our issue) images and SDKs locally.

Place this docker-compose.override.yaml on the next to your docker-compose.yaml:

services:
api:
image: langgenius/dify-api:main

worker:
image: langgenius/dify-api:main

web:
image: langgenius/dify-web:main

sandbox:
image: langgenius/dify-sandbox:main

plugin_daemon:
image: langgenius/dify-plugin-daemon:main-local
environment:
FORCE_VERIFYING_SIGNATURE: false
command: /bin/bash -c "apt-get update && apt-get install -y git && /app/entrypoint.sh"
Then in dify-official-plugins, update requirements.txt for your provider with following line to make the difypkg to use the latest SDK from GitHub:

dify_plugin @ git+https://github.com/langgenius/dify-plugin-sdks@main#subdirectory=python

Finally build the difypkg and install it.

After a quick test on my home lab, the query with IMG_Larger.png took a bit of time, but at least it responded without any errors.

Sorry, I also encountered the same problem and urgently needed to be solved. I checked your method, but I didn't understand it. What does the 'dify-official-plugins' mean after modifying docker-compose.yaml?

@ChnMig commented on GitHub (Apr 14, 2025): > [@hjlarry](https://github.com/hjlarry) [@Hzao](https://github.com/Hzao) If you are interested in this, we can test the latest (which includes fixes for our issue) images and SDKs locally. > > Place this `docker-compose.override.yaml` on the next to your `docker-compose.yaml`: > > services: > api: > image: langgenius/dify-api:main > > worker: > image: langgenius/dify-api:main > > web: > image: langgenius/dify-web:main > > sandbox: > image: langgenius/dify-sandbox:main > > plugin_daemon: > image: langgenius/dify-plugin-daemon:main-local > environment: > FORCE_VERIFYING_SIGNATURE: false > command: /bin/bash -c "apt-get update && apt-get install -y git && /app/entrypoint.sh" > Then in `dify-official-plugins`, update `requirements.txt` for your provider with following line to make the difypkg to use the latest SDK from GitHub: > > ``` > dify_plugin @ git+https://github.com/langgenius/dify-plugin-sdks@main#subdirectory=python > ``` > > Finally build the difypkg and install it. > > After a quick test on my home lab, the query with `IMG_Larger.png` took a bit of time, but at least it responded without any errors. Sorry, I also encountered the same problem and urgently needed to be solved. I checked your method, but I didn't understand it. What does the 'dify-official-plugins' mean after modifying docker-compose.yaml?
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@kurokobo commented on GitHub (Apr 14, 2025):

@ChnMig
I absolutely do not recommend trying my method outside of testing or validation. For most users, especially if it’s in a production environment, the recommended procedure is to wait for the next release and then upgrade.

The main images are not a stable release and there may be committed changes that get reverted. In this case, reverting an upgraded environment that was accidentally done requires technically complex interventions, which can cause significant disruption in a production environment. Personally, I do not recommend this procedure at all if you do not fully understand what you are doing.

@kurokobo commented on GitHub (Apr 14, 2025): @ChnMig I absolutely do not recommend trying my method outside of testing or validation. For most users, especially if it’s in a production environment, the recommended procedure is to wait for the next release and then upgrade. The `main` images are not a stable release and there may be committed changes that get reverted. In this case, reverting an upgraded environment that was accidentally done requires technically complex interventions, which can cause significant disruption in a production environment. Personally, I do not recommend this procedure at all if you do not fully understand what you are doing.
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@ChnMig commented on GitHub (Apr 14, 2025):

@ChnMig I absolutely do not recommend trying my method outside of testing or validation. For most users, especially if it’s in a production environment, the recommended procedure is to wait for the next release and then upgrade.

The main images are not a stable release and there may be committed changes that get reverted. In this case, reverting an upgraded environment that was accidentally done requires technically complex interventions, which can cause significant disruption in a production environment. Personally, I do not recommend this procedure at all if you do not fully understand what you are doing.

You are right, I noticed that the mirror of main is always updated, so I decided to wait for the new version to be released, thanks for your advice

@ChnMig commented on GitHub (Apr 14, 2025): > [@ChnMig](https://github.com/ChnMig) I absolutely do not recommend trying my method outside of testing or validation. For most users, especially if it’s in a production environment, the recommended procedure is to wait for the next release and then upgrade. > > The `main` images are not a stable release and there may be committed changes that get reverted. In this case, reverting an upgraded environment that was accidentally done requires technically complex interventions, which can cause significant disruption in a production environment. Personally, I do not recommend this procedure at all if you do not fully understand what you are doing. You are right, I noticed that the mirror of main is always updated, so I decided to wait for the new version to be released, thanks for your advice
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@jarvisaoieong commented on GitHub (Apr 15, 2025):

@hjlarry @Hzao If you are interested in this, we can test the latest (which includes fixes for our issue) images and SDKs locally.

Place this docker-compose.override.yaml on the next to your docker-compose.yaml:

services:
api:
image: langgenius/dify-api:main

worker:
image: langgenius/dify-api:main

web:
image: langgenius/dify-web:main

sandbox:
image: langgenius/dify-sandbox:main

plugin_daemon:
image: langgenius/dify-plugin-daemon:main-local
environment:
FORCE_VERIFYING_SIGNATURE: false
command: /bin/bash -c "apt-get update && apt-get install -y git && /app/entrypoint.sh"
Then in dify-official-plugins, update requirements.txt for your provider with following line to make the difypkg to use the latest SDK from GitHub:

dify_plugin @ git+https://github.com/langgenius/dify-plugin-sdks@main#subdirectory=python

Finally build the difypkg and install it.

After a quick test on my home lab, the query with IMG_Larger.png took a bit of time, but at least it responded without any errors.

thank you @kurokobo
I have tried this solution. the api node cpu problem have solved.
But I found the speed is too slow compared to version 0.15.3 or dify plugin daemon debug runtime.

My test is uploading an 3MB image for LLM to describe the image. (I have tested gemini, gpt4o-mini)
It spends average 12 seconds using the local difypkg solution or offical plugin. and the plugin daemon docker node have a brief spikes in CPU usage to 100%

Image

Image

In the local debug runtime, it only spend average 3 seconds. same as the dify version 0.15.3. and there is no cpu issue.

Image

May be there is another performance problem in the plugin daemon local runtime

@jarvisaoieong commented on GitHub (Apr 15, 2025): > [@hjlarry](https://github.com/hjlarry) [@Hzao](https://github.com/Hzao) If you are interested in this, we can test the latest (which includes fixes for our issue) images and SDKs locally. > > Place this `docker-compose.override.yaml` on the next to your `docker-compose.yaml`: > > services: > api: > image: langgenius/dify-api:main > > worker: > image: langgenius/dify-api:main > > web: > image: langgenius/dify-web:main > > sandbox: > image: langgenius/dify-sandbox:main > > plugin_daemon: > image: langgenius/dify-plugin-daemon:main-local > environment: > FORCE_VERIFYING_SIGNATURE: false > command: /bin/bash -c "apt-get update && apt-get install -y git && /app/entrypoint.sh" > Then in `dify-official-plugins`, update `requirements.txt` for your provider with following line to make the difypkg to use the latest SDK from GitHub: > > ``` > dify_plugin @ git+https://github.com/langgenius/dify-plugin-sdks@main#subdirectory=python > ``` > > Finally build the difypkg and install it. > > After a quick test on my home lab, the query with `IMG_Larger.png` took a bit of time, but at least it responded without any errors. thank you @kurokobo I have tried this solution. the api node cpu problem have solved. But I found the speed is too slow compared to version 0.15.3 or dify plugin daemon debug runtime. My test is uploading an 3MB image for LLM to describe the image. (I have tested gemini, gpt4o-mini) It spends average 12 seconds using the local difypkg solution or offical plugin. and the plugin daemon docker node have a brief spikes in CPU usage to 100% ![Image](https://github.com/user-attachments/assets/c2acfe8c-2a3e-4529-9005-5863c97f3a71) ![Image](https://github.com/user-attachments/assets/66378e98-e4f9-4dab-993f-147bdebd2d0f) In the local debug runtime, it only spend average 3 seconds. same as the dify version 0.15.3. and there is no cpu issue. ![Image](https://github.com/user-attachments/assets/7c4c2a44-dee2-45e7-a70a-6865b7baa1ae) May be there is another performance problem in the plugin daemon local runtime
yindo commented 2026-02-21 19:08:52 -05:00
Author
Owner
Copy Link
Copy Source

@kurokobo commented on GitHub (Apr 15, 2025):

@jarvisaoieong
Yes, I have also confirmed that there are CPU spikes and performance degradation under certain conditions.

The situation has undoubtedly improved since there are no errors occurring, network load has decreased, and performance degradation caused by locking has been removed. However, it is true that it is still not perfect, and it seems that there may be another issue lurking.

As for the another issue, it would be better to open a new one.

@kurokobo commented on GitHub (Apr 15, 2025): @jarvisaoieong Yes, I have also confirmed that there are CPU spikes and performance degradation under certain conditions. The situation has undoubtedly improved since there are no errors occurring, network load has decreased, and performance degradation caused by locking has been removed. However, it is true that it is still not perfect, and it seems that there may be another issue lurking. As for the another issue, it would be better to open a new one.
yindo commented 2026-02-21 19:08:53 -05:00
Author
Owner
Copy Link
Copy Source

@lichao4Java commented on GitHub (Apr 15, 2025):

Related PRs:

When will the new version be released? I can't wait any longer

@lichao4Java commented on GitHub (Apr 15, 2025): > Related PRs: > > * SDK > > * [Fix: Convert shared class variables to instance variables to solve lock contention and CPU issues dify-plugin-sdks#81](https://github.com/langgenius/dify-plugin-sdks/pull/81) > * [enhancement: remove prompt_messages from LLMResult and LLMResultChunk dify-plugin-sdks#88](https://github.com/langgenius/dify-plugin-sdks/pull/88) > * ... > * Daemon > > * [enhancement(llm): remove non-required PromptMessages field from LLMResultChunk dify-plugin-daemon#194](https://github.com/langgenius/dify-plugin-daemon/pull/194) > * ... > * Dify > > * [enhance(plugin): replace json.loads with Pydantic model_validate_json #17867](https://github.com/langgenius/dify/pull/17867) > * [feat: re-add prompt messages to result and chunks in llm #17883](https://github.com/langgenius/dify/pull/17883) > * ... When will the new version be released? I can't wait any longer
yindo commented 2026-02-21 19:08:53 -05:00
Author
Owner
Copy Link
Copy Source

@Yeuoly commented on GitHub (Apr 17, 2025):

Another patch for the performance issue https://github.com/langgenius/dify-plugin-sdks/pull/103

@Yeuoly commented on GitHub (Apr 17, 2025): Another patch for the performance issue https://github.com/langgenius/dify-plugin-sdks/pull/103
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#12752
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?