langgenius/dify-plugin-sdks
1
0
Fork 0
mirror of https://github.com/langgenius/dify-plugin-sdks.git synced 2026-07-22 02:15:22 -04:00
Code Issues 58 Packages Projects Releases 10 Wiki Activity

[PR #128] [MERGED] fix: improve buffer handling for multimodal input #166

New Issue
Closed
opened 2026-02-15 21:16:03 -05:00 by yindo · 0 comments
yindo commented 2026-02-15 21:16:03 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/langgenius/dify-plugin-sdks/pull/128
Author: @kurokobo
Created: 5/1/2025
Status: ✅ Merged
Merged: 5/6/2025
Merged by: @Yeuoly

Base: main ← Head: massive_performance_improvement

📝 Commits (5)

  • 9d8dcab fix: improve buffer handling
  • 87c5947 tests:add tests for StdioRequestReader
  • 4e1a396 fix: remove useless code
  • 1e9a8ea fix: handle empty lines in StdioRequestReader
  • 684d425 apply ruff

📊 Changes

2 files changed (+90 additions, -6 deletions)

View changed files

📝 python/dify_plugin/core/server/stdio/request_reader.py (+14 -6)
➕ python/tests/servers/test_stdio.py (+76 -0)

📄 Description

Summary

This (probably) resolves the issues that had been widely reported regarding high CPU usage and increased processing times with multimodal input.

There's still probably room for further speed improvements, such as by changing how JSON serialization is handled, but for now, I'm submitting this PR to address the main cause.

Related issues:

Detailed Description

There are mainly two major issues with the current implementation, both related to the request_reader:

  • L24: It only receives up to 512 bytes at a time from standard input
  • L32: It tries to split the buffer containing all data received so far

https://github.com/langgenius/dify-plugin-sdks/blob/a35cb4c5e6aaa3f6d419727a057b29a89300b534/python/dify_plugin/core/server/stdio/request_reader.py#L24-L32

Because of this, for example, if we attach a 5MB image, the following occurs:

  • The split operation runs about 13,000 times
    • 5 MB * 1.33 (base64 overhead) / 512 bytes
  • For each split, processing is done on up to 6.7 MB of data
    • Because split is for buffer instead of data

This PR makes the following changes:

  • Increases the maximum read size from stdin to 64 KB (x 128)
  • Before running split against buffer, it first uses find on data (up to 64 KB); if it doesn’t find a match, it does nothing
  • Only runs split against buffer when a b"\n" is found, and then proceeds with the rest of the processing

With these changes, the results are:

  • Current: split runs 13,000 times on up to 6.7 MB of data each time
  • With this PR: find runs only 100 times on up to 64 KB of data, and there is only a single 6.7 MB split

Screenshots

Tested with attached DSL and large image on https://github.com/langgenius/dify-official-plugins/issues/648 by @Hzao (thanks for good assets👍)

Current implementation: 61s
03
image

With this PR: 12s
04
image

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/langgenius/dify-plugin-sdks/pull/128 **Author:** [@kurokobo](https://github.com/kurokobo) **Created:** 5/1/2025 **Status:** ✅ Merged **Merged:** 5/6/2025 **Merged by:** [@Yeuoly](https://github.com/Yeuoly) **Base:** `main` ← **Head:** `massive_performance_improvement` --- ### 📝 Commits (5) - [`9d8dcab`](https://github.com/langgenius/dify-plugin-sdks/commit/9d8dcab39d0075d0594c6747221d910521492151) fix: improve buffer handling - [`87c5947`](https://github.com/langgenius/dify-plugin-sdks/commit/87c59472cce763b3a3cdc0290525b1081319f695) tests:add tests for StdioRequestReader - [`4e1a396`](https://github.com/langgenius/dify-plugin-sdks/commit/4e1a3969b822629628509e22be7fafb6cdde3240) fix: remove useless code - [`1e9a8ea`](https://github.com/langgenius/dify-plugin-sdks/commit/1e9a8ea9a2e1e8e147fe180da7bb9bb0c3145cc9) fix: handle empty lines in StdioRequestReader - [`684d425`](https://github.com/langgenius/dify-plugin-sdks/commit/684d425a94761c5f92dda3e53d606ddeb20513de) apply ruff ### 📊 Changes **2 files changed** (+90 additions, -6 deletions) <details> <summary>View changed files</summary> 📝 `python/dify_plugin/core/server/stdio/request_reader.py` (+14 -6) ➕ `python/tests/servers/test_stdio.py` (+76 -0) </details> ### 📄 Description ## Summary This (probably) resolves the issues that had been widely reported regarding high CPU usage and increased processing times with multimodal input. There's still probably room for further speed improvements, such as by changing how JSON serialization is handled, but for now, I'm submitting this PR to address the main cause. ## Related issues: - https://github.com/langgenius/dify/issues/17799 - https://github.com/langgenius/dify/issues/19052 - closes https://github.com/langgenius/dify/issues/16441 - closes https://github.com/langgenius/dify-official-plugins/issues/757 - closes https://github.com/langgenius/dify-official-plugins/issues/779 - and more ## Detailed Description There are mainly two major issues with the current implementation, both related to the `request_reader`: - L24: It only receives up to 512 bytes at a time from standard input - L32: It tries to `split` the `buffer` containing all data received so far https://github.com/langgenius/dify-plugin-sdks/blob/a35cb4c5e6aaa3f6d419727a057b29a89300b534/python/dify_plugin/core/server/stdio/request_reader.py#L24-L32 Because of this, for example, if we attach a 5MB image, the following occurs: - The `split` operation runs about **13,000 times** - 5 MB * 1.33 (base64 overhead) / 512 bytes - For each `split`, processing is done on up to **6.7 MB of data** - Because `split` is for `buffer` instead of `data` This PR makes the following changes: - Increases the maximum read size from stdin to **64 KB** (x 128) - Before running `split` against `buffer`, it first uses `find` on `data` (up to 64 KB); if it doesn’t find a match, it does nothing - Only runs `split` against `buffer` when a `b"\n"` is found, and then proceeds with the rest of the processing With these changes, the results are: - Current: **`split` runs 13,000 times on up to 6.7 MB of data each time** - With this PR: **`find` runs only 100 times on up to 64 KB of data, and there is only a single 6.7 MB `split`** ## Screenshots Tested with attached DSL and large image on https://github.com/langgenius/dify-official-plugins/issues/648 by @Hzao (thanks for good assets👍) **Current implementation**: **61s** ![03](https://github.com/user-attachments/assets/135c9ad5-ec63-4834-9a49-79ed06f070c1) ![image](https://github.com/user-attachments/assets/fd50cb1b-69a8-488b-82a8-1d802935e43d) **With this PR**: **12s** ![04](https://github.com/user-attachments/assets/07dc5392-df08-4717-9975-1eac5c882c57) ![image](https://github.com/user-attachments/assets/f8e1b0cf-0546-46fb-9dc1-c790ff8f2e88) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-15 21:16:03 -05:00
yindo closed this issue 2026-02-15 21:16:03 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
documentation
enhancement
pull-request
Mirrored from GitHub Pull Request
 question
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-plugin-sdks#166
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?