langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #27532] Fix: Prevent binary content from being stored in process_data for HTTP nodes #31793

New Issue
Closed
opened 2026-02-21 20:50:09 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:50:09 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/27532

State: closed
Merged: Yes

…P nodes

Fix: Prevent binary content from being stored in process_data for HTTP nodes

Summary

This PR addresses a critical issue where HTTP nodes would attempt to decode binary request content (such as audio, video, or large files) as UTF-8 text when generating logs for process_data. This caused massive string expansion (e.g., 100MB binary file becoming 1GB string), leading to database performance degradation, memory exhaustion, and workflow failures.

fix #27533

Problem

The to_log() method in core/workflow/nodes/http_request/executor.py was directly decoding binary content using content.decode("utf-8", errors="replace"), which:

  • Converts binary data to large strings with replacement characters
  • Causes exponential size expansion (100MB binary → 1GB+ string)
  • Results in database bloat and workflow execution failures
  • Stores sensitive binary content in plain text logs

Solution

Replace binary content with size placeholders instead of attempting to decode:

  • For form-data file uploads: <file_content_binary, size={len(content)} bytes>
  • For binary request bodies: <binary_content, size={len(content)} bytes>

This maintains debugging information while preventing binary data from being stored in the database.

Files Changed

  • core/workflow/nodes/http_request/executor.py: Modified to_log() method to safely handle binary content

Testing

  • Verified that HTTP nodes with binary request bodies no longer cause database issues
  • Confirmed that log entries still provide useful debugging information (content size)
  • Tested with various file types and sizes to ensure no regression

Breaking Changes

None. This change only affects the content stored in process_data logs and does not impact actual HTTP request functionality.

Important

  1. Make sure you have read our contribution guidelines
  2. Ensure there is an associated issue and you have been assigned to it
  3. Use the correct syntax to link this PR: Fixes #<issue number>.

Summary

Screenshots

Before After
... ...

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods
**Original Pull Request:** https://github.com/langgenius/dify/pull/27532 **State:** closed **Merged:** Yes --- …P nodes ## Fix: Prevent binary content from being stored in process_data for HTTP nodes ### Summary This PR addresses a critical issue where HTTP nodes would attempt to decode binary request content (such as audio, video, or large files) as UTF-8 text when generating logs for `process_data`. This caused massive string expansion (e.g., 100MB binary file becoming 1GB string), leading to database performance degradation, memory exhaustion, and workflow failures. fix #27533 ### Problem The `to_log()` method in `core/workflow/nodes/http_request/executor.py` was directly decoding binary content using `content.decode("utf-8", errors="replace")`, which: - Converts binary data to large strings with replacement characters - Causes exponential size expansion (100MB binary → 1GB+ string) - Results in database bloat and workflow execution failures - Stores sensitive binary content in plain text logs ### Solution Replace binary content with size placeholders instead of attempting to decode: - For form-data file uploads: `<file_content_binary, size={len(content)} bytes>` - For binary request bodies: `<binary_content, size={len(content)} bytes>` This maintains debugging information while preventing binary data from being stored in the database. ### Files Changed - `core/workflow/nodes/http_request/executor.py`: Modified `to_log()` method to safely handle binary content ### Testing - Verified that HTTP nodes with binary request bodies no longer cause database issues - Confirmed that log entries still provide useful debugging information (content size) - Tested with various file types and sizes to ensure no regression ### Breaking Changes None. This change only affects the content stored in process_data logs and does not impact actual HTTP request functionality. > [!IMPORTANT] > > 1. Make sure you have read our [contribution guidelines](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) > 1. Ensure there is an associated issue and you have been assigned to it > 1. Use the correct syntax to link this PR: `Fixes #<issue number>`. ## Summary <!-- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. --> ## Screenshots | Before | After | |--------|-------| | ... | ... | ## Checklist - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs) - [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!) - [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change. - [x] I've updated the documentation accordingly. - [x] I ran `dev/reformat`(backend) and `cd web && npx lint-staged`(frontend) to appease the lint gods
yindo added the pull-request label 2026-02-21 20:50:09 -05:00
yindo closed this issue 2026-02-21 20:50:09 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#31793
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?