langgenius/dify-plugin-daemon
1
0
Fork 0
mirror of https://github.com/langgenius/dify-plugin-daemon.git synced 2026-07-21 17:25:23 -04:00
Code Issues 111 Packages Projects Releases 10 Wiki Activity

Zombie Processes Accumulate in dify-plugin-daemon Container (Parent Process Fails to Reap Children) #220

New Issue
Closed
opened 2026-02-16 00:20:33 -05:00 by yindo · 5 comments
yindo commented 2026-02-16 00:20:33 -05:00
Owner
Copy Link
Copy Source

Originally created by @yakexi007 on GitHub (Dec 17, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

dify-plugin-daemon:0.5.1-local

Cloud or Self Hosted

Cloud

Steps to reproduce

Hi Dify Team,
I'm encountering a critical issue with the langgenius/dify-plugin-daemon container (version 0.5.1-local) where zombie processes () accumulate rapidly during runtime. This poses a risk of PID exhaustion on the host and potential service instability over time.

Dify Plugin Daemon Version: 0.5.1-local (custom-built based on official 0.5.1 with minor tweaks)
Deployment Environment: Docker (Docker Compose) on Linux (Ubuntu 24.04 LTS)
Key mounts: ./volumes/plugin_daemon:/app/storage

Problem Symptom:
fter the container runs for ~1 hour, hundreds of zombie Python processes appear (via ps -ef | grep defunct on the host or docker exec ps -eo pid,ppid,stat)：

root 357972 0.0 0.0 0 0 ? Z Dec17 0:00 [python]
root 358902 0.0 0.0 0 0 ? Z Dec17 0:00 [python]
root 359557 0.0 0.0 0 0 ? Z Dec17 0:00 [python]
root 360580 0.0 0.0 0 0 ? Z Dec17 0:00 [python]

Hundreds more...

Image

All zombies have the same parent process (PID 9 inside the container, which maps to the plugin-daemon core process /app/main).
The parent process fails to reap these zombies even when sent the SIGCHLD signal (kill -CHLD ).

What I've Tried (Workarounds, Not Permanent Fixes):

  1. Sending SIGCHLD Signals: Both to the parent process (inside the container) and Docker's init (PID 1) — no effect on zombie reaping.
  2. Restarting the Container: Clears all zombies, but this is disruptive to production workloads.

Core Question:
Is there an official fix or recommended configuration to permanently resolve zombie process accumulation in dify-plugin-daemon? Specifically:

  1. Does the latest version (>0.5.1) address this issue (I tried pulling 0.6.2 but the tag does not exist for dify-plugin-daemon on Docker Hub)?
  2. Should the plugin-daemon core process (/app/main) be modified to properly handle SIGCHLD signals and reap child processes?
  3. Is there a recommended init system or Docker configuration (beyond init: true) to ensure full zombie reaping?
  4. Any guidance or fixes would be greatly appreciated — this issue is critical for long-running production deployments.

Thanks!

✔️ Expected Behavior

Expected Behavior

  • When the dify-plugin-daemon container runs (even under high load with frequent plugin executions), exited child processes (e.g., Python plugin processes) should be automatically and immediately reaped by either:
    • The parent process (/app/main) via proper SIGCHLD signal handling and waitpid() system calls, or
    • The container's PID 1 init system (Tini/Docker init) if the parent process fails to reap them.
  • Zombie processes (<defunct>) should not accumulate — the count of zombies inside the container should remain at 0 (or ≤5 transient zombies) at all times, even after days of continuous runtime.
  • No manual intervention (e.g., sending SIGCHLD signals, running reaping scripts, or restarting the container) should be required to prevent zombie process buildup.
  • The container should run stably long-term without risk of PID exhaustion on the host due to unreaped zombies.

❌ Actual Behavior

No response

Originally created by @yakexi007 on GitHub (Dec 17, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version dify-plugin-daemon:0.5.1-local ### Cloud or Self Hosted Cloud ### Steps to reproduce Hi Dify Team, I'm encountering a critical issue with the langgenius/dify-plugin-daemon container (version 0.5.1-local) where zombie processes (<defunct>) accumulate rapidly during runtime. This poses a risk of PID exhaustion on the host and potential service instability over time. Dify Plugin Daemon Version: 0.5.1-local (custom-built based on official 0.5.1 with minor tweaks) Deployment Environment: Docker (Docker Compose) on Linux (Ubuntu 24.04 LTS) Key mounts: ./volumes/plugin_daemon:/app/storage Problem Symptom: fter the container runs for ~1 hour, hundreds of zombie Python processes appear (via ps -ef | grep defunct on the host or docker exec <container> ps -eo pid,ppid,stat)： root 357972 0.0 0.0 0 0 ? Z Dec17 0:00 [python] <defunct> root 358902 0.0 0.0 0 0 ? Z Dec17 0:00 [python] <defunct> root 359557 0.0 0.0 0 0 ? Z Dec17 0:00 [python] <defunct> root 360580 0.0 0.0 0 0 ? Z Dec17 0:00 [python] <defunct> # Hundreds more... <img width="1341" height="458" alt="Image" src="https://github.com/user-attachments/assets/fce737a5-a681-44a7-a8cb-aba212bca85f" /> All zombies have the same parent process (PID 9 inside the container, which maps to the plugin-daemon core process /app/main). The parent process fails to reap these zombies even when sent the SIGCHLD signal (kill -CHLD <parent-pid>). What I've Tried (Workarounds, Not Permanent Fixes): 1. Sending SIGCHLD Signals: Both to the parent process (inside the container) and Docker's init (PID 1) — no effect on zombie reaping. 2. Restarting the Container: Clears all zombies, but this is disruptive to production workloads. **Core Question:** Is there an official fix or recommended configuration to permanently resolve zombie process accumulation in dify-plugin-daemon? Specifically: 1. Does the latest version (>0.5.1) address this issue (I tried pulling 0.6.2 but the tag does not exist for dify-plugin-daemon on Docker Hub)? 2. Should the plugin-daemon core process (/app/main) be modified to properly handle SIGCHLD signals and reap child processes? 3. Is there a recommended init system or Docker configuration (beyond init: true) to ensure full zombie reaping? 4. Any guidance or fixes would be greatly appreciated — this issue is critical for long-running production deployments. Thanks! ### ✔️ Expected Behavior ### Expected Behavior * When the `dify-plugin-daemon` container runs (even under high load with frequent plugin executions), **exited child processes (e.g., Python plugin processes)** should be automatically and immediately reaped by either: - The parent process (`/app/main`) via proper `SIGCHLD` signal handling and `waitpid()` system calls, or - The container's PID 1 init system (Tini/Docker init) if the parent process fails to reap them. * Zombie processes (`<defunct>`) should **not accumulate** — the count of zombies inside the container should remain at 0 (or ≤5 transient zombies) at all times, even after days of continuous runtime. * No manual intervention (e.g., sending `SIGCHLD` signals, running reaping scripts, or restarting the container) should be required to prevent zombie process buildup. * The container should run stably long-term without risk of PID exhaustion on the host due to unreaped zombies. ### ❌ Actual Behavior _No response_
yindo added the plugin-daemonbug labels 2026-02-16 00:20:33 -05:00
yindo closed this issue 2026-02-16 00:20:33 -05:00
yindo commented 2026-02-16 01:15:01 -05:00
Author
Owner
Copy Link
Copy Source

@Yeuoly commented on GitHub (Dec 18, 2025):

Hi, thank you for the feedback, I've located the root cause: Wait was not called after the SIGKILL signal to the subprocess in StartStdout routine, a routine which takes over the lifetime management of a plugin subprocess.

Also, I added some comments there to make sure the logics and architecture design is not confused and clear to developers.

Would you mind take a look at the patch? #544

@Yeuoly commented on GitHub (Dec 18, 2025): Hi, thank you for the feedback, I've located the root cause: `Wait` was not called after the SIGKILL signal to the subprocess in `StartStdout` routine, a routine which takes over the lifetime management of a plugin subprocess. Also, I added some comments there to make sure the logics and architecture design is not confused and clear to developers. Would you mind take a look at the patch? #544
yindo commented 2026-02-16 01:15:01 -05:00
Author
Owner
Copy Link
Copy Source

@yakexi007 commented on GitHub (Dec 18, 2025):

Hi @Yeuoly ,

Thank you so much for quickly locating and fixing the root cause of the zombie process issue!

I will:

  1. I'll test it as soon as the PR is merged into the official release
  2. Feedback the test results here promptly.

Looking forward to seeing this merged!

Best regards,
yakexi007

@yakexi007 commented on GitHub (Dec 18, 2025): Hi @Yeuoly , Thank you so much for quickly locating and fixing the root cause of the zombie process issue! I will: 1. I'll test it as soon as the PR is merged into the official release 2. Feedback the test results here promptly. Looking forward to seeing this merged! Best regards, yakexi007
yindo commented 2026-02-16 01:15:01 -05:00
Author
Owner
Copy Link
Copy Source

@Yeuoly commented on GitHub (Dec 19, 2025):

@yakexi007 Would you like to try langgenius/dify-plugin-daemon:66fa57e415566678ce51b45abefdea989d0ba3e7-local ?

@Yeuoly commented on GitHub (Dec 19, 2025): @yakexi007 Would you like to try langgenius/dify-plugin-daemon:66fa57e415566678ce51b45abefdea989d0ba3e7-local ?
yindo commented 2026-02-16 01:15:01 -05:00
Author
Owner
Copy Link
Copy Source

@yakexi007 commented on GitHub (Dec 19, 2025):

Hi @Yeuoly ,

Thanks a lot for the test image and fixing the issue so quickly!

I’d like to wait for the official release version ((e.g., v0.5.2/v0.6.0 or similar tagged builds).) to test this fix, as we use release-tagged images in production for consistency. I’ll test it right after the official release and share the results here immediately.

Appreciate your hard work!

@yakexi007 commented on GitHub (Dec 19, 2025): Hi @Yeuoly , Thanks a lot for the test image and fixing the issue so quickly! I’d like to wait for the official release version ((e.g., `v0.5.2`/`v0.6.0` or similar tagged builds).) to test this fix, as we use release-tagged images in production for consistency. I’ll test it right after the official release and share the results here immediately. Appreciate your hard work!
yindo commented 2026-02-16 01:15:01 -05:00
Author
Owner
Copy Link
Copy Source

@yakexi007 commented on GitHub (Dec 23, 2025):

Hi @Yeuoly ,
Thank you so much for the quick fix! I can confirm that the issue has been resolved in the latest v0.5.2 release. We really appreciate your efficient work and support.
Best regards.

@yakexi007 commented on GitHub (Dec 23, 2025): Hi @Yeuoly , Thank you so much for the quick fix! I can confirm that the issue has been resolved in the latest v0.5.2 release. We really appreciate your efficient work and support. Best regards.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Windows
bug
cmd
documentation
enhancement
go
good first issue
plugin-daemon
pull-request
Mirrored from GitHub Pull Request
 question
No Label bug plugin-daemon
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-plugin-daemon#220
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?