langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

ERROR] Worker (pid:**) was sent SIGKILL! Perhaps out of memory? #7537

New Issue
Closed
opened 2026-02-21 18:21:06 -05:00 by yindo · 2 comments
yindo commented 2026-02-21 18:21:06 -05:00
Owner
Copy Link
Copy Source

Originally created by @berg-wang on GitHub (Jan 6, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.12.1,0.13.2

Cloud or Self Hosted

Cloud

Steps to reproduce

I deployed dify-ai on the k8s cluster. When the project runs for a while, the dify-api pod gives an error:

2025-01-06 07:28:04 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:418)
[2025-01-06 07:28:05 +0000] [1] [ERROR] Worker (pid:418) was sent SIGKILL! Perhaps out of memory?
[2025-01-06 07:28:05 +0000] [511] [INFO] Booting worker with pid: 511
[2025-01-06 07:38:01 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:424)
[2025-01-06 07:38:02 +0000] [1] [ERROR] Worker (pid:424) was sent SIGKILL! Perhaps out of memory?
[2025-01-06 07:38:02 +0000] [513] [INFO] Booting worker with pid: 513
[2025-01-06 07:39:21 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:478)
[2025-01-06 07:39:22 +0000] [1] [ERROR] Worker (pid:478) was sent SIGKILL! Perhaps out of memory?
[2025-01-06 07:39:22 +0000] [515] [INFO] Booting worker with pid: 515


why this error occurred? and how to solve this problem?

### ✔️ Expected Behavior

the api server workers can run with no " SIGKILL!" errors?

### ❌ Actual Behavior

_No response_
Originally created by @berg-wang on GitHub (Jan 6, 2025). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version 0.12.1,0.13.2 ### Cloud or Self Hosted Cloud ### Steps to reproduce I deployed dify-ai on the k8s cluster. When the project runs for a while, the dify-api pod gives an error: ```shell 2025-01-06 07:28:04 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:418) [2025-01-06 07:28:05 +0000] [1] [ERROR] Worker (pid:418) was sent SIGKILL! Perhaps out of memory? [2025-01-06 07:28:05 +0000] [511] [INFO] Booting worker with pid: 511 [2025-01-06 07:38:01 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:424) [2025-01-06 07:38:02 +0000] [1] [ERROR] Worker (pid:424) was sent SIGKILL! Perhaps out of memory? [2025-01-06 07:38:02 +0000] [513] [INFO] Booting worker with pid: 513 [2025-01-06 07:39:21 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:478) [2025-01-06 07:39:22 +0000] [1] [ERROR] Worker (pid:478) was sent SIGKILL! Perhaps out of memory? [2025-01-06 07:39:22 +0000] [515] [INFO] Booting worker with pid: 515 why this error occurred? and how to solve this problem? ### ✔️ Expected Behavior the api server workers can run with no " SIGKILL!" errors? ### ❌ Actual Behavior _No response_
yindo added the 🌚 invalid label 2026-02-21 18:21:06 -05:00
yindo closed this issue 2026-02-21 18:21:06 -05:00
yindo commented 2026-02-21 18:21:07 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 6, 2025):

A similar issue has been discussed in the Dify repository, where the "Worker was sent SIGKILL" error is often related to API rate limiting and resource allocation issues [1]. Here are some steps you can take to resolve this problem:

  1. API Rate Limiting: Ensure that your application is not exceeding the API rate limits. Implement rate limiting in your application to stay within the limits of the embedding service you are using. If necessary, consider switching to a different service provider with higher rate limits.

  2. Resource Allocation: Even if your system has abundant resources, the worker process might be terminated due to configuration issues. Adjust the worker timeout and memory allocation settings to better suit your workload, especially if you are processing large files.

  3. Worker Configuration: Reduce the concurrent processing rate by adjusting the worker concurrency settings. You can modify the CELERY_WORKER_AMOUNT, CELERY_AUTO_SCALE, CELERY_MAX_WORKERS, and CELERY_MIN_WORKERS environment variables in the docker-compose.yaml file to control the number of concurrent tasks. This can help prevent the worker from being killed due to misconfiguration [1].

By addressing these areas, you should be able to mitigate the "SIGKILL" errors and improve the stability of your Dify deployment.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other

@dosubot[bot] commented on GitHub (Jan 6, 2025): <!-- Answer --> A similar issue has been discussed in the Dify repository, where the "Worker was sent SIGKILL" error is often related to API rate limiting and resource allocation issues <sup>[[1]](https://github.com/langgenius/dify/issues/11694)</sup>. Here are some steps you can take to resolve this problem: 1. **API Rate Limiting**: Ensure that your application is not exceeding the API rate limits. Implement rate limiting in your application to stay within the limits of the embedding service you are using. If necessary, consider switching to a different service provider with higher rate limits. 2. **Resource Allocation**: Even if your system has abundant resources, the worker process might be terminated due to configuration issues. Adjust the worker timeout and memory allocation settings to better suit your workload, especially if you are processing large files. 3. **Worker Configuration**: Reduce the concurrent processing rate by adjusting the worker concurrency settings. You can modify the `CELERY_WORKER_AMOUNT`, `CELERY_AUTO_SCALE`, `CELERY_MAX_WORKERS`, and `CELERY_MIN_WORKERS` environment variables in the `docker-compose.yaml` file to control the number of concurrent tasks. This can help prevent the worker from being killed due to misconfiguration <sup>[[1]](https://github.com/langgenius/dify/issues/11694)</sup>. By addressing these areas, you should be able to mitigate the "SIGKILL" errors and improve the stability of your Dify deployment. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/a71bdf23-871f-4ef3-b891-f579396a4f41?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/a71bdf23-871f-4ef3-b891-f579396a4f41?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/a71bdf23-871f-4ef3-b891-f579396a4f41?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/a71bdf23-871f-4ef3-b891-f579396a4f41?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/a71bdf23-871f-4ef3-b891-f579396a4f41?feedback_type=hallucination) | [Other](https://app.dosu.dev/response-feedback/a71bdf23-871f-4ef3-b891-f579396a4f41?feedback_type=other)</sup>
yindo commented 2026-02-21 18:21:07 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Jan 6, 2025):

  • You chose a wrong version which is not related to cloud version
  • The error logs are clear enough to resolve this issue.
@crazywoola commented on GitHub (Jan 6, 2025): - You chose a wrong version which is not related to cloud version - The error logs are clear enough to resolve this issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🌚 invalid
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#7537
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?