langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 03:55:25 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Celery with a Redis cluster, an exception is raised: ResponseError("'BRPOP' command keys must be in the same slot") #3416

New Issue
Closed
opened 2026-02-21 17:57:33 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 17:57:33 -05:00
Owner
Copy Link
Copy Source

Originally created by @vicwang3 on GitHub (May 17, 2024).

Originally assigned to: @takatost on GitHub.

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.8

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

  1. set CELERY_BROKER_URL=redis://:password@host:port/db
  2. celery -A app.celery worker -P gevent -c 1 -Q dataset,generation,mail --loglevel INFO

✔️ Expected Behavior

-------------- celery@TAKATOST.lan v5.2.7 (dawn-chorus)
--- ***** -----
-- ******* ---- macOS-10.16-x86_64-i386-64bit 2023-07-31 12:58:08

  • *** --- * ---
  • ** ---------- [config]
  • ** ---------- .> app: app:0x7fb568572a10
  • ** ---------- .> transport: redis://:**@localhost:6379/1
  • ** ---------- .> results: postgresql://postgres:**@localhost:5432/dify
  • *** --- * --- .> concurrency: 1 (gevent)
    -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
    --- ***** -----
    -------------- [queues]
    .> dataset exchange=dataset(direct) key=dataset
    .> generation exchange=generation(direct) key=generation
    .> mail exchange=mail(direct) key=mail

[tasks]
. tasks.add_document_to_index_task.add_document_to_index_task
. tasks.clean_dataset_task.clean_dataset_task
. tasks.clean_document_task.clean_document_task
. tasks.clean_notion_document_task.clean_notion_document_task
. tasks.create_segment_to_index_task.create_segment_to_index_task
. tasks.deal_dataset_vector_index_task.deal_dataset_vector_index_task
. tasks.document_indexing_sync_task.document_indexing_sync_task
. tasks.document_indexing_task.document_indexing_task
. tasks.document_indexing_update_task.document_indexing_update_task
. tasks.enable_segment_to_index_task.enable_segment_to_index_task
. tasks.generate_conversation_summary_task.generate_conversation_summary_task
. tasks.mail_invite_member_task.send_invite_member_mail_task
. tasks.remove_document_from_index_task.remove_document_from_index_task
. tasks.remove_segment_from_index_task.remove_segment_from_index_task
. tasks.update_segment_index_task.update_segment_index_task
. tasks.update_segment_keyword_index_task.update_segment_keyword_index_task

[2023-07-31 12:58:08,831: INFO/MainProcess] Connected to redis://:@localhost:6379/1
[2023-07-31 12:58:08,840: INFO/MainProcess] mingle: searching for neighbors
[2023-07-31 12:58:09,873: INFO/MainProcess] mingle: all alone
[2023-07-31 12:58:09,886: INFO/MainProcess] pidbox: Connected to redis://:@localhost:6379/1.
[2023-07-31 12:58:09,890: INFO/MainProcess] celery@TAKATOST.lan ready.

❌ Actual Behavior

[2024-05-17 06:19:34,129: INFO/MainProcess] Connected to redis://:**@host:6379/1
[2024-05-17 06:19:34,417: INFO/MainProcess] mingle: searching for neighbors
[2024-05-17 06:19:35,437: CRITICAL/MainProcess] Unrecoverable error: ResponseError("'BRPOP' command keys must in same slot")
Traceback (most recent call last):
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/worker/worker.py", line 202, in start
self.blueprint.start(self)
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/bootsteps.py", line 116, in start
step.start(parent)
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/bootsteps.py", line 365, in start
return self.obj.start()
^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/worker/consumer/consumer.py", line 340, in start
blueprint.start(self)
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/bootsteps.py", line 116, in start
step.start(parent)
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/worker/consumer/mingle.py", line 37, in start
self.sync(c)
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/worker/consumer/mingle.py", line 41, in sync
replies = self.send_hello(c)
^^^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/worker/consumer/mingle.py", line 54, in send_hello
replies = inspect.hello(c.hostname, our_revoked._data) or {}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/app/control.py", line 389, in hello
return self._request('hello', from_node=from_node, revoked=revoked)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/app/control.py", line 106, in _request
return self._prepare(self.app.control.broadcast(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/app/control.py", line 776, in broadcast
return self.mailbox(conn)._broadcast(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/pidbox.py", line 346, in _broadcast
return self._collect(reply_ticket, limit=limit,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/pidbox.py", line 388, in _collect
self.connection.drain_events(timeout=timeout)
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/connection.py", line 341, in drain_events
return self.transport.drain_events(self.connection, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/transport/virtual/base.py", line 997, in drain_events
get(self._deliver, timeout=timeout)
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/transport/redis.py", line 591, in get
ret = self.handle_event(fileno, event)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/transport/redis.py", line 573, in handle_event
return self.on_readable(fileno), self
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/transport/redis.py", line 569, in on_readable
chan.handlerstype
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/transport/redis.py", line 962, in _brpop_read
dest__item = self.client.parse_response(self.client.connection,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/transport/redis.py", line 247, in parse_response
ret = super().parse_response(connection, command_name, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/redis/client.py", line 562, in parse_response
response = connection.read_response()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/redis/connection.py", line 536, in read_response
raise response
redis.exceptions.ResponseError: 'BRPOP' command keys must in same slot

Originally created by @vicwang3 on GitHub (May 17, 2024). Originally assigned to: @takatost on GitHub. ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version 0.6.8 ### Cloud or Self Hosted Self Hosted (Source) ### Steps to reproduce 1. set CELERY_BROKER_URL=redis://:password@host:port/db 2. celery -A app.celery worker -P gevent -c 1 -Q dataset,generation,mail --loglevel INFO ### ✔️ Expected Behavior -------------- celery@TAKATOST.lan v5.2.7 (dawn-chorus) --- ***** ----- -- ******* ---- macOS-10.16-x86_64-i386-64bit 2023-07-31 12:58:08 - *** --- * --- - ** ---------- [config] - ** ---------- .> app: app:0x7fb568572a10 - ** ---------- .> transport: redis://:**@localhost:6379/1 - ** ---------- .> results: postgresql://postgres:**@localhost:5432/dify - *** --- * --- .> concurrency: 1 (gevent) -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker) --- ***** ----- -------------- [queues] .> dataset exchange=dataset(direct) key=dataset .> generation exchange=generation(direct) key=generation .> mail exchange=mail(direct) key=mail [tasks] . tasks.add_document_to_index_task.add_document_to_index_task . tasks.clean_dataset_task.clean_dataset_task . tasks.clean_document_task.clean_document_task . tasks.clean_notion_document_task.clean_notion_document_task . tasks.create_segment_to_index_task.create_segment_to_index_task . tasks.deal_dataset_vector_index_task.deal_dataset_vector_index_task . tasks.document_indexing_sync_task.document_indexing_sync_task . tasks.document_indexing_task.document_indexing_task . tasks.document_indexing_update_task.document_indexing_update_task . tasks.enable_segment_to_index_task.enable_segment_to_index_task . tasks.generate_conversation_summary_task.generate_conversation_summary_task . tasks.mail_invite_member_task.send_invite_member_mail_task . tasks.remove_document_from_index_task.remove_document_from_index_task . tasks.remove_segment_from_index_task.remove_segment_from_index_task . tasks.update_segment_index_task.update_segment_index_task . tasks.update_segment_keyword_index_task.update_segment_keyword_index_task [2023-07-31 12:58:08,831: INFO/MainProcess] Connected to redis://:**@localhost:6379/1 [2023-07-31 12:58:08,840: INFO/MainProcess] mingle: searching for neighbors [2023-07-31 12:58:09,873: INFO/MainProcess] mingle: all alone [2023-07-31 12:58:09,886: INFO/MainProcess] pidbox: Connected to redis://:**@localhost:6379/1. [2023-07-31 12:58:09,890: INFO/MainProcess] celery@TAKATOST.lan ready. ### ❌ Actual Behavior [2024-05-17 06:19:34,129: INFO/MainProcess] Connected to redis://:**@host:6379/1 [2024-05-17 06:19:34,417: INFO/MainProcess] mingle: searching for neighbors [2024-05-17 06:19:35,437: CRITICAL/MainProcess] Unrecoverable error: ResponseError("'BRPOP' command keys must in same slot") Traceback (most recent call last): File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/worker/worker.py", line 202, in start self.blueprint.start(self) File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/bootsteps.py", line 116, in start step.start(parent) File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/bootsteps.py", line 365, in start return self.obj.start() ^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/worker/consumer/consumer.py", line 340, in start blueprint.start(self) File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/bootsteps.py", line 116, in start step.start(parent) File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/worker/consumer/mingle.py", line 37, in start self.sync(c) File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/worker/consumer/mingle.py", line 41, in sync replies = self.send_hello(c) ^^^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/worker/consumer/mingle.py", line 54, in send_hello replies = inspect.hello(c.hostname, our_revoked._data) or {} ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/app/control.py", line 389, in hello return self._request('hello', from_node=from_node, revoked=revoked) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/app/control.py", line 106, in _request return self._prepare(self.app.control.broadcast( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/celery/app/control.py", line 776, in broadcast return self.mailbox(conn)._broadcast( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/pidbox.py", line 346, in _broadcast return self._collect(reply_ticket, limit=limit, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/pidbox.py", line 388, in _collect self.connection.drain_events(timeout=timeout) File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/connection.py", line 341, in drain_events return self.transport.drain_events(self.connection, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/transport/virtual/base.py", line 997, in drain_events get(self._deliver, timeout=timeout) File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/transport/redis.py", line 591, in get ret = self.handle_event(fileno, event) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/transport/redis.py", line 573, in handle_event return self.on_readable(fileno), self ^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/transport/redis.py", line 569, in on_readable chan.handlers[type]() File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/transport/redis.py", line 962, in _brpop_read dest__item = self.client.parse_response(self.client.connection, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/kombu/transport/redis.py", line 247, in parse_response ret = super().parse_response(connection, command_name, **options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/redis/client.py", line 562, in parse_response response = connection.read_response() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wangvic/miniconda3/lib/python3.11/site-packages/redis/connection.py", line 536, in read_response raise response redis.exceptions.ResponseError: 'BRPOP' command keys must in same slot
yindo added the 🐞 bug label 2026-02-21 17:57:33 -05:00
yindo closed this issue 2026-02-21 17:57:33 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#3416
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?