mirror of
https://github.com/langgenius/dify-plugin-daemon.git
synced 2026-07-21 17:25:23 -04:00
Plugin installation on multi-instances #156
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @plutonji on GitHub (Jul 7, 2025).
Self Checks
To make sure we get to you in time, please check the following :)
Versions
Describe the bug
I am running two instances of
daemon_pluginin the system to achieve high availability (HA), with HAProxy acting as the load balancer at the traffic entry point.However, a problem has arisen: when I install a plugin through the Dify interface, the plugin is only installed on one of the instances, and the other instance remains without it.
As a result, errors occur when requests are distributed by HAProxy to the instance where the plugin is not installed.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
It should work.
Screenshots
Additional context
Add any other context about the problem here.
@homejim commented on GitHub (Jul 8, 2025):
Dify’s plugin daemon leverages Redis for state management and includes custom routing rules. When all instances share the same Redis instance, the system ensures high availability by automatically routing plugin-related requests to any instance that has the required plugin installed. This prevents errors from occurring due to missing plugins on certain nodes.
In our service, which runs multiple plugin daemon instances, we’ve observed that even during deployment — when some instances may not have completed plugin installation — the system continues to operate smoothly without raising errors.
@xiaomeixw commented on GitHub (Aug 12, 2025):
This is an issue with the official design. You need to modify the code yourself to use the forced installation mode and skip the database detection mechanism.
@xiaomeixw commented on GitHub (Aug 12, 2025):
Your response completely failed to understand the question. The user wants to implement load balancing. There are issues with your existing cluster mechanism. The user now wants to create a plugin distribution and processing mechanism. You only allow the installation of a single plugin. How is the user supposed to proceed?
@plutonji commented on GitHub (Aug 12, 2025):
Thanks
@LeoLiuYan commented on GitHub (Aug 26, 2025):
Hi, when using the forced installation mode, will this cause traffic to be routed to nodes without the plugin installed?
BTW, how to use the forced installation mode?
@xiaomeixw @plutonji
@xiaomeixw commented on GitHub (Aug 26, 2025):
@LeoLiuYan
Forced installation requires adjustments to the plugin's install-related api. Since multiple plugin servers in a cluster share the same database state, the official code will block the installation of plugins on other plugin servers in the cluster. Therefore, it is necessary to skip these database checks and implement forced installation.
@fatelei commented on GitHub (Jan 10, 2026):
Installing sync dependencies relies on the object-storage adapter layer (installedBucket/packageBucket), which can point to S3/MinIO/Azure Blob/GCS or any other remote store.
Multi-machine setups work perfectly as long as every node is configured to use the same bucket/prefix.
Each node’s local working directory (PluginWorkingPath) is used to unpack and build the runtime environment (.venv); it is strongly recommended that every node use its own local disk instead of a shared one. This prevents concurrency collisions; if you must share, add distributed locking.
Multi-node HA checklist
Both instances use the same remote object store:
– PLUGIN_STORAGE_TYPE set to s3/minio/azure/gcs (never “local”)
– Identical bucket, endpoint and credentials
– PLUGIN_INSTALLED_PATH and PLUGIN_PACKAGE_CACHE_PATH identical (same prefix)
PLATFORM=local; WatchDog scans the installedBucket once at start-up and then every 30 s on every node.
PluginWorkingPath must be node-local; if shared, enable distributed locking to serialize environment initialization for the same plugin.
Minimal example (two instances sharing one MinIO/S3 bucket) – environment variables must be identical on both sides.
@plutonji commented on GitHub (Jan 12, 2026):
@fatelei Thank you for your reply. Although our offline server intranet does not have an object storage service, we do have a NAS server that can provide NFS Server v3. Therefore, I tried setting PLUGIN_STORAGE_LOCAL_ROOT to the shared NFS directory, while keeping PLUGIN_WORKING_PATH as the local directory /app/storage/cwd. Sure enough, the error no longer occurred!
Thank you again for the information you provided!