Plugin installation on multi-instances

[plutonji]

Self Checks

To make sure we get to you in time, please check the following :)

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• "Please do not modify this template :) and fill in all the required fields."

Versions

1. dify-plugin-daemon 0.1.3
2. dify-api Version 1.5.1

Describe the bug
I am running two instances of daemon_plugin in the system to achieve high availability (HA), with HAProxy acting as the load balancer at the traffic entry point.

However, a problem has arisen: when I install a plugin through the Dify interface, the plugin is only installed on one of the instances, and the other instance remains without it.

As a result, errors occur when requests are distributed by HAProxy to the instance where the plugin is not installed.

To Reproduce
Steps to reproduce the behavior:

1. Startup two instances of plugin_daemon.
2. Install Plugin via dify web.
3. Test it.

Expected behavior
It should work.

Screenshots

Additional context
Add any other context about the problem here.

[homejim]

Dify’s plugin daemon leverages Redis for state management and includes custom routing rules. When all instances share the same Redis instance, the system ensures high availability by automatically routing plugin-related requests to any instance that has the required plugin installed. This prevents errors from occurring due to missing plugins on certain nodes.

In our service, which runs multiple plugin daemon instances, we’ve observed that even during deployment — when some instances may not have completed plugin installation — the system continues to operate smoothly without raising errors.

[xiaomeixw]

This is an issue with the official design. You need to modify the code yourself to use the forced installation mode and skip the database detection mechanism.

[xiaomeixw]

Dify’s plugin daemon leverages Redis for state management and includes custom routing rules. When all instances share the same Redis instance, the system ensures high availability by automatically routing plugin-related requests to any instance that has the required plugin installed. This prevents errors from occurring due to missing plugins on certain nodes.

In our service, which runs multiple plugin daemon instances, we’ve observed that even during deployment — when some instances may not have completed plugin installation — the system continues to operate smoothly without raising errors.

Your response completely failed to understand the question. The user wants to implement load balancing. There are issues with your existing cluster mechanism. The user now wants to create a plugin distribution and processing mechanism. You only allow the installation of a single plugin. How is the user supposed to proceed?

[plutonji]

This is an issue with the official design. You need to modify the code yourself to use the forced installation mode and skip the database detection mechanism.

Thanks

[LeoLiuYan]

This is an issue with the official design. You need to modify the code yourself to use the forced installation mode and skip the database detection mechanism.

Hi, when using the forced installation mode, will this cause traffic to be routed to nodes without the plugin installed?
BTW, how to use the forced installation mode?

@xiaomeixw @plutonji

[xiaomeixw]

This is an issue with the official design. You need to modify the code yourself to use the forced installation mode and skip the database detection mechanism.

Hi, when using the forced installation mode, will this cause traffic to be routed to nodes without the plugin installed? BTW, how to use the forced installation mode?

@xiaomeixw @plutonji

@LeoLiuYan

Forced installation requires adjustments to the plugin's install-related api. Since multiple plugin servers in a cluster share the same database state, the official code will block the installation of plugins on other plugin servers in the cluster. Therefore, it is necessary to skip these database checks and implement forced installation.