Multiple host broker url format for ZooKeeper

[koorgoo]

There is a difference in how celery and kazoo (ZooKeeper driver for kombu used by celery) accept multiple hosts.

On the one side, celery suggests a semicolon separated URL list format.

CELERY_BROKER_URL = 'amqp://foo;amqp://bar'  # Django example.

On the other side, kazoo accepts a string with comma separated list of hosts.

KazooClient(hosts='localhost:2181,localhost:2182')

The issue with ZooKeeper fault tolerant configuration is that when configuring several hosts you must provide the same chroot (some kind of offset to store keys in ZooKeeper).

There are a few options how to provide the same chroot. Please, help me to choose one to implement:

Option 1: Kombu supports special format of URL for ZooKeeper close to kazoo API.

zookeeper://<host>[:<port>][,<host>[:<port>]][/<chroot>]

Option 2: Kombu demands same chroots in every sub-URL:

zookeeper://<host>[:<port>][/<chroot>][;zookeeper://<host>[:<port>][/<chroot>]]

[georgepsarakis]

This seems already implemented in the Zookeeper transport, see here. Or is this a problem with vhost specifically?

[koorgoo]

As of now broker URL like zookeeper://localhost:2181/chroot;zookeeper://localhost:2182/chroot leads to error in the line you reference:

...
# conn_str == 'localhost:2181,localhost:2181/chroot:2181,localhost:2181/chroot:2181'
conn = KazooClient(conn_str)

That's why the issue was opened. I want to agree on the one format and write some code to support it.

[koorgoo]

@georgepsarakis this is not the problem with vhost but with the some kind of mess after if conninfo.alt: in _open method.

[georgepsarakis]

I cannot see how this is caused by _open, so I am afraid that it may be caused in the Connection constructor. I would suggest Option 2, as it is closer to the current implementation. What do you think?

[koorgoo]

Option 2 would really be consistent in terms of the Celery project.
I will try to fix it. Will discuss technical details in comments on soon PR ;)

[auvipy]

was the issue fixed?