KafkaClient.send() always fails the first time, all subsequent send() calls succeed

[jeffwidman]

The first time I call KafkaClient.send() it always immediately fails, even before I call poll(). Any subsequent calls will hang until I call poll(), then they will succeed.

This always happens on the first send() call to a new broker. So if send()/poll() are working as expected for node 1, and then I try sending to node 2, I will see the same behavior where the first call fails, then subsequent calls succeed.

Is this expected behavior or a bug?

Possibly related: #981 / #1278 / #1279

Example code:

import kafka.protocol as kp
from kafka.client_async import KafkaClient


kc = KafkaClient(bootstrap_servers=['kafka01.stg.local:9092'])

# first call to kc.send() always fails
gcr1 = kc.send(1, kp.commit.GroupCoordinatorRequest[0]('valid_group_name'))

gcr1.is_done  # always True even though I never called poll()
gcr1.failed()  # always True
gcr1.value  # `None`... the docs say this shold return a Response Struct or error?


# try it a second time:
gcr1 = kc.send(1, kp.commit.GroupCoordinatorRequest[0]('valid_group_name'))
gcr1.is_done  # always False until I call poll()

# will block until it gets a response... generally returns almost immediately
kc.poll(future=gcr1)

gcr1.value # always returns a Response struct

[dpkp]

This sounds correct to me. Until the node is connected / ready send will fail with NodeNotReadyError. You can check the exception on failure via gcr1.exception (not gcr1.value). This is because sends are still synchronous. See #981 re switching to fully async sends. Nonetheless, once the node is connected then the request can be sent and then you'll get a response future that requires calls to poll to pump the event loop for further processing.