Add option to allow `ProduceSync` to be cancellable when enable idempotent producer

[guitarpawat]

Description

When I enable idempotent and call ProduceSync, it cannot be canceled by using context or any timeout options. This is problematic if there are network connection issues between the client and Kafka broker because it will keep retrying indefinitely until the connection is restored. I understand the risk of data loss if the record could be cancelled. I want to use message deduplication from idempotent to achieve 'at most once' producer. Currently, there is no option to opt for this behavior.

This is similar to #930

Example Use Case

The API service for external service to submit time-sensitive tasks to Kafka and processes them only once, asynchronously, by internal service. The API service will try to produce the message and keep retrying to produce it in case of error until the task timeout.

External Service → API Service → Kafka → Internal Service

There will be three possible outcomes from this API service:

• Success: The task was sent to internal service and will be processed shortly.
• Fail: There is a problem within the internal service or submitted task, and the task will not be processed.
• Timeout: The task may or may not be processed. The external service could call another API to query and confirm the status of the task later.

[twmb]

You're effectively asking for the feature proposed in the other issue, AllowOutstandingProduceCancellations?

[guitarpawat]

You're effectively asking for the feature proposed in the other issue, AllowOutstandingProduceCancellations?

Yes, I need the option to be able to cancel the producer.