Kafka OAuth Support for SaslOauthbearerConfig and SetOAuthBearerTokenRefreshHandler

[demianzhang]

hi @jainharsh98 ,

azure-functions-kafka-extension is an awesome repo, will it support config parameter SaslOauthbearerConfig in the future? As for authentication demand, Will it support to call SetOAuthBearerTokenRefreshHandler(Callback) When creating ConsumerBuilder<TKey, TValue>(config) as optional function? Thanks.

[yukscck]

upvoting - this will be useful too in my scenario of connecting to Event Hubs for Kafka using user-assigned managed identity.

[raorugan]

May I know the challenges with the default values

SASL/OAUTHBEARER configuration. The format is implementation-dependent and must be parsed accordingly. The default unsecured token implementation (see https://tools.ietf.org/html/rfc7515#appendix-A.5) recognizes space-separated name=value pairs with valid names including principalClaimName, principal, scopeClaimName, scope, and lifeSeconds. The default value for principalClaimName is "sub", the default value for scopeClaimName is "scope", and the default value for lifeSeconds is 3600. The scope value is CSV format with the default value being no/empty scope. For example: principalClaimName=azp principal=admin scopeClaimName=roles scope=role1,role2 lifeSeconds=600. In addition, SASL extensions can be communicated to the broker via extension_NAME=value. For example: principal=admin extension_traceId=123
Type: string

Or by using API calls
SASL/OAUTHBEARER token refresh callback (set with rd_kafka_conf_set_oauthbearer_token_refresh_cb(), triggered by rd_kafka_poll(), et.al. This callback will be triggered when it is time to refresh the client's OAUTHBEARER token. Also see rd_kafka_conf_enable_sasl_queue().
Type: see dedicated API

[yukscck]

@raorugan sorry, I'm not sure if the question was directed to @demianzhang or me. I will just provide input from my perspective.

Currently, I am in the midst of moving my Kafka clients' connections from Kafka servers to Event Hubs. One of the requirements is to be able to connect to my Event Hubs using Kafka protocol with managed identity, without needing to manage client secrets.

I could smoothly migrate my non-FunctionApp applications and use managed identity, thanks to the ability to intercept the Kafka consumers' OAuth Token generation through the rd_kafka_conf_set_oauthbearer_token_refresh_cb, since those applications call Kafka client libraries directly.

However, for applications built on Azure Functions with Kafka extensions, I am not sure how I can achieve the same results.
You mentioned:

Or by using API calls
SASL/OAUTHBEARER token refresh callback (set with rd_kafka_conf_set_oauthbearer_token_refresh_cb(), triggered by rd_kafka_poll(), et.al. This callback will be triggered when it is time to refresh the client's OAUTHBEARER token. Also see rd_kafka_conf_enable_sasl_queue().
Type: see dedicated API

That seems to be the same approach used by Kafka client libraries under the hood.
Can you provide guidance on how we could do that in Azure Functions?

Thanks.