-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
area-transportMessage transport/broker implementationsMessage transport/broker implementationscloud-azureMicrosoft AzureMicrosoft Azurecomplexity-mediumComplexity: MediumComplexity: MediumenhancementNew feature or requestNew feature or requestnew-packageRequires creating a new NuGet packageRequires creating a new NuGet packagephase-2-functionalityPhase 2: Functionality - new featuresPhase 2: Functionality - new featurespriority-mediumPriority: Medium (⭐⭐⭐)Priority: Medium (⭐⭐⭐)
Milestone
Description
Summary
Implement Encina.AzureEventHubs transport for high-throughput event streaming.
Motivation
Azure Event Hubs is essential for:
- Massive Scale: Millions of events per second
- Azure Native: First-class Azure integration
- Kafka Compatible: Kafka protocol support
- Event Streaming: Real-time analytics pipelines
Use Cases
- Telemetry ingestion
- Real-time analytics
- Event streaming pipelines
- IoT data collection
Proposed Solution
public class AzureEventHubsTransport : IMessageTransport
{
private readonly EventHubProducerClient _producer;
private readonly EventProcessorClient _processor;
public async Task PublishAsync<T>(T message, string? partitionKey = null, CancellationToken ct = default)
{
var eventData = new EventData(JsonSerializer.SerializeToUtf8Bytes(message));
var options = partitionKey != null
? new SendEventOptions { PartitionKey = partitionKey }
: null;
await _producer.SendAsync(new[] { eventData }, options, ct);
}
}Features
- Partitioned publishing (partition key)
- Consumer groups
- Checkpointing with blob storage
- Batch publishing
- Schema Registry integration
Alternatives Considered
-
Use Azure Service Bus instead: Service Bus is message-queue focused while Event Hubs is stream-focused. Event Hubs is better for high-throughput analytics scenarios.
-
Use Kafka on Azure (Event Hubs with Kafka protocol): Event Hubs supports Kafka protocol, so this transport could optionally expose that interface for Kafka client compatibility.
-
Use Azure Event Grid: Event Grid is for event routing while Event Hubs is for event streaming. They complement each other for different patterns.
Affected Packages
- Encina (core)
- Encina.EntityFrameworkCore
- Encina.Dapper.*
- Encina.ADO.*
- Encina.AspNetCore
- Encina.OpenTelemetry
- Encina.Caching.*
- Other: Encina.AzureEventHubs (new package)
Additional Context
Acceptance Criteria
-
Encina.AzureEventHubspackage - Producer implementation
- Consumer implementation with checkpointing
- Partition key support
- Consumer group support
- Batch operations
- Azure Schema Registry integration (optional)
- ServiceCollection extensions
- Unit tests
- Integration tests
- Documentation
Package Dependencies
Azure.Messaging.EventHubsAzure.Messaging.EventHubs.Processor
Related Issues
- [FEATURE] Encina.GoogleCloudPubSub - Google Cloud Pub/Sub transport #237-[FEATURE] Encina.Dapr - Dapr PubSub integration #242 - Other transport implementations
- [FEATURE] ISchemaRegistryProvider: Abstraction and Implementations (Confluent, Azure, Apicurio) #602 - Schema Registry integration
Metadata
Metadata
Assignees
Labels
area-transportMessage transport/broker implementationsMessage transport/broker implementationscloud-azureMicrosoft AzureMicrosoft Azurecomplexity-mediumComplexity: MediumComplexity: MediumenhancementNew feature or requestNew feature or requestnew-packageRequires creating a new NuGet packageRequires creating a new NuGet packagephase-2-functionalityPhase 2: Functionality - new featuresPhase 2: Functionality - new featurespriority-mediumPriority: Medium (⭐⭐⭐)Priority: Medium (⭐⭐⭐)