Repository navigation
Add otel.sdk.processor.log.processed metric - #7486
rajkumar-rangaraj merged 16 commits into
Conversation
Implement the otel.sdk.processor.log.processed counter metric for
BatchLogRecordExportProcessor and SimpleLogRecordExportProcessor.
The metric is emitted on meter 'otel.sdk.experimental' which requires
explicit opt-in via .AddMeter("otel.sdk.experimental") — zero overhead
without a listener.
Attributes:
- otel.component.type: batching_log_processor | simple_log_processor
- otel.component.name: unique instance identifier (e.g. batching_log_processor/0)
- error.type (conditional): queue_full | already_shutdown
Design:
- SdkSelfObservability internal helper owns the Meter and Counter
- Pre-allocated KeyValuePair[] tag arrays (no per-call allocation)
- Batch processor uses TryExport return value to classify success vs queue_full
- Both processors detect post-shutdown state via OnShutdown override
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #7486 +/- ##
==========================================
+ Coverage 90.46% 90.55% +0.09%
==========================================
Files 290 292 +2
Lines 16039 16122 +83
==========================================
+ Hits 14510 14600 +90
+ Misses 1529 1522 -7
Flags with carried forward coverage won't be shown. Click here to find out more.
|
The test was flaky on Windows ARM / net462 because with maxQueueSize=1 and maxExportBatchSize=1, the worker thread could drain the queue faster than items were being emitted, resulting in no drops. Use a blocking DelegatingExporter to hold the worker in Export() while filling the queue.
c6a9a18 to
8c89abe
Compare
- Copy MeterFactory from contrib for auto-versioning - Make Meter internal (removes SA1202 suppression) - Share base tags between success/error arrays - Single Counter.Add() call with local tag assignment - Shorten README overhead note
8c89abe to
6c5df34
Compare
|
This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or pushing will instruct the bot to automatically remove the label. This bot runs once per day. |
|
@cijothomas, I asked Codex /GPT 5.6 Sol. Briefly checked findings and it looks valid to me: Following comment is AI driven Reviewed commit: 1. Count batch success at exporter submission, not queue admissionPriority: P1
The semantic convention says that, for simple and batching processors, a log The successful increment should happen at the worker/export boundary. Queue Specification: 2. The simple processor reports exported records as
|
Pull request dashboard statusMerged · refreshed 2026-08-06 00:05 UTC Status above doesn't look right?
|
…bs-processor-log-processed # Conflicts: # src/OpenTelemetry/.publicApi/Stable/PublicAPI.Unshipped.txt # src/OpenTelemetry/CHANGELOG.md
|
Hi @cijothomas — just a friendly reminder that this pull request is waiting on you. There are still items that need your attention. See the dashboard status comment for the full list. You don't need to push a code change to hand it back — replying to move each discussion forward is enough, whether that's answering a question, explaining why no change is needed, or asking a follow-up. The dashboard then automatically routes it back to reviewers. If you believe this pull request is incorrectly routed as waiting on the author, comment |
Towards #3880
Adds the
otel.sdk.processor.log.processedcounter metric toBatchLogRecordExportProcessorandSimpleLogRecordExportProcessor, following the OTel SDK self-observability semantic conventions.This is the first SDK self-observability metric in the .NET SDK. More metrics (queue size, exporter counters, etc.) will follow in subsequent PRs.
Design
otel.sdk.experimental- requires explicit opt-in via.AddMeter("otel.sdk.experimental"). Without this,Counter.Add()is a no-op with zero overhead.Counter<long>namedotel.sdk.processor.log.processedotel.component.type-batching_log_processororsimple_log_processorotel.component.name- unique per instance (e.g.batching_log_processor/0)error.type(conditional) -queue_fullwhen buffer rejects,already_shutdownwhen processor has been shut downShutdown behavior (note for reviewers)
To count
already_shutdownaccurately, the batch processor needs a well-defined shutdown boundary: a record whoseOnEndis in flight whenShutdown()is called must be classified deterministically. It uses a small barrier (activeOnEndCountplus a bounded wait inOnShutdown) so an in-flightOnEndeither finishes enqueueing before the worker snapshots its drain target — so the record is drained and counted as processed — or observes shutdown and is countedalready_shutdown. As a side benefit this also tightens a pre-existing enqueue-vs-drain-snapshot race at shutdown, so it is a slight behavior change beyond adding the metric (records at the boundary are handled deterministically rather than silently lost). That underlying race is generic toBatchExportProcessor<T>; lifting this coordination into the shared batch infrastructure (to also cover span batching) can be a follow-up.SimpleLogRecordExportProcessordeliberately does not add this coordination — it accepts a benign shutdown race and is not intended for production use.Opt-in example
Why .AddMeter() instead of env variable or compile-time flag?
Metrics in .NET are inherently opt-in: without a
MeterProvidersubscribing to the meter,Counter.Add()is a no-op at near-zero cost. The.AddMeter("otel.sdk.experimental")call provides natural gating without requiring additional mechanisms like env variables or feature flags.