[coreinternal/aggregateutil] Aggregate exponential histogram data points which have different offsets #42412
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Previously, when aggregating exponential histogram data points which have different offsets in the
transformormetricstransformprocessors, the processor would not aggregate the data points. Follow-up processors, such asdeltatocumulative, might then drop some of the data points, leading to wrong metrics values being sent forward.This pull request adjusts the bucket counts in each data point to match each other, which allows all the data points in a group to be aggregated correctly to a single data point, regardless of the offset. The offset of that data point is set to the minimum offset encountered in the different data points and zero buckets are trimmed away from the front of the list.
(Side note: I'm quite inexperienced at Golang, so please let me know if I did something stupid in the code. 🙏)
Testing
New test
testDataExpHistogramWithDifferentOffsetsis added. Additionally, this has been running in our test environment with real data fromdatadogreceiver getting aggregated correctly.