Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Base-2 exponential histogram aggregator #2261

Closed
wants to merge 11 commits into from

Conversation

jmacd
Copy link
Contributor

@jmacd jmacd commented Sep 27, 2021

Proposed auto-ranging base-2 exponential histogram aggregator.

This uses the logarithm mapping function shown in the reference implementation, see open-telemetry/oteps#179

This uses a circular backing array based on NrSketch WindowedCounterArray. This allows insertion above or below the current dense region in the histogram without moving data until it reaches maximum size. Like that implementation, this implementation uses the smallest integer width bucket possible, from uint8 through uint64.

When inserting a value that falls outside the range, the histogram may grow without a change of scale up to maxSize. The implementation will double in size until the maximum is reached.

The ideal scale of any single measurement is computed as 30-math.Abs(normalizedBase2ExponentOf(value)), which results in a bucket index in the range (-231, 231). When a new measurement falls outside the dense region or requires a 31-bit or greater index, the histogram is downscaled in a 2**N-to-1 collapse during which the backing array is rotated.

@codecov
Copy link

codecov bot commented Sep 27, 2021

Codecov Report

Merging #2261 (0a3a472) into main (3c8e185) will decrease coverage by 0.5%.
The diff coverage is 57.4%.

❗ Current head 0a3a472 differs from pull request most recent head 7044ef2. Consider uploading reports for the commit 7044ef2 to get more accurate results
Impacted file tree graph

@@           Coverage Diff           @@
##            main   #2261     +/-   ##
=======================================
- Coverage   72.4%   71.9%   -0.6%     
=======================================
  Files        168     171      +3     
  Lines      11872   12311    +439     
=======================================
+ Hits        8606    8859    +253     
- Misses      3031    3212    +181     
- Partials     235     240      +5     
Impacted Files Coverage Δ
sdk/export/metric/aggregation/aggregation.go 0.0% <ø> (ø)
...ric/aggregator/exponentialhistogram/exponential.go 53.4% <53.4%> (ø)
.../metric/aggregator/exponentialhistogram/float64.go 86.6% <86.6%> (ø)
...etric/aggregator/exponentialhistogram/logarithm.go 100.0% <100.0%> (ø)
exporters/jaeger/jaeger.go 93.4% <0.0%> (-0.9%) ⬇️
sdk/resource/os_unix.go 75.0% <0.0%> (ø)
...s/otlp/otlptrace/internal/connection/connection.go 16.4% <0.0%> (+1.5%) ⬆️

@jmacd jmacd closed this Sep 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant