RUM-5555 Benchmarks: Collect Memory Metric #1993
Conversation
maxep
force-pushed
the
maxep/RUM-5555/memory-metrics
branch
from
e7e13e0 to
436afd3
Compare
maxep
force-pushed
the
maxep/RUM-5555/memory-metrics
branch
from
c0165f1 to
121b2fd
Compare
| /// | ||
| /// - Parameter series: The timeseries. | ||
| func submit(series: [Serie]) throws { | ||
| var data = try series.reduce(Data()) { data, serie in |
There was a problem hiding this comment.
Should we expect large payloads here? If so, what happens if it exceeds a certain size?
There was a problem hiding this comment.
There is no caching so we will upload 3 metrics per payload, with only one point per metrics. Each upload will happen every 10 seconds as configured here. So each payload will be very small 👍
| run: run | ||
| ) | ||
| ) | ||
| case .profiling: |
There was a problem hiding this comment.
How will the .profiling use case differ from .metrics in practice?
There was a problem hiding this comment.
Yes, we will perform different runs from baseline, metrics, and profiling. This way we avoid skewing metrics data with profiling overhead.
| _ = meter.createDoubleObservableGauge(name: "ios.benchmark.memory") { metric in | ||
| do { | ||
| let mem = try Memory.footprint() | ||
| metric.observe(value: mem, labels: labels) |
There was a problem hiding this comment.
question/ How does it work? We create Gauge metric ✅ but when / how frequently will this closure be called?
There was a problem hiding this comment.
It is based on Asynchronous Gauge from otel specs where Callback functions will be called only when the Meter is being observed.
The meter is observed when there is a push, in our case it will be every 10s
| /// Replacement of otel `DatadogExporter` for metrics. | ||
| /// | ||
| /// This version does not store data to disk, it uploads to the intake directly. | ||
| /// Additionally, it does not crash. |
There was a problem hiding this comment.
Is this crash a known issue in opentelemetry-swift? If so let's link it here, so we can go back to the OOB exporter once fixed. If there are no reports, let's make one.
There was a problem hiding this comment.
I haven't reported it just yet, will do.
Basically, these force unwrap can fail.
| case baseline | ||
| case metrics | ||
| case profiling |
There was a problem hiding this comment.
suggestion/ This is quite crucial piece to understand the whole Benchmark automation. Can we add comments explaining what instrumentation is activated during each run?
What and why?
Collect Memory metrics during benchmarks.
How?
Scenarioabstraction: split interface from instrumentation to facilitate baseline runs.Review checklist