From f0fe061a7b660b151c3f0501b4ab988a961e9196 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Thu, 20 May 2021 14:24:19 -0700 Subject: [PATCH] Metrics SDK spec skeleton (#1673) * skeleton of metrics SDK spec * add more design questions * finish the skeleton * fix CI * add more links and clarification * nits * improve the wording for contex/baggage/trace interaction with metrics * fix typo * update the diagram to show logical in-memory state * update the diagram * update the wording * update the wording Co-authored-by: Sergey Kanzhelev --- specification/metrics/README.md | 2 +- specification/metrics/sdk.md | 144 ++++++++++++++++++++++++++++++++ 2 files changed, 145 insertions(+), 1 deletion(-) create mode 100644 specification/metrics/sdk.md diff --git a/specification/metrics/README.md b/specification/metrics/README.md index e711628c6fe..20ee764465a 100644 --- a/specification/metrics/README.md +++ b/specification/metrics/README.md @@ -107,7 +107,7 @@ SDK](../overview.md#sdk) concept concept for more information. ## Specifications * [Metrics API](./api.md) -* Metrics SDK (not available yet) +* [Metrics SDK](./sdk.md) * [Metrics Data Model and Protocol](datamodel.md) * [Semantic Conventions](./semantic_conventions/README.md) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md new file mode 100644 index 00000000000..513f88ca4c8 --- /dev/null +++ b/specification/metrics/sdk.md @@ -0,0 +1,144 @@ +# Metrics SDK + +**Status**: [Experimental](../document-status.md) + +**Owner:** + +* [Reiley Yang](https://github.com/reyang) + +**Domain Experts:** + +* [Bogdan Drutu](https://github.com/bogdandrutu) +* [Josh Suereth](https://github.com/jsuereth) +* [Joshua MacDonald](https://github.com/jmacd) + +Note: this specification is subject to major changes. To avoid thrusting +language client maintainers, we don't recommend OpenTelemetry clients to start +the implementation unless explicitly communicated via +[OTEP](https://github.com/open-telemetry/oteps#opentelemetry-enhancement-proposal-otep). + +
+ +Table of Contents + + +* [MeterProvider](#meterprovider) +* [MeasurementProcessor](#measurementprocessor) +* [MetricProcessor](#metricprocessor) +* [MetricExporter](#metricexporter) + * [Push Metric Exporter](#push-metric-exporter) + * [Pull Metric Exporter](#pull-metric-exporter) + +
+ +## MeterProvider + +TODO: + +* Allow multiple pipelines (processors, exporters). +* Configure "Views". +* Configure timing (related to [issue + 1432](https://github.com/open-telemetry/opentelemetry-specification/issues/1432)). + +## MeasurementProcessor + +`MeasurementProcessor` is an interface which allows hooks when a +[Measurement](./api.md#measurement) is recorded by an +[Instrument](./api.md#instrument). + +`MeasurementProcessor` MUST have access to: + +* The `Measurement` +* The `Instrument`, which is used to report the `Measurement` +* The `Resource`, which is associated with the `MeterProvider` + +In addition to things listed above, if the `Measurement` is reported by a +synchronous `Instrument` (e.g. [Counter](./api.md#counter)), +`MeasurementProcessor` MUST have access to: + +* [Baggage](../baggage/api.md) +* [Context](../context/context.md) +* The [Span](../trace/api.md#span) which is associated with the `Measurement` + +Depending on the programming language and runtime model, these can be provided +explicitly (e.g. as input arguments) or implicitly (e.g. [implicit +Context](../context/context.md#optional-global-operations) and the [currently +active span](../trace/api.md#context-interaction)). + +```text ++------------------+ +| MeterProvider | +----------------------+ +-----------------+ +| Meter A | Measurements... | | Metrics... | | +| Instrument X |-----------------> MeasurementProcessor +------------> In-memory state | +| Instrument Y + | | | | +| Meter B | +----------------------+ +-----------------+ +| Instrument Z | +| ... | +----------------------+ +-----------------+ +| ... | Measurements... | | Metrics... | | +| ... |-----------------> MeasurementProcessor +------------> In-memory state | +| ... | | | | | +| ... | +----------------------+ +-----------------+ ++------------------+ +``` + +## MetricProcessor + +`MetricProcessor` is an interface which allows hooks for [pre-aggregated metrics +data](./datamodel.md#timeseries-model). + +Built-in metric processors are responsible for batching and conversion of +metrics data to exportable representation and passing batches to exporters. + +The following diagram shows `MetricProcessor`'s relationship to other components +in the SDK: + +```text ++-----------------+ +-----------------+ +-----------------------+ +| | Metrics... | | Metrics... | | +> In-memory state | -----------> MetricProcessor | -----------> MetricExporter (push) |--> Another process +| | | | | | ++-----------------+ +-----------------+ +-----------------------+ + ++-----------------+ +-----------------+ +-----------------------+ +| | Metrics... | | Metrics... | | +> In-memory state |------------> MetricProcessor |------------> MetricExporter (pull) |--> Another process (scraper) +| | | | | | ++-----------------+ +-----------------+ +-----------------------+ +``` + +## MetricExporter + +`MetricExporter` defines the interface that protocol-specific exporters must +implement so that they can be plugged into OpenTelemetry SDK and support sending +of telemetry data. + +The goal of the interface is to minimize burden of implementation for +protocol-dependent telemetry exporters. The protocol exporter is expected to be +primarily a simple telemetry data encoder and transmitter. + +Metric Exporter has access to the [pre-aggregated metrics +data](./datamodel.md#timeseries-model). + +There could be multiple [Push Metric Exporters](#push-metric-exporter) or [Pull +Metric Exporters](#pull-metric-exporter) or even a mixture of both configured on +a given `MeterProvider`. Different exporters can run at different schedule, for +example: + +* Exporter A is a push exporter which sends data every 1 minute. +* Exporter B is a push exporter which sends data every 5 seconds. +* Exporter C is a pull exporter which reacts to a scraper over HTTP. +* Exporter D is a pull exporter which reacts to another scraper over a named + pipe. + +### Push Metric Exporter + +Push Metric Exporter sends the data on its own schedule. Here are some examples: + +* Sends the data based on a user configured schedule, e.g. every 1 minute. +* Sends the data when there is a severe error. + +### Pull Metric Exporter + +Pull Metric Exporter reacts to the metrics scrapers and reports the data +passively. This pattern has been widely adopted by +[Prometheus](https://prometheus.io/).