From dbbf24e783f982349add6980cc974fc1f7c339a2 Mon Sep 17 00:00:00 2001 From: James Moessis Date: Wed, 25 Jan 2023 14:07:55 +1100 Subject: [PATCH 1/3] remove featuregate and old scraper implementation --- receiver/dockerstatsreceiver/README.md | 25 +- receiver/dockerstatsreceiver/config.go | 3 - receiver/dockerstatsreceiver/config_test.go | 1 - receiver/dockerstatsreceiver/factory.go | 16 +- .../dockerstatsreceiver/integration_test.go | 1 - receiver/dockerstatsreceiver/metrics.go | 296 ------------- receiver/dockerstatsreceiver/metrics_test.go | 400 ------------------ receiver/dockerstatsreceiver/receiver.go | 221 +++++++++- receiver/dockerstatsreceiver/receiver_test.go | 1 - receiver/dockerstatsreceiver/receiver_v2.go | 223 ---------- .../dockerstatsreceiver/testdata/config.yaml | 1 - 11 files changed, 215 insertions(+), 973 deletions(-) delete mode 100644 receiver/dockerstatsreceiver/metrics.go delete mode 100644 receiver/dockerstatsreceiver/metrics_test.go delete mode 100644 receiver/dockerstatsreceiver/receiver_v2.go diff --git a/receiver/dockerstatsreceiver/README.md b/receiver/dockerstatsreceiver/README.md index 018c705ea9f0..e3f2309db75f 100644 --- a/receiver/dockerstatsreceiver/README.md +++ b/receiver/dockerstatsreceiver/README.md @@ -34,9 +34,9 @@ only unmatched container image names should be monitored. `!/my?egex/` will monitor all containers whose name doesn't match the compiled regex `my?egex`. - Globs are non-regex items (e.g. `/items/`) containing any of the following: `*[]{}?`. Negations are supported: `!my*container` will monitor all containers whose image name doesn't match the blob `my*container`. -- `provide_per_core_cpu_metrics` (default = `false`): Whether to report `cpu.usage.percpu` metrics. - `timeout` (default = `5s`): The request timeout for any docker daemon query. - `api_version` (default = `1.22`): The Docker client API version (must be 1.22+). [Docker API versions](https://docs.docker.com/engine/api/). +- `metrics` (defaults at [./documentation.md](./documentation.md)): Enables/disables individual metrics. See [./documentation.md](./documentation.md) for full detail. Example: @@ -57,7 +57,11 @@ receivers: - undesired-container - /.*undesired.*/ - another-*-container - provide_per_core_cpu_metrics: true + metrics: + container.cpu.usage.percpu: + enabled: true + container.network.io.usage.tx_dropped: + enabled: false ``` The full list of settings exposed for this receiver are documented [here](./config.go) @@ -66,21 +70,10 @@ with detailed sample configurations [here](./testdata/config.yaml). [alpha]: https://github.com/open-telemetry/opentelemetry-collector#alpha [contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib - -## Feature Gates - -See the [Collector feature gates](https://github.com/open-telemetry/opentelemetry-collector/blob/main/featuregate/README.md#collector-feature-gates) for an overview of feature gates in the collector. - -**STABLE**: `receiver.dockerstats.useScraperV2` - -The feature gate `receiver.dockerstats.useScraperV2` allows collection of selective metrics that is described in [documentation.md](./documentation.md). When the feature gate is disabled, the metrics settings are mostly ignored and not configurable with minor variation in metric name and attributes. - -This is considered a breaking change for existing users of this receiver, and it is recommended to migrate to the new implementation when possible. Leave this feature gate enabled to avoid having to migrate any visualisations or alerts. - -This feature gate is enabled by default, and eventually the old implementation will be removed (aiming for 0.71.0). - ### Migrating from ScraperV1 to ScraperV2 +*Note: These changes are now in effect and ScraperV1 have been removed as of v0.71.* + There are some breaking changes from ScraperV1 to ScraperV2. The work done for these changes is tracked in [#9794](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/9794). | Breaking Change | Action | @@ -88,4 +81,4 @@ There are some breaking changes from ScraperV1 to ScraperV2. The work done for t | Many metrics are no longer emitted by default. | See [documentation.md](./documentation.md) to see which metrics are enabled by default. Enable/disable as desired. | | BlockIO metrics names changed. The type of operation is no longer in the metric name suffix, and is now in an attribute. For example `container.blockio.io_merged_recursive.read` becomes `container.blockio.io_merged_recursive` with an `operation:read` attribute. | Be aware of the metric name changes and make any adjustments to what your downstream expects from BlockIO metrics. | | Memory metrics measured in Bytes are now [non-monotonic sums](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/data-model.md#opentelemetry-protocol-data-model-consumer-recommendations) instead of gauges. | Most likely there is no action. The aggregation type is different but the values are the same. Be aware of how your downstream handles gauges vs non-monotonic sums. | -| Config option `provide_per_core_cpu_metrics` will be deprecated and removed. | Enable the `container.cpu.usage.percpu` metric as per [documentation.md](./documentation.md). | +| Config option `provide_per_core_cpu_metrics` has been removed. | Enable the `container.cpu.usage.percpu` metric as per [documentation.md](./documentation.md). | diff --git a/receiver/dockerstatsreceiver/config.go b/receiver/dockerstatsreceiver/config.go index c07f1ed976fa..83c7c2088739 100644 --- a/receiver/dockerstatsreceiver/config.go +++ b/receiver/dockerstatsreceiver/config.go @@ -53,9 +53,6 @@ type Config struct { // A list of filters whose matching images are to be excluded. Supports literals, globs, and regex. ExcludedImages []string `mapstructure:"excluded_images"` - // Whether to report all CPU metrics. Default is false - ProvidePerCoreCPUMetrics bool `mapstructure:"provide_per_core_cpu_metrics"` - // Docker client API version. Default is 1.22 DockerAPIVersion float64 `mapstructure:"api_version"` diff --git a/receiver/dockerstatsreceiver/config_test.go b/receiver/dockerstatsreceiver/config_test.go index 9ec6a6272096..b0c910af6198 100644 --- a/receiver/dockerstatsreceiver/config_test.go +++ b/receiver/dockerstatsreceiver/config_test.go @@ -52,7 +52,6 @@ func TestLoadConfig(t *testing.T) { Timeout: 20 * time.Second, DockerAPIVersion: 1.24, - ProvidePerCoreCPUMetrics: true, ExcludedImages: []string{ "undesired-container", "another-*-container", diff --git a/receiver/dockerstatsreceiver/factory.go b/receiver/dockerstatsreceiver/factory.go index d238870aad7e..c9a05bc0a73a 100644 --- a/receiver/dockerstatsreceiver/factory.go +++ b/receiver/dockerstatsreceiver/factory.go @@ -32,12 +32,12 @@ const ( stability = component.StabilityLevelAlpha ) -var useScraperV2 = featuregate.GlobalRegistry().MustRegister( +var _ = featuregate.GlobalRegistry().MustRegister( "receiver.dockerstats.useScraperV2", featuregate.StageStable, featuregate.WithRegisterDescription("When enabled, the receiver will use the function ScrapeV2 to collect metrics. This allows each metric to be turned off/on via config. The new metrics are slightly different to the legacy implementation."), featuregate.WithRegisterReferenceURL("https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/9794"), - featuregate.WithRegisterRemovalVersion("0.71.0"), + featuregate.WithRegisterRemovalVersion("0.74.0"), ) func NewFactory() rcvr.Factory { @@ -68,17 +68,7 @@ func createMetricsReceiver( dockerConfig := config.(*Config) dsr := newReceiver(params, dockerConfig) - scrapeFunc := dsr.scrape - if useScraperV2.IsEnabled() { - scrapeFunc = dsr.scrapeV2 - } else { - params.Logger.Warn( - "You are using the deprecated ScraperV1, which will " + - "be disabled by default in an upcoming release." + - "See the dockerstatsreceiver/README.md for more info.") - } - - scrp, err := scraperhelper.NewScraper(typeStr, scrapeFunc, scraperhelper.WithStart(dsr.start)) + scrp, err := scraperhelper.NewScraper(typeStr, dsr.scrapeV2, scraperhelper.WithStart(dsr.start)) if err != nil { return nil, err } diff --git a/receiver/dockerstatsreceiver/integration_test.go b/receiver/dockerstatsreceiver/integration_test.go index 86072faeb359..615e597b75f9 100644 --- a/receiver/dockerstatsreceiver/integration_test.go +++ b/receiver/dockerstatsreceiver/integration_test.go @@ -114,7 +114,6 @@ func TestAllMetricsIntegration(t *testing.T) { consumer := new(consumertest.MetricsSink) f, config := factory() - config.ProvidePerCoreCPUMetrics = true params, ctx, cancel := paramsAndContext(t) defer cancel() diff --git a/receiver/dockerstatsreceiver/metrics.go b/receiver/dockerstatsreceiver/metrics.go deleted file mode 100644 index 43331b45b184..000000000000 --- a/receiver/dockerstatsreceiver/metrics.go +++ /dev/null @@ -1,296 +0,0 @@ -// Copyright 2020 OpenTelemetry Authors -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -package dockerstatsreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/dockerstatsreceiver" - -import ( - "fmt" - "sort" - "strconv" - "strings" - - dtypes "github.com/docker/docker/api/types" - "go.opentelemetry.io/collector/pdata/pcommon" - "go.opentelemetry.io/collector/pdata/pmetric" - conventions "go.opentelemetry.io/collector/semconv/v1.6.1" - - "github.com/open-telemetry/opentelemetry-collector-contrib/internal/docker" -) - -const ( - metricPrefix = "container." -) - -func ContainerStatsToMetrics( - now pcommon.Timestamp, - containerStats *dtypes.StatsJSON, - container docker.Container, - config *Config, -) pmetric.ResourceMetrics { - rs := pmetric.NewResourceMetrics() - rs.SetSchemaUrl(conventions.SchemaURL) - resourceAttr := rs.Resource().Attributes() - resourceAttr.PutStr(conventions.AttributeContainerRuntime, "docker") - resourceAttr.PutStr(conventions.AttributeContainerID, container.ID) - resourceAttr.PutStr(conventions.AttributeContainerImageName, container.Config.Image) - resourceAttr.PutStr(conventions.AttributeContainerName, strings.TrimPrefix(container.Name, "/")) - resourceAttr.PutStr("container.hostname", container.Config.Hostname) - updateConfiguredResourceAttributes(resourceAttr, container, config) - ils := rs.ScopeMetrics().AppendEmpty() - - appendBlockioMetrics(ils.Metrics(), &containerStats.BlkioStats, now) - appendCPUMetrics(ils.Metrics(), &containerStats.CPUStats, &containerStats.PreCPUStats, now, config.ProvidePerCoreCPUMetrics) - appendMemoryMetrics(ils.Metrics(), &containerStats.MemoryStats, now) - appendNetworkMetrics(ils.Metrics(), &containerStats.Networks, now) - - return rs -} - -func updateConfiguredResourceAttributes(resourceAttr pcommon.Map, container docker.Container, config *Config) { - for k, label := range config.EnvVarsToMetricLabels { - if v := container.EnvMap[k]; v != "" { - resourceAttr.PutStr(label, v) - } - } - - for k, label := range config.ContainerLabelsToMetricLabels { - if v := container.Config.Labels[k]; v != "" { - resourceAttr.PutStr(label, v) - } - } -} - -type blkioStat struct { - name string - unit string - entries []dtypes.BlkioStatEntry -} - -// metrics for https://www.kernel.org/doc/Documentation/cgroup-v1/blkio-controller.txt -func appendBlockioMetrics(dest pmetric.MetricSlice, blkioStats *dtypes.BlkioStats, ts pcommon.Timestamp) { - for _, blkiostat := range []blkioStat{ - {"io_merged_recursive", "1", blkioStats.IoMergedRecursive}, - {"io_queued_recursive", "1", blkioStats.IoQueuedRecursive}, - {"io_service_bytes_recursive", "By", blkioStats.IoServiceBytesRecursive}, - {"io_service_time_recursive", "ns", blkioStats.IoServiceTimeRecursive}, - {"io_serviced_recursive", "1", blkioStats.IoServicedRecursive}, - {"io_time_recursive", "ms", blkioStats.IoTimeRecursive}, - {"io_wait_time_recursive", "1", blkioStats.IoWaitTimeRecursive}, - {"sectors_recursive", "1", blkioStats.SectorsRecursive}, - } { - labelKeys := []string{"device_major", "device_minor"} - for _, stat := range blkiostat.entries { - if stat.Op == "" { - continue - } - - statName := fmt.Sprintf("%s.%s", blkiostat.name, strings.ToLower(stat.Op)) - metricName := fmt.Sprintf("blockio.%s", statName) - labelValues := []string{strconv.FormatUint(stat.Major, 10), strconv.FormatUint(stat.Minor, 10)} - populateCumulative(dest.AppendEmpty(), metricName, blkiostat.unit, int64(stat.Value), ts, labelKeys, labelValues) - } - } -} - -func appendCPUMetrics(dest pmetric.MetricSlice, cpuStats *dtypes.CPUStats, previousCPUStats *dtypes.CPUStats, ts pcommon.Timestamp, providePerCoreMetrics bool) { - populateCumulative(dest.AppendEmpty(), "cpu.usage.system", "ns", int64(cpuStats.SystemUsage), ts, nil, nil) - populateCumulative(dest.AppendEmpty(), "cpu.usage.total", "ns", int64(cpuStats.CPUUsage.TotalUsage), ts, nil, nil) - - populateCumulative(dest.AppendEmpty(), "cpu.usage.kernelmode", "ns", int64(cpuStats.CPUUsage.UsageInKernelmode), ts, nil, nil) - populateCumulative(dest.AppendEmpty(), "cpu.usage.usermode", "ns", int64(cpuStats.CPUUsage.UsageInUsermode), ts, nil, nil) - - populateCumulative(dest.AppendEmpty(), "cpu.throttling_data.periods", "1", int64(cpuStats.ThrottlingData.Periods), ts, nil, nil) - populateCumulative(dest.AppendEmpty(), "cpu.throttling_data.throttled_periods", "1", int64(cpuStats.ThrottlingData.ThrottledPeriods), ts, nil, nil) - populateCumulative(dest.AppendEmpty(), "cpu.throttling_data.throttled_time", "ns", int64(cpuStats.ThrottlingData.ThrottledTime), ts, nil, nil) - - populateGaugeF(dest.AppendEmpty(), "cpu.percent", "1", calculateCPUPercent(previousCPUStats, cpuStats), ts, nil, nil) - - if !providePerCoreMetrics { - return - } - - percpuValues := make([]int64, 0, len(cpuStats.CPUUsage.PercpuUsage)) - percpuLabelKeys := []string{"core"} - percpuLabelValues := make([][]string, 0, len(cpuStats.CPUUsage.PercpuUsage)) - for coreNum, v := range cpuStats.CPUUsage.PercpuUsage { - percpuValues = append(percpuValues, int64(v)) - percpuLabelValues = append(percpuLabelValues, []string{fmt.Sprintf("cpu%s", strconv.Itoa(coreNum))}) - } - populateCumulativeMultiPoints(dest.AppendEmpty(), "cpu.usage.percpu", "ns", percpuValues, ts, percpuLabelKeys, percpuLabelValues) -} - -// From container.calculateCPUPercentUnix() -// https://github.com/docker/cli/blob/dbd96badb6959c2b7070664aecbcf0f7c299c538/cli/command/container/stats_helpers.go -// Copyright 2012-2017 Docker, Inc. -// This product includes software developed at Docker, Inc. (https://www.docker.com). -// The following is courtesy of our legal counsel: -// Use and transfer of Docker may be subject to certain restrictions by the -// United States and other governments. -// It is your responsibility to ensure that your use and/or transfer does not -// violate applicable laws. -// For more information, please see https://www.bis.doc.gov -// See also https://www.apache.org/dev/crypto.html and/or seek legal counsel. -func calculateCPUPercent(previous *dtypes.CPUStats, v *dtypes.CPUStats) float64 { - var ( - cpuPercent = 0.0 - // calculate the change for the cpu usage of the container in between readings - cpuDelta = float64(v.CPUUsage.TotalUsage) - float64(previous.CPUUsage.TotalUsage) - // calculate the change for the entire system between readings - systemDelta = float64(v.SystemUsage) - float64(previous.SystemUsage) - onlineCPUs = float64(v.OnlineCPUs) - ) - - if onlineCPUs == 0.0 { - onlineCPUs = float64(len(v.CPUUsage.PercpuUsage)) - } - if systemDelta > 0.0 && cpuDelta > 0.0 { - cpuPercent = (cpuDelta / systemDelta) * onlineCPUs * 100.0 - } - return cpuPercent -} - -var memoryStatsThatAreCumulative = map[string]bool{ - "pgfault": true, - "pgmajfault": true, - "pgpgin": true, - "pgpgout": true, - "total_pgfault": true, - "total_pgmajfault": true, - "total_pgpgin": true, - "total_pgpgout": true, -} - -func appendMemoryMetrics(dest pmetric.MetricSlice, memoryStats *dtypes.MemoryStats, ts pcommon.Timestamp) { - totalUsage := int64(memoryStats.Usage - memoryStats.Stats["total_cache"]) - populateGauge(dest.AppendEmpty(), "memory.usage.limit", int64(memoryStats.Limit), ts) - populateGauge(dest.AppendEmpty(), "memory.usage.total", totalUsage, ts) - - pctUsed := calculateMemoryPercent(memoryStats) - - populateGaugeF(dest.AppendEmpty(), "memory.percent", "1", pctUsed, ts, nil, nil) - populateGauge(dest.AppendEmpty(), "memory.usage.max", int64(memoryStats.MaxUsage), ts) - - // Sorted iteration for reproducibility, largely for testing - sortedNames := make([]string, 0, len(memoryStats.Stats)) - for statName := range memoryStats.Stats { - sortedNames = append(sortedNames, statName) - } - sort.Strings(sortedNames) - - for _, statName := range sortedNames { - v := memoryStats.Stats[statName] - metricName := fmt.Sprintf("memory.%s", statName) - if _, exists := memoryStatsThatAreCumulative[statName]; exists { - populateCumulative(dest.AppendEmpty(), metricName, "1", int64(v), ts, nil, nil) - } else { - populateGauge(dest.AppendEmpty(), metricName, int64(v), ts) - } - } -} - -func appendNetworkMetrics(dest pmetric.MetricSlice, networks *map[string]dtypes.NetworkStats, ts pcommon.Timestamp) { - if networks == nil || *networks == nil { - return - } - - labelKeys := []string{"interface"} - for nic, stats := range *networks { - labelValues := []string{nic} - - populateCumulative(dest.AppendEmpty(), "network.io.usage.rx_bytes", "By", int64(stats.RxBytes), ts, labelKeys, labelValues) - populateCumulative(dest.AppendEmpty(), "network.io.usage.tx_bytes", "By", int64(stats.TxBytes), ts, labelKeys, labelValues) - - populateCumulative(dest.AppendEmpty(), "network.io.usage.rx_dropped", "1", int64(stats.RxDropped), ts, labelKeys, labelValues) - populateCumulative(dest.AppendEmpty(), "network.io.usage.rx_errors", "1", int64(stats.RxErrors), ts, labelKeys, labelValues) - populateCumulative(dest.AppendEmpty(), "network.io.usage.rx_packets", "1", int64(stats.RxPackets), ts, labelKeys, labelValues) - populateCumulative(dest.AppendEmpty(), "network.io.usage.tx_dropped", "1", int64(stats.TxDropped), ts, labelKeys, labelValues) - populateCumulative(dest.AppendEmpty(), "network.io.usage.tx_errors", "1", int64(stats.TxErrors), ts, labelKeys, labelValues) - populateCumulative(dest.AppendEmpty(), "network.io.usage.tx_packets", "1", int64(stats.TxPackets), ts, labelKeys, labelValues) - } -} - -func populateCumulative(dest pmetric.Metric, name string, unit string, val int64, ts pcommon.Timestamp, labelKeys []string, labelValues []string) { - populateMetricMetadata(dest, name, unit, pmetric.MetricTypeSum) - sum := dest.Sum() - sum.SetIsMonotonic(true) - sum.SetAggregationTemporality(pmetric.AggregationTemporalityCumulative) - dp := sum.DataPoints().AppendEmpty() - dp.SetIntValue(val) - dp.SetTimestamp(ts) - populateAttributes(dp.Attributes(), labelKeys, labelValues) -} - -func populateCumulativeMultiPoints(dest pmetric.Metric, name string, unit string, vals []int64, ts pcommon.Timestamp, labelKeys []string, labelValues [][]string) { - populateMetricMetadata(dest, name, unit, pmetric.MetricTypeSum) - sum := dest.Sum() - sum.SetIsMonotonic(true) - sum.SetAggregationTemporality(pmetric.AggregationTemporalityCumulative) - dps := sum.DataPoints() - dps.EnsureCapacity(len(vals)) - for i := range vals { - dp := dps.AppendEmpty() - dp.SetIntValue(vals[i]) - dp.SetTimestamp(ts) - populateAttributes(dp.Attributes(), labelKeys, labelValues[i]) - } -} - -func populateGauge(dest pmetric.Metric, name string, val int64, ts pcommon.Timestamp) { - // Unit, labelKeys, labelValues always constants, when that changes add them as argument to the func. - populateMetricMetadata(dest, name, "By", pmetric.MetricTypeGauge) - sum := dest.Gauge() - dp := sum.DataPoints().AppendEmpty() - dp.SetIntValue(val) - dp.SetTimestamp(ts) - populateAttributes(dp.Attributes(), nil, nil) -} - -func populateGaugeF(dest pmetric.Metric, name string, unit string, val float64, ts pcommon.Timestamp, labelKeys []string, labelValues []string) { - populateMetricMetadata(dest, name, unit, pmetric.MetricTypeGauge) - sum := dest.Gauge() - dp := sum.DataPoints().AppendEmpty() - dp.SetDoubleValue(val) - dp.SetTimestamp(ts) - populateAttributes(dp.Attributes(), labelKeys, labelValues) -} - -func populateMetricMetadata(dest pmetric.Metric, name string, unit string, ty pmetric.MetricType) { - dest.SetName(metricPrefix + name) - dest.SetUnit(unit) - switch ty { - case pmetric.MetricTypeGauge: - dest.SetEmptyGauge() - case pmetric.MetricTypeSum: - dest.SetEmptySum() - case pmetric.MetricTypeHistogram: - dest.SetEmptyHistogram() - case pmetric.MetricTypeExponentialHistogram: - dest.SetEmptyExponentialHistogram() - case pmetric.MetricTypeSummary: - dest.SetEmptySummary() - } -} - -func populateAttributes(dest pcommon.Map, labelKeys []string, labelValues []string) { - for i := range labelKeys { - dest.PutStr(labelKeys[i], labelValues[i]) - } -} - -func calculateMemoryPercent(memoryStats *dtypes.MemoryStats) float64 { - if float64(memoryStats.Limit) == 0 { - return 0 - } - return 100.0 * (float64(memoryStats.Usage) - float64(memoryStats.Stats["cache"])) / float64(memoryStats.Limit) -} diff --git a/receiver/dockerstatsreceiver/metrics_test.go b/receiver/dockerstatsreceiver/metrics_test.go deleted file mode 100644 index de6fc36443ec..000000000000 --- a/receiver/dockerstatsreceiver/metrics_test.go +++ /dev/null @@ -1,400 +0,0 @@ -// Copyright 2020 OpenTelemetry Authors -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -package dockerstatsreceiver - -import ( - "encoding/json" - "os" - "path/filepath" - "testing" - "time" - - dtypes "github.com/docker/docker/api/types" - "github.com/stretchr/testify/assert" - "go.opentelemetry.io/collector/pdata/pcommon" - "go.opentelemetry.io/collector/pdata/pmetric" - conventions "go.opentelemetry.io/collector/semconv/v1.6.1" - - "github.com/open-telemetry/opentelemetry-collector-contrib/internal/docker" - "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest/pmetrictest" -) - -type MetricType int32 - -const ( - MetricTypeCumulative MetricType = iota - MetricTypeGauge - MetricTypeDoubleGauge -) - -type Metric struct { - name string - mtype MetricType - unit string - labelKeys []string - values []Value -} - -type Value struct { - labelValues []string - value int64 - doubleValue float64 -} - -func metricsData( - ts pcommon.Timestamp, - resourceLabels map[string]string, - metrics ...Metric, -) pmetric.ResourceMetrics { - rLabels := mergeMaps(defaultLabels(), resourceLabels) - rs := pmetric.NewResourceMetrics() - rs.SetSchemaUrl(conventions.SchemaURL) - rsAttr := rs.Resource().Attributes() - for k, v := range rLabels { - rsAttr.PutStr(k, v) - } - - mdMetrics := rs.ScopeMetrics().AppendEmpty().Metrics() - mdMetrics.EnsureCapacity(len(metrics)) - for _, m := range metrics { - mdMetric := mdMetrics.AppendEmpty() - mdMetric.SetName(m.name) - mdMetric.SetUnit(m.unit) - - var dps pmetric.NumberDataPointSlice - switch m.mtype { - case MetricTypeCumulative: - mdMetric.SetEmptySum().SetIsMonotonic(true) - mdMetric.Sum().SetAggregationTemporality(pmetric.AggregationTemporalityCumulative) - dps = mdMetric.Sum().DataPoints() - case MetricTypeGauge, MetricTypeDoubleGauge: - dps = mdMetric.SetEmptyGauge().DataPoints() - } - - for _, v := range m.values { - dp := dps.AppendEmpty() - dp.SetTimestamp(ts) - if m.mtype == MetricTypeDoubleGauge { - dp.SetDoubleValue(v.doubleValue) - } else { - dp.SetIntValue(v.value) - } - populateAttributes(dp.Attributes(), m.labelKeys, v.labelValues) - } - } - - return rs -} - -func defaultLabels() map[string]string { - return map[string]string{ - "container.runtime": "docker", - "container.hostname": "abcdef012345", - "container.id": "a2596076ca048f02bcd16a8acd12a7ea2d3bc430d1cde095357239dd3925a4c3", - "container.image.name": "myImage", - "container.name": "my-container-name", - } -} - -func defaultMetrics() []Metric { - return []Metric{ - {name: "container.blockio.io_service_bytes_recursive.read", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 56500224}}}, - {name: "container.blockio.io_service_bytes_recursive.write", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 12103680}}}, - {name: "container.blockio.io_service_bytes_recursive.sync", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 65314816}}}, - {name: "container.blockio.io_service_bytes_recursive.async", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 3289088}}}, - {name: "container.blockio.io_service_bytes_recursive.discard", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 0}}}, - {name: "container.blockio.io_service_bytes_recursive.total", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 68603904}}}, - {name: "container.blockio.io_serviced_recursive.read", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 985}}}, - {name: "container.blockio.io_serviced_recursive.write", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 2073}}}, - {name: "container.blockio.io_serviced_recursive.sync", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 2902}}}, - {name: "container.blockio.io_serviced_recursive.async", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 156}}}, - {name: "container.blockio.io_serviced_recursive.discard", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 0}}}, - {name: "container.blockio.io_serviced_recursive.total", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 3058}}}, - {name: "container.cpu.usage.system", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 4525290000000}}}, - {name: "container.cpu.usage.total", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 8043152341}}}, - {name: "container.cpu.usage.kernelmode", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 970000000}}}, - {name: "container.cpu.usage.usermode", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 3510000000}}}, - {name: "container.cpu.throttling_data.periods", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.throttling_data.throttled_periods", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.throttling_data.throttled_time", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.percent", mtype: MetricTypeDoubleGauge, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, doubleValue: 0.19316}}}, - {name: "container.memory.usage.limit", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 1026359296}}}, - {name: "container.memory.usage.total", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 75915264}}}, - {name: "container.memory.percent", mtype: MetricTypeDoubleGauge, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, doubleValue: 7.396558329608582}}}, - {name: "container.memory.usage.max", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 325246976}}}, - {name: "container.memory.active_anon", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 72585216}}}, - {name: "container.memory.active_file", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 40316928}}}, - {name: "container.memory.cache", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 80760832}}}, - {name: "container.memory.dirty", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.hierarchical_memory_limit", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 9223372036854771712}}}, - {name: "container.memory.hierarchical_memsw_limit", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.inactive_anon", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.inactive_file", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 40579072}}}, - {name: "container.memory.mapped_file", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 37711872}}}, - {name: "container.memory.pgfault", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 21714}}}, - {name: "container.memory.pgmajfault", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 396}}}, - {name: "container.memory.pgpgin", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 85140}}}, - {name: "container.memory.pgpgout", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 47694}}}, - {name: "container.memory.rss", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 72568832}}}, - {name: "container.memory.rss_huge", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.total_active_anon", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 72585216}}}, - {name: "container.memory.total_active_file", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 40316928}}}, - {name: "container.memory.total_cache", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 80760832}}}, - {name: "container.memory.total_dirty", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.total_inactive_anon", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.total_inactive_file", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 40579072}}}, - {name: "container.memory.total_mapped_file", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 37711872}}}, - {name: "container.memory.total_pgfault", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 21714}}}, - {name: "container.memory.total_pgmajfault", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 396}}}, - {name: "container.memory.total_pgpgin", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 85140}}}, - {name: "container.memory.total_pgpgout", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 47694}}}, - {name: "container.memory.total_rss", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 72568832}}}, - {name: "container.memory.total_rss_huge", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.total_unevictable", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.total_writeback", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.unevictable", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.writeback", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.network.io.usage.rx_bytes", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 2787669}}}, - {name: "container.network.io.usage.tx_bytes", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 2275281}}}, - {name: "container.network.io.usage.rx_dropped", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 0}}}, - {name: "container.network.io.usage.rx_errors", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 0}}}, - {name: "container.network.io.usage.rx_packets", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 16598}}}, - {name: "container.network.io.usage.tx_dropped", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 0}}}, - {name: "container.network.io.usage.tx_errors", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 0}}}, - {name: "container.network.io.usage.tx_packets", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 9050}}}, - } -} - -func mergeMaps(maps ...map[string]string) map[string]string { - merged := map[string]string{} - for _, m := range maps { - for k, v := range m { - merged[k] = v - } - } - return merged -} - -func TestZeroValueStats(t *testing.T) { - stats := &dtypes.StatsJSON{ - Stats: dtypes.Stats{ - BlkioStats: dtypes.BlkioStats{}, - CPUStats: dtypes.CPUStats{}, - MemoryStats: dtypes.MemoryStats{}, - }, - Networks: nil, - } - containers := containerJSON(t) - config := &Config{} - - now := pcommon.NewTimestampFromTime(time.Now()) - md := ContainerStatsToMetrics(now, stats, containers, config) - - metrics := []Metric{ - {name: "container.cpu.usage.system", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.usage.total", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.usage.kernelmode", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.usage.usermode", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.throttling_data.periods", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.throttling_data.throttled_periods", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.throttling_data.throttled_time", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.percent", mtype: MetricTypeDoubleGauge, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, doubleValue: 0}}}, - {name: "container.memory.usage.limit", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.usage.total", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.percent", mtype: MetricTypeDoubleGauge, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, doubleValue: 0}}}, - {name: "container.memory.usage.max", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - } - assert.NoError(t, pmetrictest.CompareResourceMetrics(metricsData(now, nil, metrics...), md)) -} - -func statsJSON(t *testing.T) *dtypes.StatsJSON { - statsRaw, err := os.ReadFile(filepath.Join("testdata", "stats.json")) - if err != nil { - t.Fatal(err) - } - - var stats dtypes.StatsJSON - err = json.Unmarshal(statsRaw, &stats) - if err != nil { - t.Fatal(err) - } - return &stats -} - -func containerJSON(t *testing.T) docker.Container { - containerRaw, err := os.ReadFile(filepath.Join("testdata", "container.json")) - if err != nil { - t.Fatal(err) - } - - var container dtypes.ContainerJSON - err = json.Unmarshal(containerRaw, &container) - if err != nil { - t.Fatal(err) - } - return docker.Container{ - ContainerJSON: &container, - EnvMap: docker.ContainerEnvToMap(container.Config.Env), - } -} - -func TestStatsToDefaultMetrics(t *testing.T) { - stats := statsJSON(t) - containers := containerJSON(t) - config := &Config{} - - now := pcommon.NewTimestampFromTime(time.Now()) - md := ContainerStatsToMetrics(now, stats, containers, config) - - assert.NoError(t, pmetrictest.CompareResourceMetrics(metricsData(now, nil, defaultMetrics()...), md)) -} - -func TestStatsToAllMetrics(t *testing.T) { - stats := statsJSON(t) - containers := containerJSON(t) - config := &Config{ - ProvidePerCoreCPUMetrics: true, - } - - now := pcommon.NewTimestampFromTime(time.Now()) - md := ContainerStatsToMetrics(now, stats, containers, config) - - metrics := []Metric{ - {name: "container.blockio.io_service_bytes_recursive.read", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 56500224}}}, - {name: "container.blockio.io_service_bytes_recursive.write", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 12103680}}}, - {name: "container.blockio.io_service_bytes_recursive.sync", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 65314816}}}, - {name: "container.blockio.io_service_bytes_recursive.async", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 3289088}}}, - {name: "container.blockio.io_service_bytes_recursive.discard", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 0}}}, - {name: "container.blockio.io_service_bytes_recursive.total", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 68603904}}}, - {name: "container.blockio.io_serviced_recursive.read", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 985}}}, - {name: "container.blockio.io_serviced_recursive.write", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 2073}}}, - {name: "container.blockio.io_serviced_recursive.sync", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 2902}}}, - {name: "container.blockio.io_serviced_recursive.async", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 156}}}, - {name: "container.blockio.io_serviced_recursive.discard", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 0}}}, - {name: "container.blockio.io_serviced_recursive.total", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"device_major", "device_minor"}, values: []Value{{labelValues: []string{"202", "0"}, value: 3058}}}, - {name: "container.cpu.usage.system", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 4525290000000}}}, - {name: "container.cpu.usage.total", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 8043152341}}}, - {name: "container.cpu.usage.kernelmode", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 970000000}}}, - {name: "container.cpu.usage.usermode", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 3510000000}}}, - {name: "container.cpu.throttling_data.periods", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.throttling_data.throttled_periods", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.throttling_data.throttled_time", mtype: MetricTypeCumulative, unit: "ns", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.cpu.percent", mtype: MetricTypeDoubleGauge, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, doubleValue: 0.19316}}}, - { - name: "container.cpu.usage.percpu", - mtype: MetricTypeCumulative, - unit: "ns", - labelKeys: []string{"core"}, - values: []Value{ - {labelValues: []string{"cpu0"}, value: 8043152341}, - {labelValues: []string{"cpu1"}, value: 0}, - {labelValues: []string{"cpu2"}, value: 0}, - {labelValues: []string{"cpu3"}, value: 0}, - {labelValues: []string{"cpu4"}, value: 0}, - {labelValues: []string{"cpu5"}, value: 0}, - {labelValues: []string{"cpu6"}, value: 0}, - {labelValues: []string{"cpu7"}, value: 0}, - }, - }, - {name: "container.memory.usage.limit", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 1026359296}}}, - {name: "container.memory.usage.total", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 75915264}}}, - {name: "container.memory.percent", mtype: MetricTypeDoubleGauge, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, doubleValue: 7.396558329608582}}}, - {name: "container.memory.usage.max", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 325246976}}}, - {name: "container.memory.active_anon", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 72585216}}}, - {name: "container.memory.active_file", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 40316928}}}, - {name: "container.memory.cache", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 80760832}}}, - {name: "container.memory.dirty", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.hierarchical_memory_limit", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 9223372036854771712}}}, - {name: "container.memory.hierarchical_memsw_limit", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.inactive_anon", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.inactive_file", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 40579072}}}, - {name: "container.memory.mapped_file", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 37711872}}}, - {name: "container.memory.pgfault", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 21714}}}, - {name: "container.memory.pgmajfault", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 396}}}, - {name: "container.memory.pgpgin", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 85140}}}, - {name: "container.memory.pgpgout", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 47694}}}, - {name: "container.memory.rss", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 72568832}}}, - {name: "container.memory.rss_huge", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.total_active_anon", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 72585216}}}, - {name: "container.memory.total_active_file", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 40316928}}}, - {name: "container.memory.total_cache", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 80760832}}}, - {name: "container.memory.total_dirty", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.total_inactive_anon", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.total_inactive_file", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 40579072}}}, - {name: "container.memory.total_mapped_file", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 37711872}}}, - {name: "container.memory.total_pgfault", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 21714}}}, - {name: "container.memory.total_pgmajfault", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 396}}}, - {name: "container.memory.total_pgpgin", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 85140}}}, - {name: "container.memory.total_pgpgout", mtype: MetricTypeCumulative, unit: "1", labelKeys: nil, values: []Value{{labelValues: nil, value: 47694}}}, - {name: "container.memory.total_rss", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 72568832}}}, - {name: "container.memory.total_rss_huge", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.total_unevictable", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.total_writeback", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.unevictable", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.memory.writeback", mtype: MetricTypeGauge, unit: "By", labelKeys: nil, values: []Value{{labelValues: nil, value: 0}}}, - {name: "container.network.io.usage.rx_bytes", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 2787669}}}, - {name: "container.network.io.usage.tx_bytes", mtype: MetricTypeCumulative, unit: "By", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 2275281}}}, - {name: "container.network.io.usage.rx_dropped", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 0}}}, - {name: "container.network.io.usage.rx_errors", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 0}}}, - {name: "container.network.io.usage.rx_packets", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 16598}}}, - {name: "container.network.io.usage.tx_dropped", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 0}}}, - {name: "container.network.io.usage.tx_errors", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 0}}}, - {name: "container.network.io.usage.tx_packets", mtype: MetricTypeCumulative, unit: "1", labelKeys: []string{"interface"}, values: []Value{{labelValues: []string{"eth0"}, value: 9050}}}, - } - - assert.NoError(t, pmetrictest.CompareResourceMetrics(metricsData(now, nil, metrics...), md)) -} - -func TestEnvVarToMetricLabels(t *testing.T) { - stats := statsJSON(t) - containers := containerJSON(t) - config := &Config{ - EnvVarsToMetricLabels: map[string]string{ - "MY_ENV_VAR": "my.env.to.metric.label", - "MY_OTHER_ENV_VAR": "my.other.env.to.metric.label", - }, - } - - now := pcommon.NewTimestampFromTime(time.Now()) - md := ContainerStatsToMetrics(now, stats, containers, config) - - expectedLabels := map[string]string{ - "my.env.to.metric.label": "my_env_var_value", - "my.other.env.to.metric.label": "my_other_env_var_value", - } - - assert.NoError(t, pmetrictest.CompareResourceMetrics(metricsData(now, expectedLabels, defaultMetrics()...), md)) -} - -func TestContainerLabelToMetricLabels(t *testing.T) { - stats := statsJSON(t) - containers := containerJSON(t) - config := &Config{ - ContainerLabelsToMetricLabels: map[string]string{ - "my.specified.docker.label": "my.docker.to.metric.label", - "other.specified.docker.label": "my.other.docker.to.metric.label", - }, - } - - now := pcommon.NewTimestampFromTime(time.Now()) - md := ContainerStatsToMetrics(now, stats, containers, config) - - expectedLabels := map[string]string{ - "my.docker.to.metric.label": "my_specified_docker_label_value", - "my.other.docker.to.metric.label": "other_specified_docker_label_value", - } - - assert.NoError(t, pmetrictest.CompareResourceMetrics(metricsData(now, expectedLabels, defaultMetrics()...), md)) -} diff --git a/receiver/dockerstatsreceiver/receiver.go b/receiver/dockerstatsreceiver/receiver.go index d264f4ee039e..986731d5520d 100644 --- a/receiver/dockerstatsreceiver/receiver.go +++ b/receiver/dockerstatsreceiver/receiver.go @@ -16,9 +16,13 @@ package dockerstatsreceiver // import "github.com/open-telemetry/opentelemetry-c import ( "context" + "fmt" + "strconv" + "strings" "sync" "time" + dtypes "github.com/docker/docker/api/types" "go.opentelemetry.io/collector/component" "go.opentelemetry.io/collector/pdata/pcommon" "go.opentelemetry.io/collector/pdata/pmetric" @@ -30,11 +34,19 @@ import ( "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/dockerstatsreceiver/internal/metadata" ) +const defaultResourcesLen = 5 + const ( defaultDockerAPIVersion = 1.22 minimalRequiredDockerAPIVersion = 1.22 ) +type resultV2 struct { + stats *dtypes.StatsJSON + container *docker.Container + err error +} + type receiver struct { config *Config settings rcvr.CreateSettings @@ -43,9 +55,6 @@ type receiver struct { } func newReceiver(set rcvr.CreateSettings, config *Config) *receiver { - if config.ProvidePerCoreCPUMetrics { - config.MetricsConfig.ContainerCPUUsagePercpu.Enabled = config.ProvidePerCoreCPUMetrics - } return &receiver{ config: config, settings: set, @@ -72,14 +81,9 @@ func (r *receiver) start(ctx context.Context, _ component.Host) error { return nil } -type result struct { - md pmetric.ResourceMetrics - err error -} - -func (r *receiver) scrape(ctx context.Context) (pmetric.Metrics, error) { +func (r *receiver) scrapeV2(ctx context.Context) (pmetric.Metrics, error) { containers := r.client.Containers() - results := make(chan result, len(containers)) + results := make(chan resultV2, len(containers)) wg := &sync.WaitGroup{} wg.Add(len(containers)) @@ -88,13 +92,14 @@ func (r *receiver) scrape(ctx context.Context) (pmetric.Metrics, error) { defer wg.Done() statsJSON, err := r.client.FetchContainerStatsAsJSON(ctx, c) if err != nil { - results <- result{md: pmetric.ResourceMetrics{}, err: err} + results <- resultV2{nil, &c, err} return } - results <- result{ - md: ContainerStatsToMetrics(pcommon.NewTimestampFromTime(time.Now()), statsJSON, c, r.config), - err: nil} + results <- resultV2{ + stats: statsJSON, + container: &c, + err: nil} }(container) } @@ -102,15 +107,195 @@ func (r *receiver) scrape(ctx context.Context) (pmetric.Metrics, error) { close(results) var errs error - md := pmetric.NewMetrics() + + now := pcommon.NewTimestampFromTime(time.Now()) for res := range results { if res.err != nil { - // Don't know the number of failed metrics, but one container fetch is a partial error. + // Don't know the number of failed stats, but one container fetch is a partial error. errs = multierr.Append(errs, scrapererror.NewPartialScrapeError(res.err, 0)) continue } - res.md.MoveTo(md.ResourceMetrics().AppendEmpty()) + r.recordContainerStats(now, res.stats, res.container) + } + + return r.mb.Emit(), errs +} + +func (r *receiver) recordContainerStats(now pcommon.Timestamp, containerStats *dtypes.StatsJSON, container *docker.Container) { + r.recordCPUMetrics(now, &containerStats.CPUStats, &containerStats.PreCPUStats) + r.recordMemoryMetrics(now, &containerStats.MemoryStats) + r.recordBlkioMetrics(now, &containerStats.BlkioStats) + r.recordNetworkMetrics(now, &containerStats.Networks) + + // Always-present resource attrs + the user-configured resource attrs + resourceCapacity := defaultResourcesLen + len(r.config.EnvVarsToMetricLabels) + len(r.config.ContainerLabelsToMetricLabels) + resourceMetricsOptions := make([]metadata.ResourceMetricsOption, 0, resourceCapacity) + resourceMetricsOptions = append(resourceMetricsOptions, + metadata.WithContainerRuntime("docker"), + metadata.WithContainerHostname(container.Config.Hostname), + metadata.WithContainerID(container.ID), + metadata.WithContainerImageName(container.Config.Image), + metadata.WithContainerName(strings.TrimPrefix(container.Name, "/"))) + + for k, label := range r.config.EnvVarsToMetricLabels { + if v := container.EnvMap[k]; v != "" { + resourceMetricsOptions = append(resourceMetricsOptions, func(ras metadata.ResourceAttributesSettings, rm pmetric.ResourceMetrics) { + rm.Resource().Attributes().PutStr(label, v) + }) + } + } + for k, label := range r.config.ContainerLabelsToMetricLabels { + if v := container.Config.Labels[k]; v != "" { + resourceMetricsOptions = append(resourceMetricsOptions, func(ras metadata.ResourceAttributesSettings, rm pmetric.ResourceMetrics) { + rm.Resource().Attributes().PutStr(label, v) + }) + } + } + + r.mb.EmitForResource(resourceMetricsOptions...) +} + +func (r *receiver) recordMemoryMetrics(now pcommon.Timestamp, memoryStats *dtypes.MemoryStats) { + totalCache := memoryStats.Stats["total_cache"] + totalUsage := memoryStats.Usage - totalCache + r.mb.RecordContainerMemoryUsageMaxDataPoint(now, int64(memoryStats.MaxUsage)) + r.mb.RecordContainerMemoryPercentDataPoint(now, calculateMemoryPercent(memoryStats)) + r.mb.RecordContainerMemoryUsageTotalDataPoint(now, int64(totalUsage)) + r.mb.RecordContainerMemoryUsageLimitDataPoint(now, int64(memoryStats.Limit)) + + recorders := map[string]func(pcommon.Timestamp, int64){ + "cache": r.mb.RecordContainerMemoryCacheDataPoint, + "total_cache": r.mb.RecordContainerMemoryTotalCacheDataPoint, + "rss": r.mb.RecordContainerMemoryRssDataPoint, + "total_rss": r.mb.RecordContainerMemoryTotalRssDataPoint, + "rss_huge": r.mb.RecordContainerMemoryRssHugeDataPoint, + "total_rss_huge": r.mb.RecordContainerMemoryTotalRssHugeDataPoint, + "dirty": r.mb.RecordContainerMemoryDirtyDataPoint, + "total_dirty": r.mb.RecordContainerMemoryTotalDirtyDataPoint, + "writeback": r.mb.RecordContainerMemoryWritebackDataPoint, + "total_writeback": r.mb.RecordContainerMemoryTotalWritebackDataPoint, + "mapped_file": r.mb.RecordContainerMemoryMappedFileDataPoint, + "total_mapped_file": r.mb.RecordContainerMemoryTotalMappedFileDataPoint, + "pgpgin": r.mb.RecordContainerMemoryPgpginDataPoint, + "total_pgpgin": r.mb.RecordContainerMemoryTotalPgpginDataPoint, + "pgpgout": r.mb.RecordContainerMemoryPgpgoutDataPoint, + "total_pgpgout": r.mb.RecordContainerMemoryTotalPgpgoutDataPoint, + "swap": r.mb.RecordContainerMemorySwapDataPoint, + "total_swap": r.mb.RecordContainerMemoryTotalSwapDataPoint, + "pgfault": r.mb.RecordContainerMemoryPgfaultDataPoint, + "total_pgfault": r.mb.RecordContainerMemoryTotalPgfaultDataPoint, + "pgmajfault": r.mb.RecordContainerMemoryPgmajfaultDataPoint, + "total_pgmajfault": r.mb.RecordContainerMemoryTotalPgmajfaultDataPoint, + "inactive_anon": r.mb.RecordContainerMemoryInactiveAnonDataPoint, + "total_inactive_anon": r.mb.RecordContainerMemoryTotalInactiveAnonDataPoint, + "active_anon": r.mb.RecordContainerMemoryActiveAnonDataPoint, + "total_active_anon": r.mb.RecordContainerMemoryTotalActiveAnonDataPoint, + "inactive_file": r.mb.RecordContainerMemoryInactiveFileDataPoint, + "total_inactive_file": r.mb.RecordContainerMemoryTotalInactiveFileDataPoint, + "active_file": r.mb.RecordContainerMemoryActiveFileDataPoint, + "total_active_file": r.mb.RecordContainerMemoryTotalActiveFileDataPoint, + "unevictable": r.mb.RecordContainerMemoryUnevictableDataPoint, + "total_unevictable": r.mb.RecordContainerMemoryTotalUnevictableDataPoint, + "hierarchical_memory_limit": r.mb.RecordContainerMemoryHierarchicalMemoryLimitDataPoint, + "hierarchical_memsw_limit": r.mb.RecordContainerMemoryHierarchicalMemswLimitDataPoint, + } + + for name, val := range memoryStats.Stats { + if recorder, ok := recorders[name]; ok { + recorder(now, int64(val)) + } + } +} + +type blkioRecorder func(now pcommon.Timestamp, val int64, devMaj string, devMin string, operation string) + +func (r *receiver) recordBlkioMetrics(now pcommon.Timestamp, blkioStats *dtypes.BlkioStats) { + recordSingleBlkioStat(now, blkioStats.IoMergedRecursive, r.mb.RecordContainerBlockioIoMergedRecursiveDataPoint) + recordSingleBlkioStat(now, blkioStats.IoQueuedRecursive, r.mb.RecordContainerBlockioIoQueuedRecursiveDataPoint) + recordSingleBlkioStat(now, blkioStats.IoServiceBytesRecursive, r.mb.RecordContainerBlockioIoServiceBytesRecursiveDataPoint) + recordSingleBlkioStat(now, blkioStats.IoServiceTimeRecursive, r.mb.RecordContainerBlockioIoServiceTimeRecursiveDataPoint) + recordSingleBlkioStat(now, blkioStats.IoServicedRecursive, r.mb.RecordContainerBlockioIoServicedRecursiveDataPoint) + recordSingleBlkioStat(now, blkioStats.IoTimeRecursive, r.mb.RecordContainerBlockioIoTimeRecursiveDataPoint) + recordSingleBlkioStat(now, blkioStats.IoWaitTimeRecursive, r.mb.RecordContainerBlockioIoWaitTimeRecursiveDataPoint) + recordSingleBlkioStat(now, blkioStats.SectorsRecursive, r.mb.RecordContainerBlockioSectorsRecursiveDataPoint) +} + +func recordSingleBlkioStat(now pcommon.Timestamp, statEntries []dtypes.BlkioStatEntry, recorder blkioRecorder) { + for _, stat := range statEntries { + recorder( + now, + int64(stat.Value), + strconv.FormatUint(stat.Major, 10), + strconv.FormatUint(stat.Minor, 10), + strings.ToLower(stat.Op)) + } +} + +func (r *receiver) recordNetworkMetrics(now pcommon.Timestamp, networks *map[string]dtypes.NetworkStats) { + if networks == nil || *networks == nil { + return } - return md, errs + for netInterface, stats := range *networks { + r.mb.RecordContainerNetworkIoUsageRxBytesDataPoint(now, int64(stats.RxBytes), netInterface) + r.mb.RecordContainerNetworkIoUsageTxBytesDataPoint(now, int64(stats.TxBytes), netInterface) + r.mb.RecordContainerNetworkIoUsageRxDroppedDataPoint(now, int64(stats.RxDropped), netInterface) + r.mb.RecordContainerNetworkIoUsageTxDroppedDataPoint(now, int64(stats.TxDropped), netInterface) + r.mb.RecordContainerNetworkIoUsageRxPacketsDataPoint(now, int64(stats.RxPackets), netInterface) + r.mb.RecordContainerNetworkIoUsageTxPacketsDataPoint(now, int64(stats.TxPackets), netInterface) + r.mb.RecordContainerNetworkIoUsageRxErrorsDataPoint(now, int64(stats.RxErrors), netInterface) + r.mb.RecordContainerNetworkIoUsageTxErrorsDataPoint(now, int64(stats.TxErrors), netInterface) + } +} + +func (r *receiver) recordCPUMetrics(now pcommon.Timestamp, cpuStats *dtypes.CPUStats, prevStats *dtypes.CPUStats) { + r.mb.RecordContainerCPUUsageSystemDataPoint(now, int64(cpuStats.SystemUsage)) + r.mb.RecordContainerCPUUsageTotalDataPoint(now, int64(cpuStats.CPUUsage.TotalUsage)) + r.mb.RecordContainerCPUUsageKernelmodeDataPoint(now, int64(cpuStats.CPUUsage.UsageInKernelmode)) + r.mb.RecordContainerCPUUsageUsermodeDataPoint(now, int64(cpuStats.CPUUsage.UsageInUsermode)) + r.mb.RecordContainerCPUThrottlingDataThrottledPeriodsDataPoint(now, int64(cpuStats.ThrottlingData.ThrottledPeriods)) + r.mb.RecordContainerCPUThrottlingDataPeriodsDataPoint(now, int64(cpuStats.ThrottlingData.Periods)) + r.mb.RecordContainerCPUThrottlingDataThrottledTimeDataPoint(now, int64(cpuStats.ThrottlingData.ThrottledTime)) + r.mb.RecordContainerCPUPercentDataPoint(now, calculateCPUPercent(prevStats, cpuStats)) + + for coreNum, v := range cpuStats.CPUUsage.PercpuUsage { + r.mb.RecordContainerCPUUsagePercpuDataPoint(now, int64(v), fmt.Sprintf("cpu%s", strconv.Itoa(coreNum))) + } +} + +// From container.calculateCPUPercentUnix() +// https://github.com/docker/cli/blob/dbd96badb6959c2b7070664aecbcf0f7c299c538/cli/command/container/stats_helpers.go +// Copyright 2012-2017 Docker, Inc. +// This product includes software developed at Docker, Inc. (https://www.docker.com). +// The following is courtesy of our legal counsel: +// Use and transfer of Docker may be subject to certain restrictions by the +// United States and other governments. +// It is your responsibility to ensure that your use and/or transfer does not +// violate applicable laws. +// For more information, please see https://www.bis.doc.gov +// See also https://www.apache.org/dev/crypto.html and/or seek legal counsel. +func calculateCPUPercent(previous *dtypes.CPUStats, v *dtypes.CPUStats) float64 { + var ( + cpuPercent = 0.0 + // calculate the change for the cpu usage of the container in between readings + cpuDelta = float64(v.CPUUsage.TotalUsage) - float64(previous.CPUUsage.TotalUsage) + // calculate the change for the entire system between readings + systemDelta = float64(v.SystemUsage) - float64(previous.SystemUsage) + onlineCPUs = float64(v.OnlineCPUs) + ) + + if onlineCPUs == 0.0 { + onlineCPUs = float64(len(v.CPUUsage.PercpuUsage)) + } + if systemDelta > 0.0 && cpuDelta > 0.0 { + cpuPercent = (cpuDelta / systemDelta) * onlineCPUs * 100.0 + } + return cpuPercent +} + +func calculateMemoryPercent(memoryStats *dtypes.MemoryStats) float64 { + if float64(memoryStats.Limit) == 0 { + return 0 + } + return 100.0 * (float64(memoryStats.Usage) - float64(memoryStats.Stats["cache"])) / float64(memoryStats.Limit) } diff --git a/receiver/dockerstatsreceiver/receiver_test.go b/receiver/dockerstatsreceiver/receiver_test.go index 7de4e4896efc..c6de5defdf7c 100644 --- a/receiver/dockerstatsreceiver/receiver_test.go +++ b/receiver/dockerstatsreceiver/receiver_test.go @@ -192,7 +192,6 @@ func TestScrapeV2(t *testing.T) { cfg.Endpoint = tc.mockDockerEngine.URL cfg.EnvVarsToMetricLabels = map[string]string{"ENV_VAR": "env-var-metric-label"} cfg.ContainerLabelsToMetricLabels = map[string]string{"container.label": "container-metric-label"} - cfg.ProvidePerCoreCPUMetrics = true cfg.MetricsConfig = allMetricsEnabled receiver := newReceiver(receivertest.NewNopCreateSettings(), cfg) diff --git a/receiver/dockerstatsreceiver/receiver_v2.go b/receiver/dockerstatsreceiver/receiver_v2.go deleted file mode 100644 index 7b6947b13ac1..000000000000 --- a/receiver/dockerstatsreceiver/receiver_v2.go +++ /dev/null @@ -1,223 +0,0 @@ -// Copyright The OpenTelemetry Authors -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -package dockerstatsreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/dockerstatsreceiver" - -import ( - "context" - "fmt" - "strconv" - "strings" - "sync" - "time" - - dtypes "github.com/docker/docker/api/types" - "go.opentelemetry.io/collector/pdata/pcommon" - "go.opentelemetry.io/collector/pdata/pmetric" - "go.opentelemetry.io/collector/receiver/scrapererror" - "go.uber.org/multierr" - - "github.com/open-telemetry/opentelemetry-collector-contrib/internal/docker" - "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/dockerstatsreceiver/internal/metadata" -) - -const defaultResourcesLen = 5 - -type resultV2 struct { - stats *dtypes.StatsJSON - container *docker.Container - err error -} - -func (r *receiver) scrapeV2(ctx context.Context) (pmetric.Metrics, error) { - containers := r.client.Containers() - results := make(chan resultV2, len(containers)) - - wg := &sync.WaitGroup{} - wg.Add(len(containers)) - for _, container := range containers { - go func(c docker.Container) { - defer wg.Done() - statsJSON, err := r.client.FetchContainerStatsAsJSON(ctx, c) - if err != nil { - results <- resultV2{nil, &c, err} - return - } - - results <- resultV2{ - stats: statsJSON, - container: &c, - err: nil} - }(container) - } - - wg.Wait() - close(results) - - var errs error - - now := pcommon.NewTimestampFromTime(time.Now()) - for res := range results { - if res.err != nil { - // Don't know the number of failed stats, but one container fetch is a partial error. - errs = multierr.Append(errs, scrapererror.NewPartialScrapeError(res.err, 0)) - continue - } - r.recordContainerStats(now, res.stats, res.container) - } - - return r.mb.Emit(), errs -} - -func (r *receiver) recordContainerStats(now pcommon.Timestamp, containerStats *dtypes.StatsJSON, container *docker.Container) { - r.recordCPUMetrics(now, &containerStats.CPUStats, &containerStats.PreCPUStats) - r.recordMemoryMetrics(now, &containerStats.MemoryStats) - r.recordBlkioMetrics(now, &containerStats.BlkioStats) - r.recordNetworkMetrics(now, &containerStats.Networks) - - // Always-present resource attrs + the user-configured resource attrs - resourceCapacity := defaultResourcesLen + len(r.config.EnvVarsToMetricLabels) + len(r.config.ContainerLabelsToMetricLabels) - resourceMetricsOptions := make([]metadata.ResourceMetricsOption, 0, resourceCapacity) - resourceMetricsOptions = append(resourceMetricsOptions, - metadata.WithContainerRuntime("docker"), - metadata.WithContainerHostname(container.Config.Hostname), - metadata.WithContainerID(container.ID), - metadata.WithContainerImageName(container.Config.Image), - metadata.WithContainerName(strings.TrimPrefix(container.Name, "/"))) - - for k, label := range r.config.EnvVarsToMetricLabels { - if v := container.EnvMap[k]; v != "" { - resourceMetricsOptions = append(resourceMetricsOptions, func(ras metadata.ResourceAttributesSettings, rm pmetric.ResourceMetrics) { - rm.Resource().Attributes().PutStr(label, v) - }) - } - } - for k, label := range r.config.ContainerLabelsToMetricLabels { - if v := container.Config.Labels[k]; v != "" { - resourceMetricsOptions = append(resourceMetricsOptions, func(ras metadata.ResourceAttributesSettings, rm pmetric.ResourceMetrics) { - rm.Resource().Attributes().PutStr(label, v) - }) - } - } - - r.mb.EmitForResource(resourceMetricsOptions...) -} - -func (r *receiver) recordMemoryMetrics(now pcommon.Timestamp, memoryStats *dtypes.MemoryStats) { - totalCache := memoryStats.Stats["total_cache"] - totalUsage := memoryStats.Usage - totalCache - r.mb.RecordContainerMemoryUsageMaxDataPoint(now, int64(memoryStats.MaxUsage)) - r.mb.RecordContainerMemoryPercentDataPoint(now, calculateMemoryPercent(memoryStats)) - r.mb.RecordContainerMemoryUsageTotalDataPoint(now, int64(totalUsage)) - r.mb.RecordContainerMemoryUsageLimitDataPoint(now, int64(memoryStats.Limit)) - - recorders := map[string]func(pcommon.Timestamp, int64){ - "cache": r.mb.RecordContainerMemoryCacheDataPoint, - "total_cache": r.mb.RecordContainerMemoryTotalCacheDataPoint, - "rss": r.mb.RecordContainerMemoryRssDataPoint, - "total_rss": r.mb.RecordContainerMemoryTotalRssDataPoint, - "rss_huge": r.mb.RecordContainerMemoryRssHugeDataPoint, - "total_rss_huge": r.mb.RecordContainerMemoryTotalRssHugeDataPoint, - "dirty": r.mb.RecordContainerMemoryDirtyDataPoint, - "total_dirty": r.mb.RecordContainerMemoryTotalDirtyDataPoint, - "writeback": r.mb.RecordContainerMemoryWritebackDataPoint, - "total_writeback": r.mb.RecordContainerMemoryTotalWritebackDataPoint, - "mapped_file": r.mb.RecordContainerMemoryMappedFileDataPoint, - "total_mapped_file": r.mb.RecordContainerMemoryTotalMappedFileDataPoint, - "pgpgin": r.mb.RecordContainerMemoryPgpginDataPoint, - "total_pgpgin": r.mb.RecordContainerMemoryTotalPgpginDataPoint, - "pgpgout": r.mb.RecordContainerMemoryPgpgoutDataPoint, - "total_pgpgout": r.mb.RecordContainerMemoryTotalPgpgoutDataPoint, - "swap": r.mb.RecordContainerMemorySwapDataPoint, - "total_swap": r.mb.RecordContainerMemoryTotalSwapDataPoint, - "pgfault": r.mb.RecordContainerMemoryPgfaultDataPoint, - "total_pgfault": r.mb.RecordContainerMemoryTotalPgfaultDataPoint, - "pgmajfault": r.mb.RecordContainerMemoryPgmajfaultDataPoint, - "total_pgmajfault": r.mb.RecordContainerMemoryTotalPgmajfaultDataPoint, - "inactive_anon": r.mb.RecordContainerMemoryInactiveAnonDataPoint, - "total_inactive_anon": r.mb.RecordContainerMemoryTotalInactiveAnonDataPoint, - "active_anon": r.mb.RecordContainerMemoryActiveAnonDataPoint, - "total_active_anon": r.mb.RecordContainerMemoryTotalActiveAnonDataPoint, - "inactive_file": r.mb.RecordContainerMemoryInactiveFileDataPoint, - "total_inactive_file": r.mb.RecordContainerMemoryTotalInactiveFileDataPoint, - "active_file": r.mb.RecordContainerMemoryActiveFileDataPoint, - "total_active_file": r.mb.RecordContainerMemoryTotalActiveFileDataPoint, - "unevictable": r.mb.RecordContainerMemoryUnevictableDataPoint, - "total_unevictable": r.mb.RecordContainerMemoryTotalUnevictableDataPoint, - "hierarchical_memory_limit": r.mb.RecordContainerMemoryHierarchicalMemoryLimitDataPoint, - "hierarchical_memsw_limit": r.mb.RecordContainerMemoryHierarchicalMemswLimitDataPoint, - } - - for name, val := range memoryStats.Stats { - if recorder, ok := recorders[name]; ok { - recorder(now, int64(val)) - } - } -} - -type blkioRecorder func(now pcommon.Timestamp, val int64, devMaj string, devMin string, operation string) - -func (r *receiver) recordBlkioMetrics(now pcommon.Timestamp, blkioStats *dtypes.BlkioStats) { - recordSingleBlkioStat(now, blkioStats.IoMergedRecursive, r.mb.RecordContainerBlockioIoMergedRecursiveDataPoint) - recordSingleBlkioStat(now, blkioStats.IoQueuedRecursive, r.mb.RecordContainerBlockioIoQueuedRecursiveDataPoint) - recordSingleBlkioStat(now, blkioStats.IoServiceBytesRecursive, r.mb.RecordContainerBlockioIoServiceBytesRecursiveDataPoint) - recordSingleBlkioStat(now, blkioStats.IoServiceTimeRecursive, r.mb.RecordContainerBlockioIoServiceTimeRecursiveDataPoint) - recordSingleBlkioStat(now, blkioStats.IoServicedRecursive, r.mb.RecordContainerBlockioIoServicedRecursiveDataPoint) - recordSingleBlkioStat(now, blkioStats.IoTimeRecursive, r.mb.RecordContainerBlockioIoTimeRecursiveDataPoint) - recordSingleBlkioStat(now, blkioStats.IoWaitTimeRecursive, r.mb.RecordContainerBlockioIoWaitTimeRecursiveDataPoint) - recordSingleBlkioStat(now, blkioStats.SectorsRecursive, r.mb.RecordContainerBlockioSectorsRecursiveDataPoint) -} - -func recordSingleBlkioStat(now pcommon.Timestamp, statEntries []dtypes.BlkioStatEntry, recorder blkioRecorder) { - for _, stat := range statEntries { - recorder( - now, - int64(stat.Value), - strconv.FormatUint(stat.Major, 10), - strconv.FormatUint(stat.Minor, 10), - strings.ToLower(stat.Op)) - } -} - -func (r *receiver) recordNetworkMetrics(now pcommon.Timestamp, networks *map[string]dtypes.NetworkStats) { - if networks == nil || *networks == nil { - return - } - - for netInterface, stats := range *networks { - r.mb.RecordContainerNetworkIoUsageRxBytesDataPoint(now, int64(stats.RxBytes), netInterface) - r.mb.RecordContainerNetworkIoUsageTxBytesDataPoint(now, int64(stats.TxBytes), netInterface) - r.mb.RecordContainerNetworkIoUsageRxDroppedDataPoint(now, int64(stats.RxDropped), netInterface) - r.mb.RecordContainerNetworkIoUsageTxDroppedDataPoint(now, int64(stats.TxDropped), netInterface) - r.mb.RecordContainerNetworkIoUsageRxPacketsDataPoint(now, int64(stats.RxPackets), netInterface) - r.mb.RecordContainerNetworkIoUsageTxPacketsDataPoint(now, int64(stats.TxPackets), netInterface) - r.mb.RecordContainerNetworkIoUsageRxErrorsDataPoint(now, int64(stats.RxErrors), netInterface) - r.mb.RecordContainerNetworkIoUsageTxErrorsDataPoint(now, int64(stats.TxErrors), netInterface) - } -} - -func (r *receiver) recordCPUMetrics(now pcommon.Timestamp, cpuStats *dtypes.CPUStats, prevStats *dtypes.CPUStats) { - r.mb.RecordContainerCPUUsageSystemDataPoint(now, int64(cpuStats.SystemUsage)) - r.mb.RecordContainerCPUUsageTotalDataPoint(now, int64(cpuStats.CPUUsage.TotalUsage)) - r.mb.RecordContainerCPUUsageKernelmodeDataPoint(now, int64(cpuStats.CPUUsage.UsageInKernelmode)) - r.mb.RecordContainerCPUUsageUsermodeDataPoint(now, int64(cpuStats.CPUUsage.UsageInUsermode)) - r.mb.RecordContainerCPUThrottlingDataThrottledPeriodsDataPoint(now, int64(cpuStats.ThrottlingData.ThrottledPeriods)) - r.mb.RecordContainerCPUThrottlingDataPeriodsDataPoint(now, int64(cpuStats.ThrottlingData.Periods)) - r.mb.RecordContainerCPUThrottlingDataThrottledTimeDataPoint(now, int64(cpuStats.ThrottlingData.ThrottledTime)) - r.mb.RecordContainerCPUPercentDataPoint(now, calculateCPUPercent(prevStats, cpuStats)) - - for coreNum, v := range cpuStats.CPUUsage.PercpuUsage { - r.mb.RecordContainerCPUUsagePercpuDataPoint(now, int64(v), fmt.Sprintf("cpu%s", strconv.Itoa(coreNum))) - } -} diff --git a/receiver/dockerstatsreceiver/testdata/config.yaml b/receiver/dockerstatsreceiver/testdata/config.yaml index 204a145f5e21..821033ddcb84 100644 --- a/receiver/dockerstatsreceiver/testdata/config.yaml +++ b/receiver/dockerstatsreceiver/testdata/config.yaml @@ -13,7 +13,6 @@ docker_stats/allsettings: excluded_images: - undesired-container - another-*-container - provide_per_core_cpu_metrics: true metrics: container.cpu.usage.system: enabled: false From 21bd1bddec6170c7e1978feb2e2488c5358cc1c9 Mon Sep 17 00:00:00 2001 From: James Moessis Date: Wed, 8 Feb 2023 13:54:13 +1100 Subject: [PATCH 2/3] add changelog --- .chloggen/remove-dockerstats-featuregate-v2.yaml | 4 ++++ 1 file changed, 4 insertions(+) create mode 100755 .chloggen/remove-dockerstats-featuregate-v2.yaml diff --git a/.chloggen/remove-dockerstats-featuregate-v2.yaml b/.chloggen/remove-dockerstats-featuregate-v2.yaml new file mode 100755 index 000000000000..c5d257f83b2a --- /dev/null +++ b/.chloggen/remove-dockerstats-featuregate-v2.yaml @@ -0,0 +1,4 @@ +change_type: enhancement +component: dockerstatsreceiver +note: Removed the deprecated scraper implementation which was toggled by the featuregate `receiver.dockerstats.useScraperV2`. +issues: [18449, 9794] From 828b510acf7f25ba5f9c1d1ecdf66cc5e26f3aa5 Mon Sep 17 00:00:00 2001 From: James Moessis Date: Wed, 8 Feb 2023 14:59:52 +1100 Subject: [PATCH 3/3] change to breaking --- .chloggen/remove-dockerstats-featuregate-v2.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.chloggen/remove-dockerstats-featuregate-v2.yaml b/.chloggen/remove-dockerstats-featuregate-v2.yaml index c5d257f83b2a..8c702c00061b 100755 --- a/.chloggen/remove-dockerstats-featuregate-v2.yaml +++ b/.chloggen/remove-dockerstats-featuregate-v2.yaml @@ -1,4 +1,4 @@ -change_type: enhancement +change_type: breaking component: dockerstatsreceiver note: Removed the deprecated scraper implementation which was toggled by the featuregate `receiver.dockerstats.useScraperV2`. issues: [18449, 9794]