Cloud Monitoring metrics example in README doesn't work

[DazWilkin]

The sample as-is appears to not work:
https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/master/exporter/metric/README.md

• See Challenged by 'hello world' #94
• There is no (longer) a kv package (now label?)
• meter := pusher.Provider().Meter("...") vs. global.Meter("...") as used by other examples

Recommend: ensuring this sample is part of the package's sources so that it can be tested when the exporter is built.

[nilebox]

Thanks for reporting the issue @DazWilkin
Feel free to send a PR fixing it if you have time for that.

Recommend: ensuring this sample is part of the package's sources so that it can be tested when the exporter is built.

We do have a full code example actually: https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/tree/master/example/metric
Does it work for you?

[DazWilkin]

Thanks @nilebox I'll try that tomorrow

[DazWilkin]

@nilebox thanks for the prompt response.

Your example works partially for me.

Cloud Monitoring receives the observer's values but not the counter's.

go run github.com/GoogleCloudPlatform/opentelemetry-operations-go/example/metric
go: finding module for package github.com/GoogleCloudPlatform/opentelemetry-operations-go/example/metric
go: downloading github.com/GoogleCloudPlatform/opentelemetry-operations-go/example/metric v0.11.0
go: found github.com/GoogleCloudPlatform/opentelemetry-operations-go/example/metric in github.com/GoogleCloudPlatform/opentelemetry-operations-go/example/metric v0.11.0
2020/09/10 08:42:20 Most recent data: counter 141, observer 12.89
2020/09/10 08:42:30 Most recent data: counter 182, observer 16.89
2020/09/10 08:42:40 Most recent data: counter 136, observer 16.84
2020/09/10 08:42:50 Most recent data: counter 165, observer 15.84
2020/09/10 08:43:00 Most recent data: counter 198, observer 13.49
2020/09/10 08:43:10 Most recent data: counter 122, observer 14.49
2020/09/10 08:43:11 Error during exporting TimeSeries: currently the aggregator is not supported: &{{0 0} 1044 [100 122 136 141 165 182 198]}
2020/09/10 08:43:11 rpc error: code = InvalidArgument desc = Request was missing field timeSeries.
2020/09/10 08:43:20 Most recent data: counter 170, observer 15.24
2020/09/10 08:43:30 Most recent data: counter 142, observer 16.79
2020/09/10 08:43:40 Most recent data: counter 182, observer 15.29
2020/09/10 08:43:50 Most recent data: counter 195, observer 16.740000000000002
2020/09/10 08:44:00 Most recent data: counter 173, observer 17.04
2020/09/10 08:44:10 Most recent data: counter 174, observer 14.69
2020/09/10 08:44:10 Error during exporting TimeSeries: currently the aggregator is not supported: &{{0 0} 2080 [100 122 136 141 142 165 170 173 174 182 182 195 198]}
2020/09/10 08:44:20 Most recent data: counter 102, observer 14.94
2020/09/10 08:44:30 Most recent data: counter 195, observer 15.39
2020/09/10 08:44:40 Most recent data: counter 194, observer 13.84
2020/09/10 08:44:50 Most recent data: counter 116, observer 12.34
2020/09/10 08:45:00 Most recent data: counter 172, observer 15.84
2020/09/10 08:45:10 Most recent data: counter 167, observer 15.54
2020/09/10 08:45:10 Error during exporting TimeSeries: currently the aggregator is not supported: &{{0 0} 3026 [100 102 116 122 136 141 142 165 167 170 172 173 174 182 182 194 195 195 198]}

I've been using metric explorer but double-checked using your dashboard script:

In my mash-up example, I'm also unable to view counter values.

I tried querying the counters and observer (works) and enumerating these time-series using the Cloud Monitoring API directly. The only one that returns time-series data is observer-a suggesting that it's a Monitoring API (not UI) issue:

curl \
--header "Authorization: Bearer ${TOKEN}" \
--header 'Accept: application/json' \
"https://monitoring.googleapis.com/v3/projects/${PROJECT}/timeSeries?aggregation.alignmentPeriod=%2B60s&aggregation.crossSeriesReducer=REDUCE_NONE&aggregation.perSeriesAligner=ALIGN_NONE&filter=metric.type%3D%22custom.googleapis.com%2Fopentelemetry%2Fobserver-a%22&interval.endTime=2020-09-10T23%3A59%3A59Z&interval.startTime=2020-09-10T00%3A00%3A00Z"

Both counter-a and observer-a returns "unit": "{not_a_unit}" too

[DazWilkin]

Hmmm... I cloned the repo (rather than just go run'ing it and the example does now publish both the counter and observer values.

What I don't understand is that I'm not providing a Google Project ID to the example, yet it works.

If I try running my code without the Project ID, it (correctly) errors:

projectID := os.Getenv("PROJECT")
opts := []cloudmonitoring.Option{
	// cloudmonitoring.WithProjectID(projectID),
}

results:

Google Cloud Monitoring: no project found with application default credentials

Somehow the example makes no reference to the project and yet it publishes values to the correct project.

[DazWilkin]

Using the latest commit rather than v0.11.0:

github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.11.1-0.20200910093037-ef5e24b7fcfc

I received a more appropriate error message that the label ex.com/lemons is invalid:

rpc error: code = InvalidArgument desc = One or more TimeSeries could not be written: Field timeSeries[0].metric.labels had an invalid value of "ex.com/lemons": Label key contains invalid characters.: timeSeries[0]

After revising that, my code now correctly delivers counter measurements to Cloud Monitoring:

[DazWilkin]

Revised (but possibly still imperfect) README example: #96

[nilebox]

Somehow the example makes no reference to the project and yet it publishes values to the correct project.

If you don't provide project ID, the exporter will try to auto-discover it:

opentelemetry-operations-go/exporter/metric/cloudmonitoring.go

Line 60 in 71af267

creds, err := google.FindDefaultCredentials(o.Context, monitoring.DefaultAuthScopes()...)

If you're running in a GCP environment, you don't need to explicitly set the project ID.

[yuriatgoogle]

I was able to get the exporter to work running locally (on MacOS) with the following:

	opts := []mexporter.Option{
		mexporter.WithProjectID(projectID),
	}

[punya]

Fixed by #96 and #97.