Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[metricbuilder] hide details of pdata #8382

Conversation

codeboten
Copy link
Contributor

Refactoring the metric builder to hide details of pdata. This added a couple of methods to the metric builder interface but i think that's ok.

Fix #8358

@dmitryax @djaglowski please take a look as you're familiar with the original issue.

@codeboten codeboten force-pushed the codeboten/refactor-metricbuilder branch from faadb82 to 9570e64 Compare March 10, 2022 23:22
receiver/mysqlreceiver/testdata/scraper/expected.json Outdated Show resolved Hide resolved
}

// EnsureCapacity TODO
func (mb *MetricsBuilder) EnsureCapacity(length int) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need these functions. Capacity can be always taken from number of enabled metrics

Copy link
Member

@dmitryax dmitryax Mar 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or we can follow similar approach to how we figure out capacity for datapoints from the previous scrapes.

cmd/mdatagen/metrics_v2.tmpl Outdated Show resolved Hide resolved
cmd/mdatagen/metrics_v2.tmpl Outdated Show resolved Hide resolved
cmd/mdatagen/metrics_v2.tmpl Outdated Show resolved Hide resolved
Copy link
Member

@djaglowski djaglowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change looks really nice from the perspective of a scraper author. There's little to no "overhead" left within the ScrapeFunc.

@codeboten codeboten force-pushed the codeboten/refactor-metricbuilder branch from 3c82b6a to 98c3fbb Compare March 11, 2022 18:17
Comment on lines +98 to +102
resource := pdata.NewResource()

for _, md := range mdata {

md.initializeResource(resource)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm understanding correctly, we have a problem here.

Since the generated code is hardcoded to work with one resource, I think it is currently unable to support scrapers that actually instantiate multiple resources. Here, we're intending to create a resource per process. I believe what is actually happening in this implementation is that we're only creating one process resource, and then overwriting its attributes with each subsequent process.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One solution would be to create another builder at the resource level, and move the Record... functions to that.

The scraper author can ask the MetricsBuilder for a new ResourceBuilder as needed, attach attributes and data points to it, and then ultimately call Emit on the MetricsBuilder. Since the MetricsBuilder created the ResourceBuilders, it can call emit on each of those in turn.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emit can be called with an option WithResource which copies the resource into the generate metrics, would that not work here?

https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/8382/files#diff-806ca51937857b079bdde5bf7e3e87aeb773245b2cb93b40078d241515bdab68R138-R142

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that WithResource and other parts of the generated code are all interacting with only the first resource in a slice (ResourceMetrics().At(0)).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah i see.. Yes the code is limited to a single resource for each call to Emit which means the only way to support multiple resources at this time is with separate calls to emit. Let me see what this looks like with a resource builder

Copy link
Member

@dmitryax dmitryax Mar 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking of a bit different approach to this problem that also solves #7301. Let me submit an alternative PR and we can continue the discussion then

@codeboten codeboten force-pushed the codeboten/refactor-metricbuilder branch from 2dc134d to f91414f Compare March 15, 2022 21:20
@github-actions
Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Mar 30, 2022
@codeboten
Copy link
Contributor Author

superseeded by #8555

@codeboten codeboten closed this Apr 6, 2022
@codeboten codeboten deleted the codeboten/refactor-metricbuilder branch April 6, 2022 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Metrics builder] Generate whole pdata.Metrics object
3 participants