Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OTel resource attribute promotion feature #8271

Merged
merged 3 commits into from
Nov 22, 2024
Merged

Conversation

aknuds1
Copy link
Contributor

@aknuds1 aknuds1 commented Jun 4, 2024

What this PR does

Add support for promoting OTel resource attributes.

The flag -distributor.promote-otel-resource-attributes (also a runtime config parameter) controls which set of resource attributes to eventually promote.

Additionally, OTelResourceAttributePromotionConfig can be set on the distributor configuration in order to specialize the configuration of which OTel resource attributes to promote. This is intended for our enterprise product.

Which issue(s) this PR fixes or relates to

Checklist

  • Tests updated.
  • Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • about-versioning.md updated with experimental features.

@aknuds1 aknuds1 added enhancement New feature or request area/opentelemetry Everything related to OpenTelemetry OTLP OTel labels Jun 4, 2024
@aknuds1 aknuds1 force-pushed the feat/promote-attributes branch 6 times, most recently from c9dc734 to 55055a3 Compare June 4, 2024 14:02
@ying-jeanne
Copy link
Contributor

Since we do label validation in validate.go validateLabels, check the number of label is less than the limit, if we convert resource attribute, we need to exclude those from numLabelNames

if numLabelNames > cfg.MaxLabelNamesPerSeries(userID) {
		m.maxLabelNamesPerSeries.WithLabelValues(userID, group).Inc()
		metric, ellipsis := getMetricAndEllipsis(ls)
		return fmt.Errorf(tooManyLabelsMsgFormat, len(ls), cfg.MaxLabelNamesPerSeries(userID), metric, ellipsis)
	}

@ying-jeanne
Copy link
Contributor

pushed a couple of tests on the blessing attributes, and a discussion about how to put lable limits

@aknuds1 aknuds1 force-pushed the feat/promote-attributes branch 6 times, most recently from 0b8e46e to 2ebad7f Compare June 10, 2024 14:44
@aknuds1
Copy link
Contributor Author

aknuds1 commented Jun 10, 2024

Since we do label validation in validate.go validateLabels, check the number of label is less than the limit, if we convert resource attribute, we need to exclude those from numLabelNames

@ying-jeanne we probably want to align with Loki, which includes the promoted attributes when comparing against the label count limit.

@aknuds1 aknuds1 force-pushed the feat/promote-attributes branch 5 times, most recently from 7cf8362 to fa127fd Compare June 11, 2024 15:49
@aknuds1 aknuds1 force-pushed the feat/promote-attributes branch from 9d6ddf4 to a6bbd83 Compare June 19, 2024 15:30
@dimitarvdimitrov
Copy link
Contributor

The CHANGELOG has just been cut to prepare for the next Mimir release. Please rebase main and eventually move the CHANGELOG entry added / updated in this PR to the top of the CHANGELOG document. Thanks!

@aknuds1 aknuds1 force-pushed the feat/promote-attributes branch from e97ad8a to c3235bf Compare October 21, 2024 16:20
@aknuds1 aknuds1 changed the title WIP: Add OTel resource attribute promotion feature Add OTel resource attribute promotion feature Oct 22, 2024
@aknuds1 aknuds1 force-pushed the feat/promote-attributes branch 3 times, most recently from 0773dd8 to 39fdd1e Compare November 21, 2024 15:03
@aknuds1 aknuds1 requested a review from jesusvazquez November 21, 2024 15:06
@aknuds1 aknuds1 force-pushed the feat/promote-attributes branch 4 times, most recently from aa08f36 to 03ffca5 Compare November 22, 2024 09:00
@aknuds1 aknuds1 marked this pull request as ready for review November 22, 2024 09:01
@aknuds1 aknuds1 requested review from tacole02 and a team as code owners November 22, 2024 09:01
@aknuds1 aknuds1 force-pushed the feat/promote-attributes branch from 03ffca5 to 6f43162 Compare November 22, 2024 09:03
Copy link
Member

@jesusvazquez jesusvazquez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Contributor

@56quarters 56quarters left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with some small changes that I'd like you to take a look at.

@aknuds1 aknuds1 enabled auto-merge (squash) November 22, 2024 15:28
@aknuds1 aknuds1 merged commit 1f2762d into main Nov 22, 2024
29 checks passed
@aknuds1 aknuds1 deleted the feat/promote-attributes branch November 22, 2024 15:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/opentelemetry Everything related to OpenTelemetry OTLP OTel enhancement New feature or request release/notified-changelog-cut
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants