Track dropped spans and logs due to full buffer

[scottgerring]

Fixes #2273

Design discussion issue (if applicable) #

Changes

As suggested in #2273, this change adds a counter and a metric for dropped spans and logs; the counter is used to log total drop count at exporter shutdown, the metric is used to export this information.

Merge requirement checklist

• CONTRIBUTING guidelines followed
• Unit tests added/updated (if applicable)
• Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
• Changes in public API reviewed (if applicable)

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: cijothomas / name: Cijo Thomas (d2e30cc, 6bf0e73, eb1f83f)
• ✅ login: scottgerring / name: Scott Gerring (b626d7f, 519555f, 6866a90, 8aaf156, f1d5142, 35614a9)

[codecov]

Codecov Report

Attention: Patch coverage is 71.79487% with 11 lines in your changes missing coverage. Please review.

Project coverage is 79.5%. Comparing base (195dea8) to head (eb1f83f).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
opentelemetry-sdk/src/trace/span_processor.rs	65.0%	7 Missing ⚠️
opentelemetry-sdk/src/logs/log_processor.rs	78.9%	4 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #2357     +/-   ##
=======================================
- Coverage   79.5%   79.5%   -0.1%     
=======================================
  Files        123     123             
  Lines      21448   21482     +34     
=======================================
+ Hits       17061   17087     +26     
- Misses      4387    4395      +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[cijothomas]

Thanks for working on this! Left my comments in the PR.
OTel internal metrics is something we can add, but not right away:
https://github.com/open-telemetry/opentelemetry-rust/pull/2357/files#r1860843464

[cijothomas]

odd that we have to store this here just for logging purposes, but not an issue!

[lalitb]

I believe we can avoid this by moving the logging of dropped logs (otel_warn!) from the shutdown() method to the worker's Shutdown message processing. Also, dropped_logs_count to be shared with the shutdown worker and the processor object. Haven't tried, but if it seems to be complex, we can park it for separate PR.

[cijothomas]

Thanks!
Left some suggestions for improved log messages.

[scottgerring]

Changes applied - should be good to go once the builds are done.
And - thanks for the quick review!

[cijothomas]

@lalitb @utpilla Could you take a look? If I get one more approval, I plan to include this for today's release (0.27.1)

[utpilla]

Left a suggestion to be consistent with the message content for both the batch processor.

[lalitb]

Would be good to include a unit test to validate that the dropped logs count is correctly reflected during shutdown. Since the field is private, we could consider adding a test-only accessor method to retrieve it. That said, if this is time-sensitive and needs to be included in today's release, can be done separately.

[cijothomas]

Would be good to include a unit test to validate that the dropped logs count is correctly reflected during shutdown. Since the field is private, we could consider adding a test-only accessor method to retrieve it. That said, if this is time-sensitive and needs to be included in today's release, can be done separately.

I think we have orchestrated such a test in OTel .NET, so we can copy some ideas.

[cijothomas]

@scottgerring I edited the PR Title ("chore: Track dropped spans and logs due to full buffer"), to remove "chore" from it.