Simplified stdout exporter for logs and traces

[cijothomas]

The current implementation of stdout exporter is not very easy to read by default or even with optional json formatting. Modified the stdout exporter implementation to print logs,traces in an easier to read format. (Of course, this is subjective!). I tried to mimic OTel Collector Debug exporters detailed output format. closely. If everyone feels this is better, I'll continue and extend this for Metrics too, and fix up any gaps. (i.e ensure every field is exported etc.) This can also eliminate the transformation code we maintain for stdout exporter. Update after initial PR review: All signals are covered now.

Note that the output format is not fixed and users are already advised not to rely on any particular format. Stdout exporter is ONLY designed/meant for learning purposes and not any other scenarios.

Example Logs output
Before:

{"resourceLogs":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"basic-stdout-example"}},{"key":"telemetry.sdk.name","value":{"stringValue":"opentelemetry"}},{"key":"telemetry.sdk.language","value":{"stringValue":"rust"}},{"key":"telemetry.sdk.version","value":{"stringValue":"0.24.1"}}]},"scopeLogs":[{"scope":{"name":"opentelemetry-appender-tracing","version":"0.5.0"},"logRecords":[{"timeUnixNano":null,"time":null,"observedTimeUnixNano":1724181491423171133,"observedTime":"2024-08-20 19:18:11.423","severityNumber":17,"severityText":"ERROR","attributes":[{"key":"event_id","value":{"intValue":20}},{"key":"user_name","value":{"stringValue":"otel"}},{"key":"user_email","value":{"stringValue":"[email protected]"}}],"droppedAttributesCount":0}]}]}]}

With this PR

Resource
         telemetry.sdk.version=String(Static("0.24.1"))
         telemetry.sdk.name=String(Static("opentelemetry"))
         telemetry.sdk.language=String(Static("rust"))
         service.name=String(Static("basic-stdout-example"))
Log #0
         EventName: "my-event-name"
         Target (Scope): "my-system"
         Observed Timestamp: 2024-08-20 19:10:07.275165
         SeverityText: "ERROR"
         SeverityNumber: Error
         Attributes:
                 event_id: Int(20)
                 user_name: String(Owned("otel"))
                 user_email: String(Owned("[email protected]"))

Example Span output
Before:

{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"basic-stdout-example"}},{"key":"telemetry.sdk.name","value":{"stringValue":"opentelemetry"}},{"key":"telemetry.sdk.language","value":{"stringValue":"rust"}},{"key":"telemetry.sdk.version","value":{"stringValue":"0.24.1"}}]},"scopeSpans":[{"scope":{"name":"stdout-test"},"spans":[{"traceId":"ef2090ff07115b5e7cc26949402d5032","spanId":"0783277ce34d8f0c","parentSpanId":"0000000000000000","name":"test_span","kind":1,"startTimeUnixNano":1724181491423412359,"startTime":"2024-08-20 19:18:11.423","endTimeUnixNano":1724181491423418960,"endTime":"2024-08-20 19:18:11.423","attributes":[{"key":"test_key","value":{"stringValue":"test_value"}}],"droppedAttributesCount":0,"events":[{"name":"test_event","attributes":[{"key":"test_event_key","value":{"stringValue":"test_event_value"}}],"droppedAttributesCount":0,"timeUnixNano":1724181491423417485,"time":"2024-08-20 19:18:11.423"}],"droppedEventsCount":0,"flags":1,"droppedLinksCount":0,"status":{}}]}]}]}

With this PR

Resource
         telemetry.sdk.version=String(Static("0.24.1"))
         telemetry.sdk.name=String(Static("opentelemetry"))
         telemetry.sdk.language=String(Static("rust"))
         service.name=String(Static("basic-stdout-example"))
Span #0
         Instrumentation Scope
                 Name: "stdout-example"
                 Version: "v1."
                 SchemaUrl: "schema_url"
                 Scope Attributes:
                         scope_key: String(Static("scope_value"))

         Name: "test_span"
         TraceId: f8d3787cde7f1f976bf0ec2e31cd4ebc
         SpanId: 3688ee24904752ad
         ParentSpanId: 0000000000000000
         Kind: Internal
         Start time: 2024-08-20 19:10:07.275416
         End time: 2024-08-20 19:10:07.275421
         Status: Unset
         Attributes:
         Dropped attributes count: 0
                 test_attribute_key: String(Static("test_attribute_value"))
         Events:
                 Name: "test_event"
                 Timestamp: 2024-08-20 19:10:07.275420
                 Attributes:
                         test_event_key: String(Static("test_event_value"))

Metrics output with this PR

Metrics
Resource
         ->  service.name=String(Static("basic-stdout-example"))
         ->  telemetry.sdk.name=String(Static("opentelemetry"))
         ->  telemetry.sdk.version=String(Static("0.24.1"))
         ->  telemetry.sdk.language=String(Static("rust"))
        Instrumentation Scope #0
                Name         : stdout-example
Metric #0
                Name         : example_counter
                Description  : 
                Unit         : 
                Type         : Sum
                Sum DataPoints
                Monotonic    : true
                DataPoint #0
                        Value        : 2
                        Attributes   :
                                 ->  name: apple
                                 ->  color: green
                DataPoint #1
                        Value        : 2
                        Attributes   :
                                 ->  name: apple
                                 ->  color: red
                DataPoint #2
                        Value        : 12
                        Attributes   :
                                 ->  name: banana
                                 ->  color: yellow

[cijothomas]

I'll leave this open for a day or two, to make sure others have a chance to share feedback.
(There might be some merge conflicts with other in-flight Logs PR, but this PR is easy to resolve them, so this can go last anyway)

[lalitb]

Thanks for the effort on simplifying the output. I agree the existing json format (even with serde pretty-print) is not easy read.
I have another suggestion. If we can keep the current design (transform, encoder, writer) intact, and instead provide a a built-in encoder that formats logs/spans in a human-readable way (as in this PR) and make it the default option in the builder. This way, we can maintain the existing flexibility provided by the current design by abstracting the writing and encoding process. I understand this is not for production, and most of the users won't care of the existing flexibility, but preserving ensures the advance use-cases, without much of maintenance effort.

[cijothomas]

Thanks for the effort on simplifying the output. I agree the existing json format (even with serde pretty-print) is not easy read. I have another suggestion. If we can keep the current design (transform, encoder, writer) intact, and instead provide a a built-in encoder that formats logs/spans in a human-readable way (as in this PR) and make it the default option in the builder. This way, we can maintain the existing flexibility provided by the current design by abstracting the writing and encoding process. I understand this is not for production, and most of the users won't care of the existing flexibility, but preserving ensures the advance use-cases, without much of maintenance effort.

I was initially thinking of keeping existing and providing an option to do the current output format... Then I felt that it'd possibly convey wrong intent that we intend to keep such flexibility! Removing all such flexibility, makes our intent pretty clear - this is only for learning/testing purpose and nothing more.

[lalitb]

Thanks for the effort on simplifying the output. I agree the existing json format (even with serde pretty-print) is not easy read. I have another suggestion. If we can keep the current design (transform, encoder, writer) intact, and instead provide a a built-in encoder that formats logs/spans in a human-readable way (as in this PR) and make it the default option in the builder. This way, we can maintain the existing flexibility provided by the current design by abstracting the writing and encoding process. I understand this is not for production, and most of the users won't care of the existing flexibility, but preserving ensures the advance use-cases, without much of maintenance effort.

I was initially thinking of keeping existing and providing an option to do the current output format... Then I felt that it'd possibly convey wrong intent that we intend to keep such flexibility! Removing all such flexibility, makes our intent pretty clear - this is only for learning/testing purpose and nothing more.

I see your point in potential confusion coming with the flexibility provided. We can also have a balance by documenting that the existing flexibility doesn’t make the current exporter suitable for any production use-case. No strong opinion though, i eventually go with what all agree :)

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 311 lines in your changes missing coverage. Please review.

Project coverage is 77.9%. Comparing base (322c985) to head (c8dcc6b).
Report is 1 commits behind head on main.

Files	Patch %	Lines
opentelemetry-stdout/src/metrics/exporter.rs	0.0%	152 Missing ⚠️
opentelemetry-stdout/src/trace/exporter.rs	0.0%	97 Missing ⚠️
opentelemetry-stdout/src/logs/exporter.rs	0.0%	62 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #2040     +/-   ##
=======================================
+ Coverage   77.3%   77.9%   +0.5%     
=======================================
  Files        124     121      -3     
  Lines      21282   21136    -146     
=======================================
  Hits       16472   16472             
+ Misses      4810    4664    -146     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[cijothomas]

Given there is general acceptance to move towards this PR, I'll continue polishing this (and add metrics too). Marking draft while I work through them and address review feedbacks.

[cijothomas]

Merging now, given there is general agreement for this direction. Happy to address more feedback in follow ups.

[aochagavia]

Hello there! Is there any suggested crate for people like me who were relying on the older behavior against all warnings? I was using opentelemetry-stdout to store a program run's traces as a JSON file, then loading it into Jaeger for inspection. I understand that you can use an exporter that talks to Jaeger directly, but I'm using this for debugging / profiling and it's useful to have the results in a JSON file.

Update: I ended up using Jaeger's ability to save a trace as a JSON file, which is good enough for my use case.

[cijothomas]

Hello there! Is there any suggested crate for people like me who were relying on the older behavior against all warnings? I was using opentelemetry-stdout to store a program run's traces as a JSON file, then loading it into Jaeger for inspection. I understand that you can use an exporter that talks to Jaeger directly, but I'm using this for debugging / profiling and it's useful to have the results in a JSON file.

Based on open-telemetry/opentelemetry-specification#4183 we can add a new exporter that'll output OTLP as json, and this time without any "don't use for prod" warning!
No timelines yet, but we are happy to get contributions for the same.