[Logs] Serilog extensions project #3438
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <OutputType>Exe</OutputType> | ||
| <TargetFramework>net6.0</TargetFramework> | ||
| <ImplicitUsings>enable</ImplicitUsings> | ||
| <Nullable>enable</Nullable> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Exporter.Console\OpenTelemetry.Exporter.Console.csproj" /> | ||
| <ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Extensions.Serilog\OpenTelemetry.Extensions.Serilog.csproj" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| // <copyright file="Program.cs" company="OpenTelemetry Authors"> | ||
| // Copyright The OpenTelemetry Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
| // </copyright> | ||
|
|
||
| using OpenTelemetry.Logs; | ||
| using OpenTelemetry.Resources; | ||
| using Serilog; | ||
|
|
||
| var resourceBuilder = ResourceBuilder.CreateDefault().AddService("Examples.LogEmitter"); | ||
|
|
||
| // Note: It is important that OpenTelemetryLoggerProvider is disposed when the | ||
| // app is shutdown. In this example we allow Serilog to do that by calling CloseAndFlush. | ||
| var openTelemetryLoggerProvider = new OpenTelemetryLoggerProvider(options => | ||
| { | ||
| options.IncludeFormattedMessage = true; | ||
| options | ||
| .SetResourceBuilder(resourceBuilder) | ||
| .AddConsoleExporter(); | ||
| }); | ||
|
|
||
| // Configure Serilog global logger | ||
| Log.Logger = new LoggerConfiguration() | ||
| .WriteTo.OpenTelemetry(openTelemetryLoggerProvider, disposeProvider: true) // <- Register OpenTelemetry Serilog sink | ||
| .CreateLogger(); | ||
|
|
||
| // Note: Serilog ForContext API is used to set "CategoryName" on log messages | ||
| ILogger programLogger = Log.Logger.ForContext<Program>(); | ||
|
|
||
| programLogger.Information("Application started {Greeting} {Location}", "Hello", "World"); | ||
|
|
||
| programLogger.Information("Message {Array}", new string[] { "value1", "value2" }); | ||
|
|
||
| // Note: For Serilog this call flushes all logs and disposes | ||
| // OpenTelemetryLoggerProvider. | ||
| Log.CloseAndFlush(); | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # OpenTelemetry Logging Extensions Example | ||
|
|
||
| This project contains examples of the `LogEmitter` API being used to extend | ||
| existing logging platforms to write into OpenTelemetry logs. | ||
|
|
||
| * Serilog: Using OpenTelemetry.Extensions.Serilog | ||
|
|
||
| ## References | ||
|
|
||
| * [OpenTelemetry Project](https://opentelemetry.io/) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| #nullable enable |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| #nullable enable | ||
| Serilog.OpenTelemetrySerilogExtensions | ||
| static Serilog.OpenTelemetrySerilogExtensions.OpenTelemetry(this Serilog.Configuration.LoggerSinkConfiguration! loggerConfiguration, OpenTelemetry.Logs.OpenTelemetryLoggerProvider! openTelemetryLoggerProvider, bool disposeProvider = true) -> Serilog.LoggerConfiguration! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| // <copyright file="AssemblyInfo.cs" company="OpenTelemetry Authors"> | ||
| // Copyright The OpenTelemetry Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
| // </copyright> | ||
|
|
||
| using System; | ||
| using System.Runtime.CompilerServices; | ||
|
|
||
| [assembly: CLSCompliant(false)] | ||
| [assembly: InternalsVisibleTo("DynamicProxyGenAssembly2" + AssemblyInfo.MoqPublicKey)] | ||
|
|
||
| #if SIGNED | ||
| internal static class AssemblyInfo | ||
| { | ||
| public const string PublicKey = ", PublicKey=002400000480000094000000060200000024000052534131000400000100010051C1562A090FB0C9F391012A32198B5E5D9A60E9B80FA2D7B434C9E5CCB7259BD606E66F9660676AFC6692B8CDC6793D190904551D2103B7B22FA636DCBB8208839785BA402EA08FC00C8F1500CCEF28BBF599AA64FFB1E1D5DC1BF3420A3777BADFE697856E9D52070A50C3EA5821C80BEF17CA3ACFFA28F89DD413F096F898"; | ||
| public const string MoqPublicKey = ", PublicKey=0024000004800000940000000602000000240000525341310004000001000100c547cac37abd99c8db225ef2f6c8a3602f3b3606cc9891605d02baa56104f4cfc0734aa39b93bf7852f7d9266654753cc297e7d2edfe0bac1cdcf9f717241550e0a7b191195b7667bb4f64bcb8e2121380fd1d9d46ad2d92d2d15605093924cceaf74c4861eff62abf69b9291ed0a340e113be11e6a7d3113e92484cf7045cc7"; | ||
| } | ||
| #else | ||
| internal static class AssemblyInfo | ||
| { | ||
| public const string PublicKey = ""; | ||
| public const string MoqPublicKey = ""; | ||
| } | ||
| #endif |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| # Changelog | ||
|
|
||
| ## Unreleased | ||
|
|
||
| Initial release. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
| <PropertyGroup> | ||
| <!-- OmniSharp/VS Code requires TargetFrameworks to be in descending order for IntelliSense and analysis. --> | ||
| <TargetFrameworks>netstandard2.0</TargetFrameworks> | ||
| <Description>Extensions to enable OpenTelemetry logging when using the Serilog library</Description> | ||
| <PackageTags>$(PackageTags);serilog;logging</PackageTags> | ||
| <Nullable>enable</Nullable> | ||
| <AnalysisMode>AllEnabledByDefault</AnalysisMode> | ||
| <AnalysisLevel>latest</AnalysisLevel> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="$(RepoRoot)\src\OpenTelemetry\OpenTelemetry.csproj" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <Compile Include="$(RepoRoot)\src\OpenTelemetry.Api\Internal\Guard.cs" Link="Includes\Guard.cs" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="Serilog" Version="$(SerilogPkgVer)" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| // <copyright file="OpenTelemetrySerilogExtensions.cs" company="OpenTelemetry Authors"> | ||
| // Copyright The OpenTelemetry Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
| // </copyright> | ||
|
|
||
| using OpenTelemetry.Internal; | ||
| using OpenTelemetry.Logs; | ||
| using Serilog.Configuration; | ||
|
|
||
| namespace Serilog | ||
| { | ||
| /// <summary> | ||
| /// Contains Serilog extension methods. | ||
| /// </summary> | ||
| public static class OpenTelemetrySerilogExtensions | ||
| { | ||
| /// <summary> | ||
| /// Adds a sink to Serilog <see cref="LoggerConfiguration"/> which will | ||
| /// write to OpenTelemetry. | ||
| /// </summary> | ||
| /// <param name="loggerConfiguration"><see | ||
| /// cref="LoggerSinkConfiguration"/>.</param> | ||
| /// <param name="openTelemetryLoggerProvider"><see | ||
| /// cref="OpenTelemetryLoggerProvider"/>.</param> | ||
| /// <param name="disposeProvider">Controls whether or not the supplied | ||
| /// <paramref name="openTelemetryLoggerProvider"/> will be disposed when | ||
| /// the logger is disposed. Default value: <see | ||
| /// langword="true"/>.</param> | ||
| /// <returns>Supplied <see cref="LoggerConfiguration"/> for chaining calls.</returns> | ||
| public static LoggerConfiguration OpenTelemetry( | ||
| this LoggerSinkConfiguration loggerConfiguration, | ||
| OpenTelemetryLoggerProvider openTelemetryLoggerProvider, | ||
| bool disposeProvider = true) | ||
| { | ||
| Guard.ThrowIfNull(loggerConfiguration); | ||
| Guard.ThrowIfNull(openTelemetryLoggerProvider); | ||
|
|
||
| #pragma warning disable CA2000 // Dispose objects before losing scope | ||
| return loggerConfiguration.Sink(new OpenTelemetrySerilogSink(openTelemetryLoggerProvider, disposeProvider)); | ||
| #pragma warning restore CA2000 // Dispose objects before losing scope | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| // <copyright file="OpenTelemetrySerilogSink.cs" company="OpenTelemetry Authors"> | ||
| // Copyright The OpenTelemetry Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
| // </copyright> | ||
|
|
||
| using System; | ||
| using System.Collections.Generic; | ||
| using System.Diagnostics; | ||
| using Microsoft.Extensions.Logging; | ||
| using Serilog.Core; | ||
| using Serilog.Events; | ||
|
|
||
| namespace OpenTelemetry.Logs | ||
| { | ||
| internal sealed class OpenTelemetrySerilogSink : ILogEventSink, IDisposable | ||
| { | ||
| private readonly OpenTelemetryLoggerProvider openTelemetryLoggerProvider; | ||
| private readonly bool includeFormattedMessage; | ||
| private readonly LogEmitter logEmitter; | ||
| private readonly bool disposeProvider; | ||
|
|
||
| public OpenTelemetrySerilogSink(OpenTelemetryLoggerProvider openTelemetryLoggerProvider, bool disposeProvider) | ||
cijothomas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| { | ||
| Debug.Assert(openTelemetryLoggerProvider != null, "openTelemetryLoggerProvider was null"); | ||
|
|
||
| this.openTelemetryLoggerProvider = openTelemetryLoggerProvider!; | ||
| this.disposeProvider = disposeProvider; | ||
|
|
||
| var logEmitter = this.openTelemetryLoggerProvider.CreateEmitter(); | ||
| Debug.Assert(logEmitter != null, "logEmitter was null"); | ||
|
|
||
| this.logEmitter = logEmitter!; | ||
|
|
||
| // TODO: This project can only access IncludeFormattedMessage | ||
| // because it can see SDK internals. At some point this is likely | ||
| // not to be the case. Need to figure out where to put | ||
| // IncludeFormattedMessage so that extensions can see it. Ideas: | ||
| // Make it public on OpenTelemetryLoggerProvider or expose it on | ||
| // LogEmitter instance. | ||
| this.includeFormattedMessage = this.openTelemetryLoggerProvider.IncludeFormattedMessage; | ||
| } | ||
|
|
||
| public void Emit(LogEvent logEvent) | ||
cijothomas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| { | ||
| Debug.Assert(logEvent != null, "LogEvent was null."); | ||
|
|
||
| LogRecordData data = new(Activity.Current) | ||
| { | ||
| Timestamp = logEvent!.Timestamp.UtcDateTime, | ||
| LogLevel = (LogLevel)(int)logEvent.Level, | ||
| Message = this.includeFormattedMessage ? logEvent.RenderMessage() : logEvent.MessageTemplate.Text, | ||
| Exception = logEvent.Exception, | ||
| }; | ||
|
|
||
| LogRecordAttributeList attributes = default; | ||
| foreach (KeyValuePair<string, LogEventPropertyValue> property in logEvent.Properties) | ||
| { | ||
| // TODO: Serilog supports complex type logging. This is not yet | ||
| // supported in OpenTelemetry. | ||
| if (property.Key == Constants.SourceContextPropertyName | ||
| && property.Value is ScalarValue sourceContextValue) | ||
| { | ||
| data.CategoryName = sourceContextValue.Value as string; | ||
| } | ||
| else if (property.Value is ScalarValue scalarValue) | ||
| { | ||
| attributes.Add(property.Key, scalarValue.Value); | ||
| } | ||
| else if (property.Value is SequenceValue sequenceValue) | ||
| { | ||
| IReadOnlyList<LogEventPropertyValue> elements = sequenceValue.Elements; | ||
| if (elements.Count > 0) | ||
| { | ||
| // Note: The goal here is to build a typed array (eg | ||
| // int[]) if all the element types match otherwise | ||
| // fallback to object[] | ||
|
Comment on lines
+85
to
+87
There was a problem hiding this comment. @alanwest It was a bit tricky, but we now build a typed array when we can. There was a problem hiding this comment. Interesting, yea looks like this does the thing. Going out on a limb here, but I'd suspect more often than not it'll all be of one type. So, most folks won't hit the perf penalty. There was a problem hiding this comment. I didn't stress the perf too much. For two reasons. The first is I'm assuming logging arrays is kind of a rare thing? The second is Serilog is already destroying perf 🤣 Whatever you log it allocates/wraps/boxes into a spiderweb of reference types. Joined the darkside Serilog has. |
||
|
|
||
| Type? elementType = null; | ||
| Array? values = null; | ||
|
|
||
| for (int i = 0; i < elements.Count; i++) | ||
| { | ||
| if (elements[i] is ScalarValue value) | ||
| { | ||
| Type currentElementType = value.Value?.GetType() ?? typeof(object); | ||
|
There was a problem hiding this comment. If this were of type We don't do anything special earlier in the pipeline for attributes on traces or metrics, but is there any reason we'd want to consider coercing things into one of the primitive types earlier for log data? There was a problem hiding this comment. Not totally following you. But I don't think a complex type would end up as a I have this test at the moment: That is verifying a "destructed" complex type is currently dropped on the floor, because we don't yet have support for that. What happens in that case is instead of a Right now we have basic support. You can use Serilog to do structured log messaging with primitives + arrays of primitives and it will work into OTel. My goal is to get support into OTel somehow for complex types and then come back to this and properly support fully Serilog's destructuring mechanism 😄 There was a problem hiding this comment. Ah thanks I missed that aspect of the test. Definitely seems like a bridge to cross later once complex types are supported. If serilog does all the work then my question is moot. I wasn't sure if a complex type could end up being added to the attributes of a log record. |
||
|
|
||
| if (values == null) | ||
| { | ||
| elementType = currentElementType; | ||
| values = Array.CreateInstance(elementType, elements.Count); | ||
| } | ||
| else if (!elementType!.IsAssignableFrom(currentElementType)) | ||
| { | ||
| // Array with mixed types detected | ||
| object[] newValues = new object[elements.Count]; | ||
| values.CopyTo(newValues, 0); | ||
| values = newValues; | ||
| elementType = typeof(object); | ||
| } | ||
|
|
||
| values.SetValue(value.Value, i); | ||
| } | ||
| } | ||
|
|
||
| if (values != null) | ||
| { | ||
| attributes.Add(property.Key, values); | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| this.logEmitter.Emit(in data, in attributes); | ||
| } | ||
|
|
||
| public void Dispose() | ||
| { | ||
| if (this.disposeProvider) | ||
| { | ||
| this.openTelemetryLoggerProvider.Dispose(); | ||
| } | ||
| } | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of assigning this to a variable
openTelemetryLoggerProviderand explaining the object ownership/lifecycle, I suggest putting thenew OpenTelemetryLoggerProviderexpression directly in Ln:35WriteTo.OpenTelemetry(new OpenTelemetryLoggerProvider(...), disposeProvider: true): it increases the locality when folks read the code, making the code more self-contained rather than highly contextual, it avoids accidental misuse (e.g. code holding a reference to disposed object).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here is the version from the other PR: https://github.com/open-telemetry/opentelemetry-dotnet/pull/3305/files#diff-eebf3c37db933ef5eddad43dc03d2553b800b073573f291b7e0ac5999a1391c8
Where I am heading with this is two different types of extensions sharing a provider.
So the question is, what do we want to demo in the example? Sharing a provider or independent providers? I feel like shared will be the more common case. But LMK your thoughts on this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here is my take:
examples/LoggingExtensions/Program.cslooks an example for the end-user - who is using the OpenTelemetry.Extensions.Serilog package to pipe Serilog via OpenTelemetry plumbing. For these users, normally they will only use Serilog in a single application, it makes more sense for them to have something simple, and not to worry about object lifecycle management except for the top levelCloseAndFlush.examples/LogEmitter/Program.csis more targeting the plumbers, might make sense to expose more complexity / duty.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@reyang Let me merge and then I'll follow-up on this on my next PR where the example project is going to get more interesting?