[Perf] Linux/x64: 14 Regressions on 10/20/2023 2:39:28 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	72e69532d3dddd7894ffab37e356c01aea3a75a8
Compare	0151c4696eb14f52f567ceee91d2896ff2586edf
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
CreateInstance_5 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.63 μs	8.63 μs	2.38	0.09	False
CreateInstance_2 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	960.54 ns	5.75 μs	5.98	0.14	False
CreateInstance_4 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.09 μs	5.51 μs	5.04	0.18	False
CreateInstance_1 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	783.36 ns	4.15 μs	5.29	0.22	False
Factory_3Injected - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	553.63 ns	612.68 ns	1.11	0.14	False
CreateInstance_3 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	659.92 ns	1.80 μs	2.73	0.24	False
CreateInstance_0 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	460.43 ns	1.62 μs	3.52	0.20	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'

Payloads

Baseline
Compare

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_5

ETL Files

Histogram

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_2

ETL Files

Histogram

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_4

ETL Files

Histogram

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_1

ETL Files

Histogram

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory_3Injected

ETL Files

Histogram

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_3

ETL Files

Histogram

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_0

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	72e69532d3dddd7894ffab37e356c01aea3a75a8
Compare	0151c4696eb14f52f567ceee91d2896ff2586edf
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Diagnostics.Tracing.Perf_EventListener

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
WriteEvent_NoParams - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	31.53 ns	163.20 ns	5.18	0.27	False
WriteEvent_MixedParams - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	33.10 ns	358.43 ns	10.83	0.16	False
WriteEvent_IntParams - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	14.80 ns	254.04 ns	17.16	0.28	False
WriteEvent_StringParams - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	18.06 ns	480.71 ns	26.62	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Tracing.Perf_EventListener*'

Payloads

Baseline
Compare

System.Diagnostics.Tracing.Perf_EventListener.WriteEvent_NoParams

ETL Files

Histogram

JIT Disasms

System.Diagnostics.Tracing.Perf_EventListener.WriteEvent_MixedParams

ETL Files

Histogram

JIT Disasms

System.Diagnostics.Tracing.Perf_EventListener.WriteEvent_IntParams

ETL Files

Histogram

JIT Disasms

System.Diagnostics.Tracing.Perf_EventListener.WriteEvent_StringParams

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	72e69532d3dddd7894ffab37e356c01aea3a75a8
Compare	0151c4696eb14f52f567ceee91d2896ff2586edf
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Logging.EventSourceLogger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
NestedScopes_TwoMessages - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	915.26 ns	10.65 μs	11.64	0.10	False
NestedScopes_TwoMessages - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	944.81 ns	4.99 μs	5.28	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.EventSourceLogger*'

Payloads

Baseline
Compare

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: True, Json: True)

ETL Files

Histogram

JIT Disasms

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: True, Json: False)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	72e69532d3dddd7894ffab37e356c01aea3a75a8
Compare	0151c4696eb14f52f567ceee91d2896ff2586edf
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Benchmark.GetChildKeysTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
AddChainedConfigurationEmpty - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	29.87 ms	32.59 ms	1.09	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchmark.GetChildKeysTests*'

Payloads

Baseline
Compare

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[radekdoulik]

This looks like [wasm] Fix perf pipeline runs

@radical does it mean we were measuring NET8 performance before? do you know why were the results much faster before the fix?

[radical]

This looks like [wasm] Fix perf pipeline runs

Likely that in combination of bunch of others. The perf pipeline was in a broken state due to various reasons like the sdk/installer having a bad mix of 8.0/9.0 bits, linker issues causing microbenchmarks to fail.

@radical does it mean we were measuring NET8 performance before?

Yeah, possibly.

do you know why were the results much faster before the fix?

I'm not sure about that. Runs are not there for a some time before this because of the pipeline breaks.