Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Changes at 7/6/2022 11:58:32 AM #6768

Closed
performanceautofiler bot opened this issue Jul 14, 2022 · 13 comments
Closed

[Perf] Changes at 7/6/2022 11:58:32 AM #6768

performanceautofiler bot opened this issue Jul 14, 2022 · 13 comments
Assignees
@tannergooding
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro os-linux perf-improvement PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-coreclr

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jul 14, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Join_Enumerable - Duration of single invocation 653.40 ns 333.91 ns 0.51 0.17 False
ToUpperInvariant - Duration of single invocation 57.48 ns 19.78 ns 0.34 0.16 False
ToLower - Duration of single invocation 130.95 ns 94.83 ns 0.72 0.05 False
Join_List - Duration of single invocation 158.77 ns 78.26 ns 0.49 0.03 False
ToLower - Duration of single invocation 63.53 ns 30.39 ns 0.48 0.11 False
ToLowerInvariant - Duration of single invocation 54.30 ns 20.45 ns 0.38 0.19 False
ToLowerInvariant - Duration of single invocation 54.93 ns 20.43 ns 0.37 0.19 False
Join_Array - Duration of single invocation 130.02 ns 73.53 ns 0.57 0.03 False
TrimEnd - Duration of single invocation 39.88 ns 16.30 ns 0.41 0.32 False
ToLower - Duration of single invocation 65.67 ns 30.59 ns 0.47 0.07 False
ToUpperInvariant - Duration of single invocation 120.35 ns 83.66 ns 0.70 0.08 False
ToLowerInvariant - Duration of single invocation 120.16 ns 83.52 ns 0.70 0.07 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Join_Enumerable

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 333.9072620074223 < 623.7721058774449.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 18.39681126448772 (T) = (0 -337.007138172066) / Math.Sqrt((3756.7589035735923 / (283)) + (29.81654583945833 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.17024461974199068 = (406.15239887600956 - 337.007138172066) / 406.15239887600956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "test")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.782004299615515 < 54.08922689898475.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 12.625792123874094 (T) = (0 -20.264307652610814) / Math.Sqrt((45.239788075819895 / (283)) + (0.21430480593568896 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.20244853663194287 = (25.408150549977943 - 20.264307652610814) / 25.408150549977943 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 94.82964681651482 < 125.55030587021281.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 38.550112433435416 (T) = (0 -94.54401471693298) / Math.Sqrt((103.76388905074424 / (284)) + (0.4924120241298504 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.2007452839516677 = (118.29021814769716 - 94.54401471693298) / 118.29021814769716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Join_List

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 78.25578733163074 < 151.03846347997234.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 22.074419942035153 (T) = (0 -78.87134227048571) / Math.Sqrt((210.66821880467242 / (283)) + (0.6641175198461 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.19648739776411078 = (98.1581894932511 - 78.87134227048571) / 98.1581894932511 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "TEST")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.389351006912445 < 63.03714167342331.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 12.940834104936117 (T) = (0 -30.654494828191233) / Math.Sqrt((42.19592962885175 / (284)) + (0.43722945463564766 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.14487810895607622 = (35.848099726190554 - 30.654494828191233) / 35.848099726190554 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "TEST")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.445663891143717 < 52.066405693332655.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 12.454475761125858 (T) = (0 -20.208970095677035) / Math.Sqrt((43.21160011474982 / (284)) + (0.2834081786850016 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.19788692582233705 = (25.194664874893782 - 20.208970095677035) / 25.194664874893782 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "TeSt")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.429994861331593 < 51.89073373535458.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 12.255816283235802 (T) = (0 -20.269258998741094) / Math.Sqrt((44.01215288790112 / (284)) + (0.26413718596912045 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.19598425760610408 = (25.21002752805674 - 20.269258998741094) / 25.21002752805674 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Join_Array

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.53201039369002 < 124.25867043760647.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 27.39272050277857 (T) = (0 -72.96918488357582) / Math.Sqrt((115.85069532602093 / (283)) + (0.5534794471528437 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.19664745012327883 = (90.83083746329471 - 72.96918488357582) / 90.83083746329471 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimEnd(s: "Test ")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.299595129364306 < 38.664034711270276.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 17.026332883990246 (T) = (0 -16.712878642938467) / Math.Sqrt((22.55329807831837 / (283)) + (0.24409298043404948 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.23071907624506288 = (21.72532572543361 - 16.712878642938467) / 21.72532572543361 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "TeSt")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.58885162206132 < 62.35866669641114.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 12.960440760322056 (T) = (0 -30.743190660298314) / Math.Sqrt((41.435964219766 / (283)) + (0.39004898450103687 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.14335029549300785 = (35.887703571894924 - 30.743190660298314) / 35.887703571894924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 83.6578049868783 < 113.87045878917574.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 39.106163524565225 (T) = (0 -83.59276789166339) / Math.Sqrt((103.87038774787617 / (283)) + (0.317421354556362 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.2229384399232639 = (107.57547688166233 - 83.59276789166339) / 107.57547688166233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 83.52185406199922 < 114.01059304601556.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 39.13291169411014 (T) = (0 -83.69276307003122) / Math.Sqrt((99.1127453133418 / (284)) + (0.29815919069644314 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.2184898952519759 = (107.09108245889615 - 83.69276307003122) / 107.09108245889615 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Memory.MemoryMarshal<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CastToInt - Duration of single invocation 14.99 ns 0.25 ns 0.02 0.78 False
AsBytes - Duration of single invocation 15.40 ns 0.62 ns 0.04 0.65 False
CastToByte - Duration of single invocation 15.73 ns 0.45 ns 0.03 0.68 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.MemoryMarshal&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.MemoryMarshal<Int32>.CastToInt

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.245100900835234 < 14.246467817963541.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.5898549466857976 (T) = (0 -0.2239969521001966) / Math.Sqrt((8.747708693355829 / (284)) + (0.00289575495272233 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.73797782687585 = (0.8548778503339236 - 0.2239969521001966) / 0.8548778503339236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.MemoryMarshal&lt;Int32&gt;.AsBytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.6189854337406197 < 14.950121307252713.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.886838754041036 (T) = (0 -0.5885285427532173) / Math.Sqrt((8.763463790133104 / (284)) + (0.0016916117424943483 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.6374792536136435 = (1.6234341030678365 - 0.5885285427532173) / 1.6234341030678365 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.MemoryMarshal&lt;Int32&gt;.CastToByte

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.4481353045015596 < 15.236663814436225.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 6.744578516563523 (T) = (0 -0.4144010857852918) / Math.Sqrt((8.98670261572162 / (284)) + (0.001919294340864822 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.7434373440161772 = (1.6152042244660163 - 0.4144010857852918) / 1.6152042244660163 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor - Duration of single invocation 59.56 μs 37.86 μs 0.64 0.32 False
Ctor - Duration of single invocation 41.11 μs 29.16 μs 0.71 0.25 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:?[^\s#])?(?:#[^\s])?", Options: Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.862508114998235 < 56.44584357383566.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 20.525917903686064 (T) = (0 -39248.62403692026) / Math.Sqrt((16958217.251173552 / (284)) + (1591521.029595961 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.14501462579544258 = (45905.6087052197 - 39248.62403692026) / 45905.6087052197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.164148769901512 < 38.7753500529256.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 20.196204875430716 (T) = (0 -29492.31207384367) / Math.Sqrt((4878441.6909913365 / (283)) + (545986.6939373435 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.11039424077990546 = (33152.11459478318 - 29492.31207384367) / 33152.11459478318 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ModPow - Duration of single invocation 5.99 ms 2.56 ms 0.43 0.01 False
ToStringD - Duration of single invocation 68.64 ns 41.57 ns 0.61 0.19 False
ModPow - Duration of single invocation 348.79 μs 133.10 μs 0.38 0.03 False
ToStringX - Duration of single invocation 101.75 ns 57.52 ns 0.57 0.05 False
Parse - Duration of single invocation 167.39 ns 151.01 ns 0.90 0.08 False
ToStringX - Duration of single invocation 111.15 ns 68.26 ns 0.61 0.01 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.ModPow(arguments: 16384,16384,64 bits)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.557793114874815 < 5.746440592549362.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.7682488367836955 (T) = (0 -2525427.2111348067) / Math.Sqrt((509822189553.914 / (283)) + (267669021.86147717 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.059678398719712 = (2685705.8347870875 - 2525427.2111348067) / 2685705.8347870875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 123)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.56894919810358 < 65.58233974623856.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 16.479994479239057 (T) = (0 -42.19147938457414) / Math.Sqrt((25.405396483936084 / (284)) + (0.6307401773574294 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.11351679527318496 = (47.59422305984484 - 42.19147938457414) / 47.59422305984484 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ModPow(arguments: 1024,1024,64 bits)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 133.0997433262712 < 332.3701098975695.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.949718371660405 (T) = (0 -133863.07280900772) / Math.Sqrt((1854480250.3715558 / (283)) + (1237274.3653873 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.07040225534896835 = (144001.07313003382 - 133863.07280900772) / 144001.07313003382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 123)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 57.52259554689382 < 96.69256939906028.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 12.942525350298009 (T) = (0 -58.07086367009216) / Math.Sqrt((66.49150261158395 / (284)) + (0.6262921900148791 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.10062757581429782 = (64.56820568261242 - 58.07086367009216) / 64.56820568261242 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 123)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 151.011264698202 < 160.7407650136913.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 15.736189377527518 (T) = (0 -149.0808542165411) / Math.Sqrt((54.701601812296765 / (284)) + (3.6975517478391375 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.05450439528175778 = (157.67482521610157 - 149.0808542165411) / 157.67482521610157 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: -2147483648)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.26097130763263 < 105.59222622813036.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 14.437303977048229 (T) = (0 -68.76607134037515) / Math.Sqrt((61.256627066621725 / (284)) + (0.3576962605392567 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.09073450621078104 = (75.62815460400186 - 68.76607134037515) / 75.62815460400186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 14, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 16.97 μs 11.48 μs 0.68 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeObjectProperty

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.476344363475258 < 16.042943826697265.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 20.67283687030887 (T) = (0 -11109.561103214008) / Math.Sqrt((1067280.1449969034 / (283)) + (30085.677565362046 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.11239696760385882 = (12516.362267513932 - 11109.561103214008) / 12516.362267513932 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Formats.Cbor.Tests.Perf_CborReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SkipValue - Duration of single invocation 1.70 μs 716.49 ns 0.42 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Formats.Cbor.Tests.Perf_CborReader*'

Payloads

Baseline
Compare

Histogram

System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Map, Canonical))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 716.4948433873269 < 1.6080502764565772.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 11.287953938345547 (T) = (0 -721.1593580158118) / Math.Sqrt((33472.09391087258 / (283)) + (56.71683379546988 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.14631382428908013 = (844.759325539336 - 721.1593580158118) / 844.759325539336 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Net.NetworkInformation.Tests.PhysicalAddressTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
PAMedium - Duration of single invocation 58.59 ns 26.95 ns 0.46 0.34 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.NetworkInformation.Tests.PhysicalAddressTests*'

Payloads

Baseline
Compare

Histogram

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PAMedium

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.94733956288362 < 55.683363520873.
IsChangePoint: Marked as a change because one of 6/22/2022 11:10:17 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 10.518552760839952 (T) = (0 -27.857267400035948) / Math.Sqrt((34.50052887162223 / (284)) + (3.014541987357883 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.14679046348831398 = (32.64997190951393 - 27.857267400035948) / 32.64997190951393 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 14, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsPrefix_FirstHalf - Duration of single invocation 2.04 μs 275.02 ns 0.13 0.02 False
IsSuffix_SecondHalf - Duration of single invocation 2.03 μs 275.84 ns 0.14 0.02 False
IndexOf_Word_NotFound - Duration of single invocation 2.23 μs 565.06 ns 0.25 0.02 False
IsPrefix_FirstHalf - Duration of single invocation 2.03 μs 230.48 ns 0.11 0.02 False
IsPrefix_FirstHalf - Duration of single invocation 2.03 μs 275.53 ns 0.14 0.01 False
IsSuffix_SecondHalf - Duration of single invocation 2.04 μs 275.71 ns 0.13 0.01 False
IsPrefix_FirstHalf - Duration of single invocation 1.93 μs 229.94 ns 0.12 0.01 False
IsSuffix_SecondHalf - Duration of single invocation 2.04 μs 230.29 ns 0.11 0.02 False
IsPrefix_FirstHalf - Duration of single invocation 51.07 ns 24.61 ns 0.48 0.06 False
IndexOf_Word_NotFound - Duration of single invocation 2.26 μs 565.19 ns 0.25 0.02 False
IsSuffix_SecondHalf - Duration of single invocation 2.04 μs 230.34 ns 0.11 0.02 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreCase, False))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 275.0220511895178 < 1.8813601638831396.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.580767058235799 (T) = (0 -275.3954461661248) / Math.Sqrt((122124.40173643001 / (284)) + (0.08827753428797547 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.21236681314219955 = (349.6493682100838 - 275.3954461661248) / 349.6493682100838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, IgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 275.84351894521257 < 1.932304455923547.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.556469522636984 (T) = (0 -275.7091050941596) / Math.Sqrt((122555.68941731732 / (284)) + (0.5605201432762585 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.211336822757116 = (349.5904374006924 - 275.7091050941596) / 349.5904374006924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 565.0628425365331 < 2.128839420248826.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.1940441274185245 (T) = (0 -560.4073922987109) / Math.Sqrt((116597.65770667755 / (283)) + (46.84636336530418 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.10384298315330574 = (625.3450921699136 - 560.4073922987109) / 625.3450921699136 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 230.47577893873964 < 1.9274190309935912.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.525852916248297 (T) = (0 -230.20911518663368) / Math.Sqrt((128134.50711998512 / (284)) + (0.06661074284722006 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.24546768101960445 = (305.10172910514524 - 230.20911518663368) / 305.10172910514524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, IgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 275.5335181290877 < 1.9290990473265575.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.588806039653803 (T) = (0 -275.3645627409369) / Math.Sqrt((118237.7379527133 / (284)) + (0.08247137589433791 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.21006488974057813 = (348.59137056270947 - 275.3645627409369) / 348.59137056270947 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 275.7072403915052 < 1.9327007798131466.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.554149036675186 (T) = (0 -275.78689554334517) / Math.Sqrt((121536.96217210295 / (283)) + (0.26447225300249116 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.21077826967913643 = (349.4415890338221 - 275.78689554334517) / 349.4415890338221 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, None, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 229.94387572192997 < 1.8516068270917625.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.5264275247805283 (T) = (0 -230.2319874793021) / Math.Sqrt((121044.68810695029 / (284)) + (0.05007887608694021 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.24024623392445138 = (303.0350065502778 - 230.2319874793021) / 303.0350065502778 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreNonSpace, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 230.2905292303777 < 1.8788346862428271.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.6349503434975405 (T) = (0 -230.31250478615067) / Math.Sqrt((126027.3735011378 / (284)) + (0.0605054766943823 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.24951518960828892 = (306.8849650213979 - 230.31250478615067) / 306.8849650213979 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.61471627087212 < 48.90631298233133.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.664882154064285 (T) = (0 -23.938251205583573) / Math.Sqrt((30.046551033697934 / (283)) + (0.29130489601604653 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.06185978784890811 = (25.516709438021834 - 23.938251205583573) / 25.516709438021834 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 565.185382856833 < 2.1485423061735593.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.1476056605524394 (T) = (0 -560.63971855544) / Math.Sqrt((118338.55968441919 / (283)) + (41.41778980113708 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.10311392253543902 = (625.0957982761114 - 560.63971855544) / 625.0957982761114 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, None, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 230.34091778118108 < 1.8837659892854592.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.6275856467829097 (T) = (0 -230.33974660150636) / Math.Sqrt((127869.43824921161 / (284)) + (0.10621687435884652 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.25047335740753834 = (307.3136210406178 - 230.33974660150636) / 307.3136210406178 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Hashtable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 30.90 μs 19.31 μs 0.62 0.02 False
SerializeObjectProperty - Duration of single invocation 31.25 μs 20.12 μs 0.64 0.01 False
SerializeToUtf8Bytes - Duration of single invocation 30.42 μs 18.56 μs 0.61 0.01 False
SerializeToStream - Duration of single invocation 30.03 μs 17.97 μs 0.60 0.02 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.309944925742574 < 29.823122618011443.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 21.386634930225103 (T) = (0 -19455.92755185179) / Math.Sqrt((4525322.592596052 / (283)) + (18065.864595372143 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.12375088814480084 = (22203.64881244741 - 19455.92755185179) / 22203.64881244741 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.117287392665112 < 30.525259432498803.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 21.670317222674033 (T) = (0 -19858.929669146) / Math.Sqrt((4580071.621082453 / (284)) + (40534.56389032661 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.12546513278241153 = (22707.99074292941 - 19858.929669146) / 22707.99074292941 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;.SerializeToUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.555582387329935 < 29.034276609338676.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 14.074723479214144 (T) = (0 -18315.895355358156) / Math.Sqrt((4767460.478357076 / (284)) + (35283.19974746755 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.0929774967702721 = (20193.429920579558 - 18315.895355358156) / 20193.429920579558 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;.SerializeToStream

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.968983402937788 < 28.68019881088327.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 7.604212203812095 (T) = (0 -17998.495927600612) / Math.Sqrt((5558923.902920165 / (284)) + (24514.1187209264 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.0567437795938196 = (19081.23746043274 - 17998.495927600612) / 19081.23746043274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_Guid

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryWriteBytes - Duration of single invocation 14.18 ns 0.00 ns 0.00 0.82 False
ParseExactD - Duration of single invocation 310.34 ns 31.93 ns 0.10 0.01 False
ctor_str - Duration of single invocation 314.25 ns 39.53 ns 0.13 0.02 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Guid*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Guid.TryWriteBytes

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 13.300763301521137.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.0188315482628814 (T) = (0 -0.1790810701781279) / Math.Sqrt((7.473165123391615 / (284)) + (0.31587856101767015 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.7601320062715048 = (0.7465817652222017 - 0.1790810701781279) / 0.7465817652222017 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Guid.ParseExactD

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.93385561506935 < 294.2240576352269.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.004335969143778 (T) = (0 -32.352296453240534) / Math.Sqrt((3103.520471439508 / (284)) + (0.14219143996810277 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.29039584329463947 = (45.592033456300264 - 32.352296453240534) / 45.592033456300264 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Guid.ctor_str

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.52756073959771 < 299.6494949407266.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.13153158205511 (T) = (0 -38.76774714021129) / Math.Sqrt((3096.7097834807496 / (283)) + (0.8033946199264332 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.26084748035740163 = (52.44891427679448 - 38.76774714021129) / 52.44891427679448 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NewCachedCustomConverter - Duration of single invocation 2.35 μs 1.68 μs 0.72 0.23 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCachedCustomConverter

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.682830891487523 < 2.2032581073603494.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 32.33997492211227 (T) = (0 -1756.1944678914795) / Math.Sqrt((30540.749279259362 / (283)) + (2342.8374610594865 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.19582131764677457 = (2183.836137949368 - 1756.1944678914795) / 2183.836137949368 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 327.80 ns 212.90 ns 0.65 0.02 False
ToString - Duration of single invocation 321.00 ns 208.25 ns 0.65 0.02 False
ToString - Duration of single invocation 331.15 ns 219.10 ns 0.66 0.01 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ja)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 212.89904496306443 < 309.06196387718336.
IsChangePoint: Marked as a change because one of 6/1/2022 7:59:43 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 38.104558175060994 (T) = (0 -212.87397593943675) / Math.Sqrt((403.8077952368704 / (284)) + (0.47569129015329586 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.17658945272827137 = (258.52714255940606 - 212.87397593943675) / 258.52714255940606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: da)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 208.2545561667318 < 309.821661062169.
IsChangePoint: Marked as a change because one of 6/1/2022 7:59:43 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 35.692084371812946 (T) = (0 -208.6319285918238) / Math.Sqrt((358.0602038755409 / (284)) + (0.5535730393621966 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.16198316995970272 = (248.95911527432142 - 208.6319285918238) / 248.95911527432142 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: fr)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 219.103457900995 < 312.99861603306476.
IsChangePoint: Marked as a change because one of 6/1/2022 7:59:43 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 38.105112302241146 (T) = (0 -219.69266811132397) / Math.Sqrt((379.72823015129313 / (284)) + (0.6596548199560212 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.16803171146306367 = (264.06375235487167 - 219.69266811132397) / 264.06375235487167 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 14, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 2.62 μs 1.63 μs 0.62 0.02 False
DeserializeFromUtf8Bytes - Duration of single invocation 2.48 μs 1.54 μs 0.62 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6317186999569615 < 2.5046486579045215.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 15.246084496454372 (T) = (0 -1628.7036550190396) / Math.Sqrt((35047.172764624636 / (284)) + (204.8365403284239 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.09618843907142595 = (1802.0389707625702 - 1628.7036550190396) / 1802.0389707625702 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Location&gt;.DeserializeFromUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5370059008724783 < 2.3520444537817715.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 16.900114059952386 (T) = (0 -1526.0862430073332) / Math.Sqrt((29910.04726931485 / (284)) + (71.30011258982108 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.10293094969437089 = (1701.1914996815458 - 1526.0862430073332) / 1701.1914996815458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 539.61 ns 304.57 ns 0.56 0.19 False
SerializeToString - Duration of single invocation 584.06 ns 328.26 ns 0.56 0.18 False
SerializeObjectProperty - Duration of single invocation 913.68 ns 660.01 ns 0.72 0.23 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToUtf8Bytes

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 304.57100950289947 < 508.4620767111909.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 16.32315439470638 (T) = (0 -307.3304720190361) / Math.Sqrt((1891.6997635303749 / (284)) + (92.54260828612962 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.13942623611271976 = (357.1227533486495 - 307.3304720190361) / 357.1227533486495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;.SerializeToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 328.26131379879945 < 539.8666033828788.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 18.87865191485143 (T) = (0 -329.656512478571) / Math.Sqrt((1976.0600585211705 / (284)) + (70.6294287654516 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1464468108204728 = (386.2167193065629 - 329.656512478571) / 386.2167193065629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 660.0140003277094 < 869.282353192685.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 20.394990110041853 (T) = (0 -639.5514605047111) / Math.Sqrt((4010.4695469368335 / (284)) + (603.8549128094725 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.15155172267573025 = (753.7895680826281 - 639.5514605047111) / 753.7895680826281 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayAssign3D - Duration of single invocation 1.65 ms 884.00 μs 0.53 0.46 False
ArrayRetrieve2D - Duration of single invocation 3.21 ms 2.80 ms 0.87 0.50 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.ArrayAssign3D

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 884.0017261574075 < 1.5669377805989582.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 6.040014102034459 (T) = (0 -941314.3630420464) / Math.Sqrt((22422778558.585197 / (284)) + (9651243696.25575 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.10781756721388792 = (1055069.3764530926 - 941314.3630420464) / 1055069.3764530926 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.ArrayRetrieve2D

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.804501640625 < 3.051599424438244.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 7.599275654527137 (T) = (0 -2673559.3080768147) / Math.Sqrt((36739543825.61966 / (284)) + (19400651332.76546 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.06917858268011182 = (2872258.0489980425 - 2673559.3080768147) / 2872258.0489980425 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 109.75 ns 59.34 ns 0.54 0.37 False
IsMatch - Duration of single invocation 105.98 ns 54.45 ns 0.51 0.36 False
IsMatch - Duration of single invocation 124.64 ns 61.02 ns 0.49 0.33 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 59.34042987074918 < 103.19340310623866.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 9.944979456303829 (T) = (0 -56.43008639485436) / Math.Sqrt((99.15430270668189 / (284)) + (8.984708542895895 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.12062725487503076 = (64.17083848423682 - 56.43008639485436) / 64.17083848423682 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.45306905760615 < 101.90253750099416.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 9.122594618394588 (T) = (0 -57.13271364336503) / Math.Sqrt((105.34973012081335 / (284)) + (9.570896297766815 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.11361484716329401 = (64.45585585512428 - 57.13271364336503) / 64.45585585512428 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 61.02192639488487 < 110.59799342667918.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 8.612222713832452 (T) = (0 -57.255659966853415) / Math.Sqrt((135.1219210844155 / (284)) + (10.629652717314883 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.11723231740876543 = (64.85926149764325 - 57.255659966853415) / 64.85926149764325 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation 58.13 μs 36.69 μs 0.63 0.04 False
DeserializeFromString - Duration of single invocation 61.45 μs 38.02 μs 0.62 0.02 False
DeserializeFromStream - Duration of single invocation 60.55 μs 41.80 μs 0.69 0.02 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.69035718591987 < 55.047366854982855.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 24.555431087524067 (T) = (0 -36784.29644749235) / Math.Sqrt((15220465.332147889 / (284)) + (239445.03057141008 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.14097254156518382 = (42820.862227750986 - 36784.29644749235) / 42820.862227750986 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;IndexViewModel&gt;.DeserializeFromString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.019188106416266 < 56.64787587285733.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 26.432687615287364 (T) = (0 -38110.843497414455) / Math.Sqrt((15480449.502236217 / (284)) + (399361.8824173762 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.15115613093091185 = (44897.35378451862 - 38110.843497414455) / 44897.35378451862 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;IndexViewModel&gt;.DeserializeFromStream

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.804720971154914 < 57.45413511096373.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 24.901351893566037 (T) = (0 -41874.268778367164) / Math.Sqrt((12481060.464188818 / (284)) + (366791.78745621117 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.12183194218662721 = (47683.66192074164 - 41874.268778367164) / 47683.66192074164 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBasicUtf8 - Duration of single invocation 2.04 μs 808.94 ns 0.40 0.01 False
WriteBasicUtf16 - Duration of single invocation 2.78 μs 1.06 μs 0.38 0.01 False
WriteBasicUtf8 - Duration of single invocation 2.06 μs 748.06 ns 0.36 0.01 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 10)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 808.9388666299594 < 1.9692384980844408.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.160220359090647 (T) = (0 -811.5593631549552) / Math.Sqrt((66268.41052978772 / (283)) + (9.723631310863533 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.0562682215496444 = (859.9470545408224 - 811.5593631549552) / 859.9470545408224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 10)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0597711288846363 < 2.635586178493382.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 2.710767990718683 (T) = (0 -1059.4327421916419) / Math.Sqrt((122304.54145596238 / (284)) + (4.482255268240973 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.05042819849245587 = (1115.6952433819981 - 1059.4327421916419) / 1115.6952433819981 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 10)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 748.0573897125231 < 1.9395695328374887.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 2.8625074477574444 (T) = (0 -747.1214458021581) / Math.Sqrt((68842.45010138955 / (283)) + (4.65986006735912 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.056402206637667256 = (791.7795601661294 - 747.1214458021581) / 791.7795601661294 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 14, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 1.71 μs 833.39 ns 0.49 0.04 False
SerializeToUtf8Bytes - Duration of single invocation 1.69 μs 775.45 ns 0.46 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 833.3881039592407 < 1.6698785563056853.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 10.110567058703179 (T) = (0 -805.1438189042009) / Math.Sqrt((32770.58193180622 / (284)) + (202.00319533935001 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.12143826833798627 = (916.4339737186993 - 805.1438189042009) / 916.4339737186993 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;.SerializeToUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 775.4492784875047 < 1.6193047849584241.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 8.5832218571978 (T) = (0 -753.5644947659134) / Math.Sqrt((33369.664527976456 / (284)) + (176.30528710478154 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.11196723408469728 = (848.5773540002302 - 753.5644947659134) / 848.5773540002302 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Formats.Cbor.Tests.Perf_CborWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteCoseKey - Duration of single invocation 1.27 μs 701.03 ns 0.55 0.23 False
WriteCoseKey - Duration of single invocation 1.35 μs 780.76 ns 0.58 0.30 False
WriteCoseKey - Duration of single invocation 1.24 μs 651.29 ns 0.52 0.05 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Formats.Cbor.Tests.Perf_CborWriter*'

Payloads

Baseline
Compare

Histogram

System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P384)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 701.0307650323656 < 1.250616415594285.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 6.265910024224804 (T) = (0 -726.531293693574) / Math.Sqrt((13417.312780731869 / (283)) + (632.5779220137792 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.06523800123174013 = (777.236659867354 - 726.531293693574) / 777.236659867354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P521)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 780.7555681098592 < 1.3165891288666893.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 7.761157805415193 (T) = (0 -788.8792237694323) / Math.Sqrt((13516.787007189643 / (284)) + (1425.7150861822308 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.08463714919186806 = (861.8213237220267 - 788.8792237694323) / 861.8213237220267 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P256)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 651.2891921968893 < 1.1835355911377392.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.281324097682334 (T) = (0 -660.1944033843043) / Math.Sqrt((13721.353048060142 / (283)) + (209.22278314522487 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.05574569010272755 = (699.170124472218 - 660.1944033843043) / 699.170124472218 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ExpandEnvironmentVariables - Duration of single invocation 555.02 ns 526.39 ns 0.95 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Environment*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Environment.ExpandEnvironmentVariables

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 526.3897463643681 < 529.9398398243267.
IsChangePoint: Marked as a change because one of 5/24/2022 11:43:29 AM, 6/23/2022 8:25:58 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 14.226129328355409 (T) = (0 -517.2185152473115) / Math.Sqrt((362.0372731646255 / (284)) + (117.10633323976796 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.055823681518669004 = (547.7986527762488 - 517.2185152473115) / 547.7986527762488 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Numerics.Tests.Constructor

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SpanCastBenchmark_Byte - Duration of single invocation 18.24 ns 0.64 ns 0.03 0.54 False
SpanCastBenchmark_UInt16 - Duration of single invocation 19.63 ns 0.79 ns 0.04 0.55 False
SpanCastBenchmark_Int16 - Duration of single invocation 19.19 ns 0.55 ns 0.03 0.60 False
SpanCastBenchmark_UInt32 - Duration of single invocation 19.53 ns 1.15 ns 0.06 0.45 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Constructor*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Constructor.SpanCastBenchmark_Byte

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.638270922988141 < 17.393355199196066.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.6658537485581517 (T) = (0 -0.48979425056982445) / Math.Sqrt((12.798868606237638 / (284)) + (0.004101313371846825 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.6140386867763119 = (1.2690242099107973 - 0.48979425056982445) / 1.2690242099107973 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_UInt16

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7933432030243992 < 18.475487076208047.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.6710965013737424 (T) = (0 -0.7546868616086505) / Math.Sqrt((14.180817134165808 / (284)) + (0.003324427044856678 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.521075238357195 = (1.575794199948919 - 0.7546868616086505) / 1.575794199948919 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_Int16

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.5483818332637955 < 18.317846376259197.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.61784183561198 (T) = (0 -0.7535036175783446) / Math.Sqrt((13.884161983849987 / (284)) + (0.004735639330529632 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.5152873089472336 = (1.5545365976322607 - 0.7535036175783446) / 1.5545365976322607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_UInt32

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1471844120011496 < 18.612797141666892.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.1396401234239004 (T) = (0 -1.0557656272991234) / Math.Sqrt((13.979321504614518 / (284)) + (0.016385874537508435 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.3986430113695546 = (1.7556387424773536 - 1.0557656272991234) / 1.7556387424773536 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BuilderToString - Duration of single invocation 336.59 ns 230.49 ns 0.68 0.04 False
GetComponents - Duration of single invocation 85.39 ns 46.87 ns 0.55 0.03 False
Unescape - Duration of single invocation 146.86 ns 82.60 ns 0.56 0.01 False
DnsSafeHost - Duration of single invocation 596.61 ns 393.80 ns 0.66 0.02 False
ParseAbsoluteUri - Duration of single invocation 683.26 ns 508.57 ns 0.74 0.02 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.BuilderToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 230.4927984718215 < 325.10082179141926.
IsChangePoint: Marked as a change because one of 5/21/2022 4:54:11 AM, 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 27.43773516846254 (T) = (0 -235.91251264592862) / Math.Sqrt((486.5863921101977 / (284)) + (17.3808074163304 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.14740986187648747 = (276.70096344904306 - 235.91251264592862) / 276.70096344904306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Uri.GetComponents

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.86990819524007 < 82.31864961665035.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 10.87351285914967 (T) = (0 -47.47651877065799) / Math.Sqrt((51.68947229073494 / (284)) + (1.5016561133605533 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.09797466372743759 = (52.633243060383556 - 47.47651877065799) / 52.633243060383556 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Uri.Unescape

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.6036895448695 < 138.76482373830603.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 11.023679014964703 (T) = (0 -83.37988912819439) / Math.Sqrt((140.7947855989603 / (284)) + (0.5120606080468258 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.08630139854217651 = (91.25535378423497 - 83.37988912819439) / 91.25535378423497 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Uri.DnsSafeHost

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 393.7969338036976 < 564.0283498004999.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 19.85805937512082 (T) = (0 -392.3024447937849) / Math.Sqrt((1282.2366264254229 / (284)) + (3.252233646148312 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.09800913608125386 = (434.92951036045594 - 392.3024447937849) / 434.92951036045594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Uri.ParseAbsoluteUri

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 508.56968541593545 < 648.9157229144237.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 21.190122617034707 (T) = (0 -509.1663365191705) / Math.Sqrt((1151.106429719021 / (283)) + (38.23652791927994 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.08637099672587033 = (557.3009774147874 - 509.1663365191705) / 557.3009774147874 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
StringAsSpan - Duration of single invocation 8.80 ns 0.97 ns 0.11 0.65 False
IndexOfString - Duration of single invocation 167.80 ns 38.03 ns 0.23 0.01 False
IndexOfString - Duration of single invocation 156.39 ns 28.86 ns 0.18 0.01 False
IndexOfString - Duration of single invocation 70.07 ns 27.88 ns 0.40 0.01 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.StringAsSpan

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.9692201630860287 < 8.343359679792686.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.30478590729346 (T) = (0 -0.9622467484858337) / Math.Sqrt((2.519134840787096 / (284)) + (0.03477019649463613 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.2543406043825239 = (1.290464190676499 - 0.9622467484858337) / 1.290464190676499 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "foobardzsdzs", value: "rddzs", comparisonType: InvariantCulture)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.02529887033399 < 159.71922299513434.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.1862985617259794 (T) = (0 -37.930837242138495) / Math.Sqrt((695.1159461891747 / (284)) + (0.03259419529900739 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1161749290302532 = (42.91667942901889 - 37.930837242138495) / 42.91667942901889 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "string1", value: "string2", comparisonType: InvariantCulture)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.85996367375084 < 148.61817217948675.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.4340289258331356 (T) = (0 -28.84184226568721) / Math.Sqrt((666.6675114426181 / (284)) + (0.01178420616390673 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.15428737328419598 = (34.10359660549246 - 28.84184226568721) / 34.10359660549246 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.88054957129768 < 66.5611844960094.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.4575917814822557 (T) = (0 -27.73956229845127) / Math.Sqrt((72.80014139240345 / (284)) + (0.008984763075611633 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.05938931465154767 = (29.491013370930457 - 27.73956229845127) / 29.491013370930457 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 97.48 ns 63.37 ns 0.65 0.02 False
ToString - Duration of single invocation 58.93 ns 43.34 ns 0.74 0.09 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTime.ToString(format: "o")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 63.36623626873521 < 93.4451846474593.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 26.52804164820658 (T) = (0 -63.003224755548544) / Math.Sqrt((43.78674604102975 / (283)) + (0.4215836605330024 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.146720568668905 = (73.8365680012526 - 63.003224755548544) / 73.8365680012526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTime.ToString(format: "r")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.33576660434224 < 55.89643057546454.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 33.71227788688088 (T) = (0 -42.82224576868668) / Math.Sqrt((18.699424762323186 / (283)) + (0.6726326933741957 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.18693919637279055 = (52.66795002003416 - 42.82224576868668) / 52.66795002003416 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 20.48 ns 2.00 ns 0.10 0.70 False
TryParse - Duration of single invocation 85.04 ns 6.64 ns 0.08 0.37 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Boolean*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Boolean.TryParse(value: "0")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0020380916602245 < 19.432630049727525.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.0276193658157933 (T) = (0 -1.9108278306201636) / Math.Sqrt((14.214204013175232 / (283)) + (0.1429447955233931 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.2696746015388738 = (2.616406104246795 - 1.9108278306201636) / 2.616406104246795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Boolean.TryParse(value: "Bogus")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.640234408283027 < 80.80292709468026.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.6397686418564765 (T) = (0 -6.56177942262437) / Math.Sqrt((248.36317337304365 / (286)) + (0.07676830384581938 / (35))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (286) + (35) - 2, .975) and 0.34104736863493706 = (9.957892434591573 - 6.56177942262437) / 9.957892434591573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 14, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteStringsUtf16 - Duration of single invocation 199.93 ms 87.74 ms 0.44 0.01 False
WriteStringsUtf16 - Duration of single invocation 199.37 ms 88.05 ms 0.44 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: AllEscaped)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 87.73706815 < 190.45614313107143.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.4319447920184873 (T) = (0 -87796141.12846938) / Math.Sqrt((523005018343278.1 / (284)) + (57010061691.43339 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.05039567736419859 = (92455498.60680398 - 87796141.12846938) / 92455498.60680398 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 88.05310595 < 189.46968685166664.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.4942436386934026 (T) = (0 -87942681.42112246) / Math.Sqrt((515295189557379.2 / (284)) + (38918066491.433655 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.050816591037065775 = (92650883.47594225 - 87942681.42112246) / 92650883.47594225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfValue - Duration of single invocation 29.24 ns 17.04 ns 0.58 0.10 False
StartsWith - Duration of single invocation 42.20 ns 18.64 ns 0.44 0.13 False
BinarySearch - Duration of single invocation 35.19 ns 12.47 ns 0.35 0.04 False
IndexOfAnyTwoValues - Duration of single invocation 30.20 ns 21.52 ns 0.71 0.12 False
SequenceEqual - Duration of single invocation 53.97 ns 39.15 ns 0.73 0.08 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.LastIndexOfValue(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.041452115734902 < 27.764657265465928.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/6/2022 8:15:49 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 8.57469195930765 (T) = (0 -17.07050316973357) / Math.Sqrt((9.357527900101294 / (284)) + (0.8577389781175888 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.10746447419921532 = (19.12585289467092 - 17.07050316973357) / 19.12585289467092 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.StartsWith(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.641145290094055 < 40.16378171797671.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.567420515466365 (T) = (0 -19.059571214743315) / Math.Sqrt((21.371417589913666 / (284)) + (0.10091514708167473 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.06278003396670265 = (20.33628380262886 - 19.059571214743315) / 20.33628380262886 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.BinarySearch(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.46639952967727 < 33.86395219328168.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.336003327914031 (T) = (0 -11.818378880355912) / Math.Sqrt((23.61646048252055 / (284)) + (0.04174207266548671 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.07576867412198873 = (12.787252010884353 - 11.818378880355912) / 12.787252010884353 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.515851946916587 < 28.720709087594432.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/6/2022 8:15:49 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.896428251541537 (T) = (0 -21.579099174375102) / Math.Sqrt((15.781019592173644 / (284)) + (0.22948364273281027 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.06376316955493151 = (23.048761245716882 - 21.579099174375102) / 23.048761245716882 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.SequenceEqual(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.14504627256698 < 51.655850683710526.
IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 10.873434049521245 (T) = (0 -38.94955126632746) / Math.Sqrt((11.561697818868446 / (283)) + (0.1730067507476754 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.05637453046519315 = (41.27649424885596 - 38.94955126632746) / 41.27649424885596 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Memory.MemoryMarshal<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CastToByte - Duration of single invocation 15.09 ns 0.17 ns 0.01 0.75 False
AsBytes - Duration of single invocation 15.76 ns 0.23 ns 0.01 0.69 False
CastToInt - Duration of single invocation 15.13 ns 0.23 ns 0.01 0.73 False
GetReference - Duration of single invocation 8.63 ns 0.56 ns 0.06 0.70 False
Read - Duration of single invocation 8.29 ns 2.72 ns 0.33 0.03 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.MemoryMarshal&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.MemoryMarshal<Byte>.CastToByte

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.16576626280205872 < 14.329080096605784.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.901064261516155 (T) = (0 -0.18157593713854428) / Math.Sqrt((8.273808159692829 / (285)) + (0.0033347222731468203 / (35))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (285) + (35) - 2, .975) and 0.8472450778344705 = (1.1886748692901925 - 0.18157593713854428) / 1.1886748692901925 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.MemoryMarshal&lt;Byte&gt;.AsBytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.22923113961759234 < 14.846440378342622.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.5519442294389565 (T) = (0 -0.20070960651688205) / Math.Sqrt((9.19889544134953 / (284)) + (0.005587887599254312 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.8330719861825354 = (1.2023722197782665 - 0.20070960651688205) / 1.2023722197782665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.MemoryMarshal&lt;Byte&gt;.CastToInt

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.22616624192597048 < 14.316083458077443.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.570781771419626 (T) = (0 -0.225786384924851) / Math.Sqrt((8.578162187797043 / (284)) + (0.0022915051645374654 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.8110591025263613 = (1.1950106511818228 - 0.225786384924851) / 1.1950106511818228 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.MemoryMarshal&lt;Byte&gt;.GetReference

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.5585572880622143 < 8.196178591458308.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.675499491817492 (T) = (0 -0.5965779470113544) / Math.Sqrt((2.490881515783525 / (283)) + (0.006373711007994036 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.3686689380395915 = (0.9449526293841162 - 0.5965779470113544) / 0.9449526293841162 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.MemoryMarshal&lt;Byte&gt;.Read

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7221037173479377 < 7.878696374031548.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.661221973187153 (T) = (0 -2.6349657832175524) / Math.Sqrt((1.3305549174318085 / (286)) + (0.0014397180414618314 / (35))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (286) + (35) - 2, .975) and 0.08691728098098417 = (2.8857908799856244 - 2.6349657832175524) / 2.8857908799856244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToDateTime_String - Duration of single invocation 796.65 ns 424.67 ns 0.53 0.01 False
ToBase64String - Duration of single invocation 1.79 μs 636.32 ns 0.36 0.02 False
FromBase64String - Duration of single invocation 117.12 ns 87.72 ns 0.75 0.01 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ToDateTime_String(value: "12/12/1999 11:59:59 PM")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 424.67011543062745 < 758.6218334336332.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.742331315572167 (T) = (0 -424.98176517674176) / Math.Sqrt((5446.762917116132 / (284)) + (6.206486463932703 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.05611093513579887 = (450.2454589172347 - 424.98176517674176) / 450.2454589172347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToBase64String(formattingOptions: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 636.3233988140238 < 1.6976832789580765.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/10/2022 3:20:59 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 148.12833946080863 (T) = (0 -640.4906519426603) / Math.Sqrt((27783.625118898166 / (299)) + (18.71793572127995 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.6916007437172076 = (2076.8229458872324 - 640.4906519426603) / 2076.8229458872324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.FromBase64String

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 87.72064709869663 < 111.69090132646966.
IsChangePoint: Marked as a change because one of 5/11/2022 7:18:06 PM, 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 23.184760066300584 (T) = (0 -88.54297167355594) / Math.Sqrt((46.10854761749882 / (284)) + (0.5942816284585801 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.0998181457633691 = (98.36120474639188 - 88.54297167355594) / 98.36120474639188 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse2 - Duration of single invocation 99.43 ns 32.40 ns 0.33 0.19 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.Parse2

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.404178771903496 < 95.94942861843764.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 8.541055779744843 (T) = (0 -33.173838517989275) / Math.Sqrt((168.6213299823272 / (284)) + (0.7650462031195521 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.16805717050098312 = (39.87514206711301 - 33.173838517989275) / 39.87514206711301 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 14, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateProperties - Duration of single invocation 8.05 μs 1.29 μs 0.16 0.01 False
Parse - Duration of single invocation 63.51 μs 30.42 μs 0.48 0.02 False
PropertyIndexer - Duration of single invocation 29.12 μs 3.78 μs 0.13 0.01 False
EnumerateProperties - Duration of single invocation 8.05 μs 1.29 μs 0.16 0.01 False
EnumerateProperties - Duration of single invocation 9.44 μs 2.24 μs 0.24 0.01 False
PropertyIndexer - Duration of single invocation 28.06 μs 2.66 μs 0.09 0.01 False
Parse - Duration of single invocation 58.67 μs 26.59 μs 0.45 0.01 False
PropertyIndexer - Duration of single invocation 27.84 μs 2.67 μs 0.10 0.01 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: NumericProperties)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.288075165950532 < 7.654695860906863.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 12.058401403777095 (T) = (0 -1287.0135303769614) / Math.Sqrt((1538886.7041231752 / (284)) + (10.354211121297489 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.4081803053985725 = (2174.671681454815 - 1287.0135303769614) / 2174.671681454815 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: NumericProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.42183736219195 < 63.400839206585566.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.207196697598271 (T) = (0 -30291.979459555852) / Math.Sqrt((53172935.24116772 / (283)) + (48744.0577347505 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.05698239203146575 = (32122.39008432874 - 30291.979459555852) / 32122.39008432874 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase: ObjectProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7799681673728815 < 27.658966765197164.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.7230224802618004 (T) = (0 -3795.312243065299) / Math.Sqrt((26115514.53370187 / (284)) + (240.44999584456133 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.229273875869334 = (4924.333202466945 - 3795.312243065299) / 4924.333202466945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: StringProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.287099684582647 < 7.652013694505857.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 12.0439800696514 (T) = (0 -1287.4367442099506) / Math.Sqrt((1539354.7542691624 / (284)) + (6.0481545517079125 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.4078458329749789 = (2174.1580417782498 - 1287.4367442099506) / 2174.1580417782498 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: ObjectProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2414409180087698 < 8.949427648562756.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 11.776418031464996 (T) = (0 -2236.6477950489716) / Math.Sqrt((1735759.108117631 / (283)) + (41.2554267120821 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.29198060458000036 = (3159.020514857766 - 2236.6477950489716) / 3159.020514857766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase: NumericProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6645167657372566 < 26.502819015430532.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.3228136974845564 (T) = (0 -2666.999070197475) / Math.Sqrt((26109670.18133587 / (284)) + (17.304776485550445 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.27418869604481816 = (3674.5074865383463 - 2666.999070197475) / 3674.5074865383463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: StringProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.592365125605323 < 56.59200729074917.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.759883190243788 (T) = (0 -26230.481759252885) / Math.Sqrt((45776231.81829531 / (283)) + (32914.374613180495 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.0546566806895501 = (27747.043030235684 - 26230.481759252885) / 27747.043030235684 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase: StringProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.666376918383563 < 26.462904356715427.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.3198414573077253 (T) = (0 -2666.480460503864) / Math.Sqrt((25814125.919592876 / (284)) + (35.549253898792315 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.2729189686879389 = (3667.3772876346957 - 2666.480460503864) / 3667.3772876346957 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.IO.Tests.BinaryWriterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteSingle - Duration of single invocation 25.45 ns 3.48 ns 0.14 0.22 False
WriteAsciiChar - Duration of single invocation 43.00 ns 7.44 ns 0.17 0.12 False
WriteUInt16 - Duration of single invocation 25.34 ns 2.43 ns 0.10 0.15 False
WriteUInt32 - Duration of single invocation 25.34 ns 3.51 ns 0.14 0.18 False
WriteDouble - Duration of single invocation 25.50 ns 3.43 ns 0.13 0.17 False
WriteNonAsciiChar - Duration of single invocation 44.45 ns 8.20 ns 0.18 0.19 False
WriteUInt64 - Duration of single invocation 25.27 ns 3.76 ns 0.15 0.21 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.BinaryWriterTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterTests.WriteSingle

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.481812141036364 < 24.397583180368713.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.831964020726606 (T) = (0 -3.219816766175306) / Math.Sqrt((20.05846900930081 / (284)) + (0.1971330296493614 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.24735945230803374 = (4.278027241621698 - 3.219816766175306) / 4.278027241621698 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterTests.WriteAsciiChar

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.444146267506267 < 41.39069189982784.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.3428444688657346 (T) = (0 -7.371992158657457) / Math.Sqrt((56.16649323602685 / (283)) + (0.05187532695476791 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.1685821889536835 = (8.866771989620966 - 7.371992158657457) / 8.866771989620966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterTests.WriteUInt16

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4320625186543023 < 23.995786490439567.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.744498621796873 (T) = (0 -3.398934338849622) / Math.Sqrt((19.534761559904876 / (284)) + (0.155822583805854 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.22966473275471616 = (4.412279280688003 - 3.398934338849622) / 4.412279280688003 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterTests.WriteUInt32

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.510390790070066 < 24.230763768525996.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.4975125271880776 (T) = (0 -3.442085800751726) / Math.Sqrt((19.65648844559687 / (284)) + (0.0921637019223462 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.21405746999609718 = (4.379564242101306 - 3.442085800751726) / 4.379564242101306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterTests.WriteDouble

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.4323315965608456 < 24.36294050617598.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.5522599906608248 (T) = (0 -3.364091015785909) / Math.Sqrt((19.747611898085157 / (284)) + (0.07958752760597111 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.2205515590789123 = (4.315989152291465 - 3.364091015785909) / 4.315989152291465 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterTests.WriteNonAsciiChar

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.199009526304648 < 43.13805598844061.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.424140937745634 (T) = (0 -8.049236150690756) / Math.Sqrt((58.45509295352417 / (283)) + (0.19281997799476538 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.16380849976053208 = (9.62606789041221 - 8.049236150690756) / 9.62606789041221 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterTests.WriteUInt64

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.761088985955089 < 23.888582777058243.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.670784843589388 (T) = (0 -3.532841562933951) / Math.Sqrt((18.917985644689743 / (284)) + (0.1569499638499266 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.216946077276922 = (4.511619775364203 - 3.532841562933951) / 4.511619775364203 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Hashtable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation 78.08 μs 59.86 μs 0.77 0.12 False
DeserializeFromStream - Duration of single invocation 78.84 μs 61.39 μs 0.78 0.17 False
DeserializeFromString - Duration of single invocation 77.43 μs 64.09 μs 0.83 0.11 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Hashtable&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromUtf8Bytes

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 59.85921442261619 < 74.55656967640529.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 26.076221776899573 (T) = (0 -61592.991749424924) / Math.Sqrt((17502151.091015782 / (284)) + (2735075.1363630053 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.13665031896585642 = (71341.88278803459 - 61592.991749424924) / 71341.88278803459 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Hashtable&gt;.DeserializeFromStream

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 61.38657040124212 < 75.2039546045316.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 24.890194788882997 (T) = (0 -63882.749008118204) / Math.Sqrt((21717216.404290006 / (284)) + (3212259.806627222 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.13779407896979007 = (74092.21793766815 - 63882.749008118204) / 74092.21793766815 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Hashtable&gt;.DeserializeFromString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 64.08962783354784 < 74.74412570528092.
IsChangePoint: Marked as a change because one of 5/24/2022 7:58:36 AM, 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 27.64218590800283 (T) = (0 -62486.18072751134) / Math.Sqrt((18453780.60190265 / (284)) + (2051657.2392013043 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.13459034092037903 = (72204.16374132747 - 62486.18072751134) / 72204.16374132747 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Xml.Tests.Perf_XmlConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TimeSpan_ToString - Duration of single invocation 224.65 ns 103.97 ns 0.46 0.01 False
DateTime_ToString_RoundtripKind - Duration of single invocation 140.75 ns 85.89 ns 0.61 0.02 False
DateTime_ToString - Duration of single invocation 140.69 ns 91.74 ns 0.65 0.01 False
DateTime_ToString_Unspecified - Duration of single invocation 136.56 ns 86.42 ns 0.63 0.01 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'

Payloads

Baseline
Compare

Histogram

System.Xml.Tests.Perf_XmlConvert.TimeSpan_ToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 103.96537378419573 < 209.39364625114067.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 11.679607754338383 (T) = (0 -104.33123594481727) / Math.Sqrt((469.5568940457101 / (284)) + (0.6498680052317729 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1264481673714757 = (119.43336622726068 - 104.33123594481727) / 119.43336622726068 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_RoundtripKind

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 85.88758432980968 < 129.14678297083583.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 18.6594874253917 (T) = (0 -86.02234880340619) / Math.Sqrt((82.2872000351168 / (284)) + (0.35968314459625456 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.10619553784297206 = (96.24291715416986 - 86.02234880340619) / 96.24291715416986 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 91.73611565722427 < 133.78197558069348.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 21.133716385919033 (T) = (0 -91.44492234753541) / Math.Sqrt((79.8052218683601 / (284)) + (0.4075162082903112 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.11112902514672865 = (102.87761096331276 - 91.44492234753541) / 102.87761096331276 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Unspecified

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 86.42086610530002 < 127.59514969601679.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 18.530729528000588 (T) = (0 -86.84179286749743) / Math.Sqrt((79.26077670375454 / (285)) + (0.5979439651855555 / (35))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (285) + (35) - 2, .975) and 0.10389078892945591 = (96.90983174221721 - 86.84179286749743) / 96.90983174221721 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseThenWrite - Duration of single invocation 108.19 μs 33.71 μs 0.31 0.01 False
ParseThenWrite - Duration of single invocation 22.44 μs 8.27 μs 0.37 0.01 False
ParseThenWrite - Duration of single invocation 85.25 μs 23.25 μs 0.27 0.01 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_ParseThenWrite*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: BroadTree)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.70656233091462 < 102.4351571855101.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.8595463920766475 (T) = (0 -33946.670069447544) / Math.Sqrt((220140733.85780728 / (284)) + (80615.4138465767 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.09111375336314136 = (37349.745576038826 - 33946.670069447544) / 37349.745576038826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.272526348682401 < 20.967100713957993.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.393077538478301 (T) = (0 -8188.74330118106) / Math.Sqrt((7693487.900249845 / (284)) + (4616.791781043639 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.08131583065710515 = (8913.556556698048 - 8188.74330118106) / 8913.556556698048 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.254272739321163 < 80.95236221809098.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.6957229899257333 (T) = (0 -23267.933354017332) / Math.Sqrt((154170253.18896225 / (284)) + (1112.3282462678108 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.10476851392826728 = (25990.96849923901 - 23267.933354017332) / 25990.96849923901 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 14, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 24.07 ms 5.22 ms 0.22 0.01 False
Count - Duration of single invocation 141.71 ns 95.10 ns 0.67 0.38 False
Count - Duration of single invocation 9.64 ms 4.21 ms 0.44 0.01 False
Count - Duration of single invocation 10.74 ms 1.35 ms 0.13 0.02 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\s[a-zA-Z]{0,12}ing\s", Options: Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.223982147321428 < 22.99508235080128.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.8838088250953264 (T) = (0 -5223922.182004296) / Math.Sqrt((14545620510953.848 / (284)) + (381410070.4087807 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1440356442011144 = (6102966.959563081 - 5223922.182004296) / 6102966.959563081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 95.09629691158278 < 138.5702326563908.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.5424501609990715 (T) = (0 -98.76005660866811) / Math.Sqrt((136.6648448594679 / (283)) + (21.021403319184852 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.05519291239755522 = (104.52933504053506 - 98.76005660866811) / 104.52933504053506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.20803950625 < 9.209935914166666.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.703928004612483 (T) = (0 -4207779.277474816) / Math.Sqrt((1194589172584.5757 / (284)) + (117671415.99897957 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.06762712960917622 = (4512979.099993586 - 4207779.277474816) / 4512979.099993586 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Lu}", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.352640013392857 < 10.189076971082255.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.7638959861139876 (T) = (0 -1331871.713494789) / Math.Sqrt((3569548169859.174 / (284)) + (412197225.8824916 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.24068457861593934 = (1754042.7548107577 - 1331871.713494789) / 1754042.7548107577 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Date_IsNotMatch - Duration of single invocation 575.36 ns 144.53 ns 0.25 0.11 False
Date_IsMatch - Duration of single invocation 305.41 ns 92.94 ns 0.30 0.32 False
ReplaceWords - Duration of single invocation 9.63 μs 5.14 μs 0.53 0.02 False
MatchWord - Duration of single invocation 383.62 ns 211.15 ns 0.55 0.19 False
Date_IsMatch - Duration of single invocation 303.56 ns 86.06 ns 0.28 0.32 False
Date_IsNotMatch - Duration of single invocation 575.35 ns 144.80 ns 0.25 0.08 False
MatchWord - Duration of single invocation 414.68 ns 240.47 ns 0.58 0.16 False
IP_IsMatch - Duration of single invocation 502.36 ns 156.01 ns 0.31 0.06 False
IP_IsNotMatch - Duration of single invocation 470.62 ns 154.86 ns 0.33 0.07 False
IP_IsMatch - Duration of single invocation 486.83 ns 155.97 ns 0.32 0.08 False
SplitWords - Duration of single invocation 7.75 μs 4.29 μs 0.55 0.07 False
SplitWords - Duration of single invocation 8.94 μs 5.05 μs 0.56 0.05 False
IP_IsNotMatch - Duration of single invocation 480.85 ns 155.92 ns 0.32 0.04 False
ReplaceWords - Duration of single invocation 8.30 μs 4.38 μs 0.53 0.06 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: IgnoreCase, Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 144.53123966518348 < 543.6095321079633.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.331095253639029 (T) = (0 -144.60264687882656) / Math.Sqrt((7320.251610589571 / (284)) + (15.55187844070132 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.13297457727691428 = (166.78016940341826 - 144.60264687882656) / 166.78016940341826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 92.93932793162914 < 290.51400140743294.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.256508545710904 (T) = (0 -91.70238605757851) / Math.Sqrt((1786.3134915814992 / (283)) + (9.058837449407216 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.12811189341455537 = (105.17678285199973 - 91.70238605757851) / 105.17678285199973 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.143370778178784 < 9.127372241213772.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 26.525097678331605 (T) = (0 -5182.352477415549) / Math.Sqrt((662559.5746986906 / (284)) + (1190.739121599465 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1993696863814693 = (6472.840697216892 - 5182.352477415549) / 6472.840697216892 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 211.15386815155625 < 365.155293727761.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 18.467595677956567 (T) = (0 -219.07280095977674) / Math.Sqrt((982.4403405364707 / (283)) + (60.024365665472544 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.1610607467254818 = (261.1307077415909 - 219.07280095977674) / 261.1307077415909 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 86.06064870241698 < 289.4723870524946.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.23269949051005 (T) = (0 -91.17168843922266) / Math.Sqrt((1845.8138320837472 / (283)) + (10.102524109590133 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.13027255788805364 = (104.827885179559 - 91.17168843922266) / 104.827885179559 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 144.8011578168269 < 545.006033561647.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.394167702436062 (T) = (0 -143.7394139440901) / Math.Sqrt((7578.409339633849 / (284)) + (9.789881656366513 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.13699596158808375 = (166.5570583060036 - 143.7394139440901) / 166.5570583060036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 240.47163218675735 < 409.2176248659289.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 19.319326518036572 (T) = (0 -240.79679221410646) / Math.Sqrt((1261.1793481996685 / (283)) + (19.052767679205434 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.15212307636627623 = (283.9997003127883 - 240.79679221410646) / 283.9997003127883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 156.00584329584527 < 464.2186839842752.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.704751898478225 (T) = (0 -153.02338032528937) / Math.Sqrt((4510.512440094331 / (284)) + (12.271616468269086 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.11021923178591066 = (171.97874554248688 - 153.02338032528937) / 171.97874554248688 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 154.85870018725836 < 457.5441242580996.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.433249769579797 (T) = (0 -154.12119387003636) / Math.Sqrt((4474.7231350533575 / (284)) + (8.970438978340438 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1032220555777565 = (171.8610441175939 - 154.12119387003636) / 171.8610441175939 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 155.9687771126369 < 461.024472021195.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.37496160301356 (T) = (0 -154.44787518912517) / Math.Sqrt((4421.813220505573 / (284)) + (15.740122423559459 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.10182986358487366 = (171.95837283743836 - 154.44787518912517) / 171.95837283743836 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.287139365243343 < 7.330791577180089.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 33.29677427378543 (T) = (0 -4313.871514412786) / Math.Sqrt((468437.1936563466 / (284)) + (2809.3852553299153 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.24300394467615347 = (5698.671061855523 - 4313.871514412786) / 5698.671061855523 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.050953650336214 < 8.365021219031524.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 30.275686050007874 (T) = (0 -5008.339999835264) / Math.Sqrt((595684.8449299927 / (284)) + (2463.36858476698 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.21963990194547536 = (6417.985763651034 - 5008.339999835264) / 6417.985763651034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 155.92050952249704 < 461.7548905969233.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.376356208854782 (T) = (0 -154.29764608354554) / Math.Sqrt((4463.147468324262 / (284)) + (21.279539240849523 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1028113466863487 = (171.97904310722942 - 154.29764608354554) / 171.97904310722942 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.3766602751345065 < 7.838223371204528.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 29.073754338122242 (T) = (0 -4430.400668901469) / Math.Sqrt((511555.968184899 / (284)) + (937.1041726273985 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.21910100755167577 = (5673.461883989624 - 4430.400668901469) / 5673.461883989624 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Memory.Constructors<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MemoryMarshalCreateReadOnlySpan - Duration of single invocation 5.61 ns 0.11 ns 0.02 0.71 False
MemoryMarshalCreateSpan - Duration of single invocation 5.56 ns 0.12 ns 0.02 0.76 False
ArrayAsSpan - Duration of single invocation 6.28 ns 0.15 ns 0.02 0.72 False
ArrayAsSpanStartLength - Duration of single invocation 6.39 ns 0.32 ns 0.05 0.62 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Constructors&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Constructors<Byte>.MemoryMarshalCreateReadOnlySpan

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.10697475950343946 < 5.281344695458538.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.3618532023354826 (T) = (0 -0.08737279343087957) / Math.Sqrt((1.231543561347569 / (284)) + (0.0036375465243844753 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.719413201968838 = (0.31139310204172865 - 0.08737279343087957) / 0.31139310204172865 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;Byte&gt;.MemoryMarshalCreateSpan

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.12194928986698152 < 5.315862653248867.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.422620314771195 (T) = (0 -0.08800466206694278) / Math.Sqrt((1.2363979571600967 / (284)) + (0.0017263689389730628 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.7207166150816554 = (0.3151088350374769 - 0.08800466206694278) / 0.3151088350374769 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;Byte&gt;.ArrayAsSpan

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.15063623779939672 < 6.08455980781704.
IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 8.023603231579091 (T) = (0 -0.1929743534166804) / Math.Sqrt((1.5400674546311737 / (283)) + (0.0020816766688018236 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.7551388348295155 = (0.7880970152303326 - 0.1929743534166804) / 0.7880970152303326 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Constructors&lt;Byte&gt;.ArrayAsSpanStartLength

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.3164747771257901 < 6.121673030792578.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.3260432429757434 (T) = (0 -0.3291604516339967) / Math.Sqrt((1.4747481728133505 / (283)) + (0.004607273055633138 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.4248210821365908 = (0.5722748894495542 - 0.3291604516339967) / 0.5722748894495542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterUInt32 - Duration of single invocation 41.33 ns 7.82 ns 0.19 0.21 False
FormatterUInt32 - Duration of single invocation 21.40 ns 1.05 ns 0.05 0.76 False
FormatterUInt64 - Duration of single invocation 39.92 ns 7.68 ns 0.19 0.25 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 12345)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.818836077345761 < 38.74339442700278.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.460492036260294 (T) = (0 -7.460371466678612) / Math.Sqrt((44.942740743430114 / (286)) + (0.03441407944784639 / (35))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (286) + (35) - 2, .975) and 0.1557262821760767 = (8.836436938848903 - 7.460371466678612) / 8.836436938848903 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 0)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0541804421995158 < 19.58150712303131.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.4193284948372993 (T) = (0 -0.9009544434729417) / Math.Sqrt((16.412072412610637 / (284)) + (0.06306253114548001 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.4809136567156936 = (1.7356542993840316 - 0.9009544434729417) / 1.7356542993840316 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.67651564732656 < 38.32103272971116.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.5161787373052564 (T) = (0 -7.446203406177082) / Math.Sqrt((44.37285636902633 / (284)) + (0.033829962068957005 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.15770302450274937 = (8.840353963970024 - 7.446203406177082) / 8.840353963970024 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 14, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 6.74 μs 2.14 μs 0.32 0.01 False
Parse - Duration of single invocation 12.77 μs 3.33 μs 0.26 0.01 False
Parse - Duration of single invocation 3.56 ms 1.43 ms 0.40 0.01 False
Parse - Duration of single invocation 40.07 ms 6.72 ms 0.17 0.17 False
Parse - Duration of single invocation 631.47 ns 317.90 ns 0.50 0.11 False
Parse - Duration of single invocation 4.18 μs 1.83 μs 0.44 0.01 False
Parse - Duration of single invocation 3.18 μs 1.38 μs 0.43 0.01 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: BasicJson)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.137235293070282 < 6.398460758080555.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 6.084529231264501 (T) = (0 -2139.477630881822) / Math.Sqrt((799849.5688930199 / (284)) + (368.2399130893573 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.13134701902360563 = (2462.983121841105 - 2139.477630881822) / 2462.983121841105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.3340757824917597 < 12.164878110818561.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.3023977535863756 (T) = (0 -3329.996466080929) / Math.Sqrt((3524997.823336481 / (284)) + (1102.090010533609 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1508317365187299 = (3921.4801227134863 - 3329.996466080929) / 3921.4801227134863 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4322507425287354 < 3.3760016742366568.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.8946288234497173 (T) = (0 -1431424.2931848352) / Math.Sqrt((181080407510.55743 / (284)) + (38953091.44190588 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.0643386082474252 = (1529852.8995662134 - 1431424.2931848352) / 1529852.8995662134 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.718950185031186 < 37.568697209374996.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 2.847618050357637 (T) = (0 -7185087.847749857) / Math.Sqrt((43355282080799.375 / (284)) + (1740629398349.8306 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.15131812300921552 = (8466173.300679399 - 7185087.847749857) / 8466173.300679399 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 317.90004516775736 < 606.6748435988462.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.749025976397137 (T) = (0 -311.5561322395184) / Math.Sqrt((4095.6781611817923 / (284)) + (51.078616794776174 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.06849799019259753 = (334.4664090461123 - 311.5561322395184) / 334.4664090461123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.832571517593886 < 3.9910789153519572.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.871085571082595 (T) = (0 -1796.873806224944) / Math.Sqrt((236204.68163422818 / (284)) + (383.35919862437254 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.058856848493570466 = (1909.2460093332236 - 1796.873806224944) / 1909.2460093332236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: BasicJson)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3812981032354141 < 3.019675783356556.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.68620566207685 (T) = (0 -1391.3292798891246) / Math.Sqrt((125233.49658285061 / (284)) + (275.11212572323274 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.06660325142083946 = (1490.6086634724625 - 1391.3292798891246) / 1490.6086634724625 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 881.51 μs 427.69 μs 0.49 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.XmlSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 427.6918423939542 < 853.5684789132642.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 8.369496711907095 (T) = (0 -431146.2815477151) / Math.Sqrt((7984831924.701558 / (284)) + (11382267.580494702 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.09381309886385579 = (475780.74788673234 - 431146.2815477151) / 475780.74788673234 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Load - Duration of single invocation 334.05 μs 105.99 μs 0.32 0.04 False
Load - Duration of single invocation 96.39 μs 66.64 μs 0.69 0.19 False
Load - Duration of single invocation 264.53 μs 163.38 μs 0.62 0.26 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "deep.xml")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 105.9856405769768 < 317.57077746897187.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 10.245189492147562 (T) = (0 -102549.24285721786) / Math.Sqrt((1922688752.14588 / (284)) + (4058781.886052364 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.20770912444324208 = (129433.83045419342 - 102549.24285721786) / 129433.83045419342 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "simple.xml")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 66.64483786666666 < 93.09405027211925.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 30.381611309396757 (T) = (0 -68625.61246619395) / Math.Sqrt((70636037.60990098 / (284)) + (4496869.0701707415 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.21377522649044933 = (87284.9785181188 - 68625.61246619395) / 87284.9785181188 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "repeated.xml")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 163.37534164332396 < 260.7583901505887.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 17.917376321012057 (T) = (0 -157870.62370514614) / Math.Sqrt((473725119.29820174 / (284)) + (35449449.475932196 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1567077585954557 = (187207.489828443 - 157870.62370514614) / 187207.489828443 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseSpan - Duration of single invocation 27.67 ns 11.07 ns 0.40 0.25 False
TryParseHex - Duration of single invocation 80.04 ns 10.18 ns 0.13 0.21 False
TryFormat - Duration of single invocation 23.67 ns 11.01 ns 0.46 0.07 False
TryParse - Duration of single invocation 22.51 ns 9.99 ns 0.44 0.34 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.ParseSpan(value: "0")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.072567457886068 < 26.401728965330925.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.927796281307471 (T) = (0 -11.15931732841284) / Math.Sqrt((12.426035871729503 / (283)) + (0.2513535629941668 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.07369411144301544 = (12.047119063225452 - 11.15931732841284) / 12.047119063225452 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParseHex(value: "3039")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.17578259259559 < 79.46975757545592.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.6214869463051103 (T) = (0 -10.001756499240848) / Math.Sqrt((217.98493608472177 / (284)) + (0.04014237870329734 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.2409636469118888 = (13.176913672907856 - 10.001756499240848) / 13.176913672907856 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.006427171193252 < 22.524888549474074.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.470003057692222 (T) = (0 -11.086059069258319) / Math.Sqrt((6.8993314847248115 / (283)) + (0.15828948244764446 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.07739103353646365 = (12.015988866607731 - 11.086059069258319) / 12.015988866607731 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParse(value: "0")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.985798362738418 < 21.505292457427796.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.363751325923624 (T) = (0 -9.991295842581723) / Math.Sqrt((6.919069345264837 / (284)) + (0.5642843148960953 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.06344319210361427 = (10.668115119491066 - 9.991295842581723) / 10.668115119491066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.IO.Tests.Perf_FileInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ctor_str - Duration of single invocation 298.03 ns 201.90 ns 0.68 0.57 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_FileInfo*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_FileInfo.ctor_str

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 201.90219570078787 < 286.8218962220882.
IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 6/23/2022 3:07:42 AM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 9.060231775203256 (T) = (0 -198.2462272612158) / Math.Sqrt((728.9539174051928 / (283)) + (248.9155474214726 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.12453290878205317 = (226.44623567222422 - 198.2462272612158) / 226.44623567222422 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation 580.46 ns 391.76 ns 0.67 0.21 False
DeserializeFromString - Duration of single invocation 744.62 ns 477.92 ns 0.64 0.24 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromUtf8Bytes

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 391.756809728938 < 553.2665379431882.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 22.13949885016279 (T) = (0 -402.5031890099942) / Math.Sqrt((1138.525955410839 / (284)) + (65.99628788442622 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.11781048116151797 = (456.25478473145125 - 402.5031890099942) / 456.25478473145125 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;SimpleStructWithProperties&gt;.DeserializeFromString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 477.92420219805956 < 708.835352395232.
IsChangePoint: Marked as a change because one of 6/22/2022 11:10:17 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 15.983578588034511 (T) = (0 -484.99457942894605) / Math.Sqrt((2297.90269518917 / (284)) + (124.13340363509764 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.10106531301152638 = (539.5214874327847 - 484.99457942894605) / 539.5214874327847 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Decode_DecodingRequired - Duration of single invocation 176.96 ns 129.95 ns 0.73 0.08 False
Decode_NoDecodingRequired - Duration of single invocation 105.18 ns 82.30 ns 0.78 0.05 False
HtmlDecode_Entities - Duration of single invocation 898.29 ns 231.17 ns 0.26 0.01 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.Decode_DecodingRequired

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 129.94529216490167 < 167.73469610211328.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 34.90701974768813 (T) = (0 -131.33584084632335) / Math.Sqrt((149.03522813990517 / (283)) + (5.05235477457113 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.17878832983162105 = (159.92934053091892 - 131.33584084632335) / 159.92934053091892 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.30351776289503 < 100.77408331092724.
IsChangePoint: Marked as a change because one of 5/17/2022 3:30:02 AM, 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 24.342488530300695 (T) = (0 -81.1086346834559) / Math.Sqrt((24.966063728844635 / (284)) + (0.5986661337274548 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.08863637361803105 = (88.99700661244276 - 81.1086346834559) / 88.99700661244276 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 231.1712154815835 < 810.8360392356366.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.836313559347183 (T) = (0 -230.48369938442056) / Math.Sqrt((15501.230143242385 / (284)) + (0.862831063632789 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.10952596934280844 = (258.8325896649877 - 230.48369938442056) / 258.8325896649877 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 3.84 μs 2.30 μs 0.60 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.XmlSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2999992517475034 < 3.662730354524672.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 38.15309242880867 (T) = (0 -2216.78618878647) / Math.Sqrt((265220.8171834989 / (283)) + (1258.875853338812 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.34934013924718177 = (3406.9816235807634 - 2216.78618878647) / 3406.9816235807634 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 14, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 205.02 ns 162.97 ns 0.79 0.02 False
ToStringWithFormat - Duration of single invocation 261.56 ns 215.82 ns 0.83 0.03 False
ToStringWithFormat - Duration of single invocation 226.57 ns 178.59 ns 0.79 0.01 False
ToStringWithFormat - Duration of single invocation 206.51 ns 166.38 ns 0.81 0.01 False
ToStringWithFormat - Duration of single invocation 222.25 ns 176.46 ns 0.79 0.01 False
ToStringWithFormat - Duration of single invocation 260.67 ns 214.33 ns 0.82 0.02 False
ToStringWithFormat - Duration of single invocation 428.56 ns 336.37 ns 0.78 0.12 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "E")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 162.96856492270854 < 194.64449629459958.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 23.466962822037424 (T) = (0 -162.39313019987145) / Math.Sqrt((56.70994538617436 / (284)) + (3.048546375984912 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.07182823875523876 = (174.96021424104495 - 162.39313019987145) / 174.96021424104495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 215.82200652482194 < 249.1653017228039.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 18.51950005492445 (T) = (0 -217.14015757952524) / Math.Sqrt((73.57181809168235 / (284)) + (5.48574687472229 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.052128620329694335 = (229.08187992241335 - 217.14015757952524) / 229.08187992241335 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 178.58896200418928 < 215.46404119975952.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 19.500469481493464 (T) = (0 -180.49860264990113) / Math.Sqrt((68.17015394305243 / (283)) + (3.8526010433418065 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.06015284638722556 = (192.0510180363521 - 180.49860264990113) / 192.0510180363521 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 166.3774877775061 < 196.6328244280963.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 24.019390182619784 (T) = (0 -166.75094867906466) / Math.Sqrt((51.73686636180182 / (284)) + (2.4894567022868523 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.0675951109847514 = (178.83963355788197 - 166.75094867906466) / 178.83963355788197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 176.46270306855467 < 212.19949439887364.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 20.36385692903103 (T) = (0 -176.90984346288477) / Math.Sqrt((69.44831668358898 / (283)) + (1.6372058079951126 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.05857563707865982 = (187.9172139904205 - 176.90984346288477) / 187.9172139904205 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 214.33261802386806 < 246.46144547190445.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 19.39232150056259 (T) = (0 -214.57740422835118) / Math.Sqrt((70.09004114186239 / (284)) + (4.366731175027502 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.05221211103748484 = (226.39812844964268 - 214.57740422835118) / 226.39812844964268 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "F50")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 336.3674947684531 < 388.9048510363398.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 25.45729381513424 (T) = (0 -337.6407543413243) / Math.Sqrt((291.7586359012022 / (284)) + (12.631046804785015 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.08158724500450569 = (367.63508836828026 - 337.6407543413243) / 367.63508836828026 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Memory.Slice<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MemorySpanStart - Duration of single invocation 17.77 ns 5.55 ns 0.31 0.01 False
ReadOnlyMemoryStartSpan - Duration of single invocation 10.31 ns 3.83 ns 0.37 0.02 False
ReadOnlyMemorySpanStartLength - Duration of single invocation 17.61 ns 4.84 ns 0.27 0.01 False
MemoryStartSpan - Duration of single invocation 10.23 ns 3.93 ns 0.38 0.01 False
MemoryStartLengthSpan - Duration of single invocation 8.85 ns 3.60 ns 0.41 0.01 False
ReadOnlyMemorySpanStart - Duration of single invocation 17.76 ns 5.79 ns 0.33 0.02 False
ReadOnlyMemoryStartLengthSpan - Duration of single invocation 8.39 ns 3.65 ns 0.43 0.02 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Slice&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Slice<Byte>.MemorySpanStart

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.545643997184116 < 17.0047999965109.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.6508351152185203 (T) = (0 -5.533016814755936) / Math.Sqrt((6.162534137950305 / (284)) + (0.001800856658952777 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.08868190505780597 = (6.07144403854606 - 5.533016814755936) / 6.07144403854606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;Byte&gt;.ReadOnlyMemoryStartSpan

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.8344737395640744 < 9.76393494453481.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.1416194489138345 (T) = (0 -3.8831739853674088) / Math.Sqrt((1.6418449870971141 / (284)) + (0.000492545358931975 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.07509568777255425 = (4.198460245055585 - 3.8831739853674088) / 4.198460245055585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;Byte&gt;.ReadOnlyMemorySpanStartLength

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.838458067328826 < 16.71175333418411.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.640001887648925 (T) = (0 -4.824929576486259) / Math.Sqrt((6.615185411927436 / (284)) + (0.000841713960974073 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1032985920572298 = (5.380753876093165 - 4.824929576486259) / 5.380753876093165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;Byte&gt;.MemoryStartSpan

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.9317661194569333 < 9.727753949574327.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.371477552335129 (T) = (0 -3.9387994702087425) / Math.Sqrt((1.6349877041214342 / (283)) + (0.0030883988451139445 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.06152274393910414 = (4.1970111100413945 - 3.9387994702087425) / 4.1970111100413945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;Byte&gt;.MemoryStartLengthSpan

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.5982115790025953 < 8.218485910905729.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.795866892106419 (T) = (0 -3.5885659003281276) / Math.Sqrt((1.068540653756358 / (283)) + (0.0006129379944814308 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.06116281553724751 = (3.8223516917703644 - 3.5885659003281276) / 3.8223516917703644 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;Byte&gt;.ReadOnlyMemorySpanStart

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.7930114672449955 < 16.94479721822966.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 2.145903140898563 (T) = (0 -5.7714872742711565) / Math.Sqrt((6.1449589754526315 / (284)) + (0.0012436381894267514 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.05189612428155803 = (6.087399727058085 - 5.7714872742711565) / 6.087399727058085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;Byte&gt;.ReadOnlyMemoryStartLengthSpan

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.6485361828151306 < 8.230621127914711.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.92820644623752 (T) = (0 -3.584351977263428) / Math.Sqrt((1.0248539977659723 / (284)) + (0.002847743171616986 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.06241753364603498 = (3.822972491371473 - 3.584351977263428) / 3.822972491371473 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringHex - Duration of single invocation 29.26 ns 21.35 ns 0.73 0.28 False
ToStringHex - Duration of single invocation 26.90 ns 16.68 ns 0.62 0.33 False
ToStringHex - Duration of single invocation 37.85 ns 29.84 ns 0.79 0.15 False
Parse - Duration of single invocation 26.19 ns 10.96 ns 0.42 0.13 False
ToStringHex - Duration of single invocation 33.77 ns 25.75 ns 0.76 0.22 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ToStringHex(value: 12345)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.345860026329696 < 28.54808519082146.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 24.294582717224372 (T) = (0 -20.71595013663092) / Math.Sqrt((4.7494842292793376 / (284)) + (0.517990245711163 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.17233540476453488 = (25.029402315725935 - 20.71595013663092) / 25.029402315725935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToStringHex(value: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.684552728217355 < 25.410301879765942.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 19.726873579495027 (T) = (0 -16.41489563919488) / Math.Sqrt((4.463561982099868 / (284)) + (0.2824963875488173 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.15636955267391386 = (19.457448093797968 - 16.41489563919488) / 19.457448093797968 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToStringHex(value: 2147483647)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.837792098208276 < 35.938633654338155.
IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 30.003576921755915 (T) = (0 -30.634597409238026) / Math.Sqrt((6.873875742276839 / (284)) + (0.6095994589175289 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.16653452512620734 = (36.75568854711932 - 30.634597409238026) / 36.75568854711932 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.Parse(value: "4")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.961469940089899 < 25.505347965776533.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.525655143325884 (T) = (0 -11.00350064541019) / Math.Sqrt((10.476638674453879 / (284)) + (0.06738179847254508 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.07495680853725775 = (11.895120948904768 - 11.00350064541019) / 11.895120948904768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToStringHex(value: -2147483648)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.74977160485886 < 32.485170011314054.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 29.261818673916224 (T) = (0 -25.00341922502847) / Math.Sqrt((6.108115616555903 / (284)) + (0.45971943019038847 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.17886310853365198 = (30.44975750678858 - 25.00341922502847) / 30.44975750678858 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 762.76 μs 425.54 μs 0.56 0.09 False
SerializeToString - Duration of single invocation 822.84 μs 476.85 μs 0.58 0.08 False
SerializeObjectProperty - Duration of single invocation 819.74 μs 478.43 μs 0.58 0.12 False
SerializeToStream - Duration of single invocation 770.61 μs 425.90 μs 0.55 0.11 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 425.53938292619546 < 725.484950377747.
IsChangePoint: Marked as a change because one of 6/1/2022 7:59:43 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 18.69180984444353 (T) = (0 -422363.51807284233) / Math.Sqrt((4249201475.113541 / (284)) + (32634920.29520026 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.14997497015398775 = (496883.6248849711 - 422363.51807284233) / 496883.6248849711 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;.SerializeToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 476.8546472222221 < 778.3787991845238.
IsChangePoint: Marked as a change because one of 6/1/2022 7:59:43 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 17.273200670402353 (T) = (0 -469814.60299661476) / Math.Sqrt((4341771679.066597 / (284)) + (50895175.03923939 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1307641237999647 = (540491.5004779409 - 469814.60299661476) / 540491.5004779409 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 478.43250202922076 < 777.8877168229168.
IsChangePoint: Marked as a change because one of 6/2/2022 1:01:27 AM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 18.480574139312736 (T) = (0 -470280.2979387979) / Math.Sqrt((4143345718.0333834 / (284)) + (58965156.83084273 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.13683553824316994 = (544832.7853786048 - 470280.2979387979) / 544832.7853786048 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;.SerializeToStream

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 425.90386402027025 < 734.9465940815305.
IsChangePoint: Marked as a change because one of 6/1/2022 7:59:43 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 21.03467875006001 (T) = (0 -424572.02943429304) / Math.Sqrt((4189946903.9417253 / (284)) + (23394071.488045104 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.16287092570441583 = (507176.30347692326 - 424572.02943429304) / 507176.30347692326 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 14, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in JetStream.Poker

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Play - Duration of single invocation 16.98 ms 11.66 ms 0.69 0.28 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'JetStream.Poker*'

Payloads

Baseline
Compare

Histogram

JetStream.Poker.Play

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.662061017006803 < 15.66713673375425.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 14.481711135446567 (T) = (0 -11874619.526080137) / Math.Sqrt((866333289513.8492 / (283)) + (172400800796.33585 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.09827962875871056 = (13168849.129729412 - 11874619.526080137) / 13168849.129729412 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 2.52 μs 1.61 μs 0.64 0.07 False
DeserializeFromStream - Duration of single invocation 3.02 μs 2.08 μs 0.69 0.02 False
DeserializeFromUtf8Bytes - Duration of single invocation 2.36 μs 1.49 μs 0.63 0.07 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.606978881367238 < 2.3916959770817607.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 11.92918598148379 (T) = (0 -1604.0778990651406) / Math.Sqrt((31129.174572700496 / (284)) + (1031.332494128249 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.08063098352941946 = (1744.7595800249273 - 1604.0778990651406) / 1744.7595800249273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;.DeserializeFromStream

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0808234374516665 < 2.879235292359319.
IsChangePoint: Marked as a change because one of 6/10/2022 5:01:18 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 19.810648899605496 (T) = (0 -2073.753370635517) / Math.Sqrt((32353.220967717167 / (284)) + (401.46311953808384 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.09663543132636282 = (2295.5885614157974 - 2073.753370635517) / 2295.5885614157974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;.DeserializeFromUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.486859131765156 < 2.2440562857490893.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 12.25050501250827 (T) = (0 -1506.3283405839823) / Math.Sqrt((25997.581956418715 / (284)) + (386.7307168724447 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.0761044778149326 = (1630.4098292644896 - 1506.3283405839823) / 1630.4098292644896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SequenceEqual - Duration of single invocation 44.35 ns 31.16 ns 0.70 0.39 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: Array, input2: Array)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.161905301331913 < 46.10668782507257.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.086689797211231 (T) = (0 -32.11784862583382) / Math.Sqrt((24.282640705375464 / (283)) + (3.4621611732562236 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.06373093507887798 = (34.30407970228052 - 32.11784862583382) / 34.30407970228052 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FirstSpanArray - Duration of single invocation 33.82 ns 5.88 ns 0.17 0.02 False
IterateForEachSingleSegment - Duration of single invocation 23.50 ns 21.72 ns 0.92 0.08 False
FirstSpanMemory - Duration of single invocation 35.81 ns 6.87 ns 0.19 0.23 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanArray

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.876851581734715 < 31.679825582816445.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.905857170006765 (T) = (0 -5.876916466057289) / Math.Sqrt((29.76822280676229 / (283)) + (0.0008916771871597347 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.21307579163320517 = (7.468211555283591 - 5.876916466057289) / 7.468211555283591 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;.IterateForEachSingleSegment

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.724378664685354 < 22.05425769049198.
IsChangePoint: Marked as a change because one of 5/27/2022 3:44:45 PM, 7/6/2022 9:44:27 PM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 25.850377978038367 (T) = (0 -21.759111469845298) / Math.Sqrt((0.3601083295077011 / (287)) + (0.10158454656998454 / (32))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (287) + (32) - 2, .975) and 0.07327279322452715 = (23.479521601136167 - 21.759111469845298) / 23.479521601136167 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;.FirstSpanMemory

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.873643592829278 < 34.17290620354105.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.11541981281359 (T) = (0 -6.996928404596379) / Math.Sqrt((33.82618128487826 / (283)) + (0.029149497524036592 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.20232178401637343 = (8.771617758131182 - 6.996928404596379) / 8.771617758131182 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf8 - Duration of single invocation 45.56 ns 20.15 ns 0.44 0.15 False
EncodeUtf8 - Duration of single invocation 185.29 ns 46.37 ns 0.25 0.02 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,16)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.147004858848472 < 43.4219344602158.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.694378732970567 (T) = (0 -19.52286839158148) / Math.Sqrt((28.889004903229168 / (284)) + (0.26919595348962594 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.073676967735467 = (21.075659042888045 - 19.52286839158148) / 21.075659042888045 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+&lt;World&gt;!,16)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.37389320023247 < 175.6479462566705.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.954403285998692 (T) = (0 -46.13492024690672) / Math.Sqrt((856.033104739537 / (284)) + (0.28483360859777657 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.12968747069644934 = (53.009601371389216 - 46.13492024690672) / 53.009601371389216 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsDefinedMethodBaseHit - Duration of single invocation 807.01 ns 703.08 ns 0.87 0.15 False
IsDefinedMethodBaseHitInherit - Duration of single invocation 811.17 ns 728.35 ns 0.90 0.19 False
IsDefinedClassHit - Duration of single invocation 750.49 ns 676.43 ns 0.90 0.23 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Attributes*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Attributes.IsDefinedMethodBaseHit

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 703.079655932406 < 767.0144605367726.
IsChangePoint: Marked as a change because one of 5/1/2022 2:52:05 AM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 14.418265408007086 (T) = (0 -705.2679944234865) / Math.Sqrt((763.0577836188296 / (284)) + (161.21238607665737 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.05232030732239108 = (744.2050303207368 - 705.2679944234865) / 744.2050303207368 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.IsDefinedMethodBaseHitInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 728.3509549943135 < 758.9849139607929.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 14.48149019561413 (T) = (0 -705.5210427590022) / Math.Sqrt((777.8728681399256 / (284)) + (155.2450268587963 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.052114301952383806 = (744.3102519767747 - 705.5210427590022) / 744.3102519767747 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.IsDefinedClassHit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 676.4306685676161 < 714.8036811733281.
IsChangePoint: Marked as a change because one of 6/15/2022 9:01:28 AM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 11.442426997224167 (T) = (0 -683.581139687136) / Math.Sqrt((877.2568562855123 / (283)) + (281.5069387953617 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.05291931707610499 = (721.7770903918508 - 683.581139687136) / 721.7770903918508 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 39.62 μs 19.87 μs 0.50 0.03 False
SerializeToStream - Duration of single invocation 40.37 μs 19.72 μs 0.49 0.02 False
SerializeToString - Duration of single invocation 42.04 μs 22.24 μs 0.53 0.03 False
SerializeObjectProperty - Duration of single invocation 42.65 μs 22.38 μs 0.52 0.06 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToUtf8Bytes

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.86632651515151 < 37.703106093315256.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 14.49810189770805 (T) = (0 -20004.792056504786) / Math.Sqrt((13497146.778119856 / (283)) + (48341.36443140484 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.13833786897848535 = (23216.515309531795 - 20004.792056504786) / 23216.515309531795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToStream

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.719377790178577 < 38.173321822076716.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.0932252910829 (T) = (0 -19996.672732146286) / Math.Sqrt((15924052.075242104 / (283)) + (57391.56768663684 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.057752561343448264 = (21222.315828905164 - 19996.672732146286) / 21222.315828905164 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.24345738823143 < 40.02334051202857.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 22.035128759996347 (T) = (0 -22401.645919842023) / Math.Sqrt((12809586.370376863 / (283)) + (60377.65206064512 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.17574856112522202 = (27178.170232160588 - 22401.645919842023) / 27178.170232160588 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.382510848302847 < 40.54905278313162.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 21.93520313253469 (T) = (0 -22721.355766325054) / Math.Sqrt((13338764.408052323 / (284)) + (81702.80863046933 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1765193799251078 = (27591.85245216656 - 22721.355766325054) / 27591.85245216656 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 14, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NextBytes_unseeded - Duration of single invocation 1.19 μs 253.38 ns 0.21 0.02 False
NextBytes_span_unseeded - Duration of single invocation 1.11 μs 250.22 ns 0.23 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Random.NextBytes_unseeded

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 253.3840202682956 < 1.0569520196471776.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.560146988468234 (T) = (0 -253.10354694074002) / Math.Sqrt((31482.305723383473 / (283)) + (0.2919920149993948 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.12919532065384726 = (290.6547851015039 - 253.10354694074002) / 290.6547851015039 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.NextBytes_span_unseeded

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 250.21842604298357 < 1.0565246816845004.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.5516072341997758 (T) = (0 -251.37072133770687) / Math.Sqrt((30135.22682417423 / (284)) + (0.4248501201611722 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1270571103862397 = (287.9578083841517 - 251.37072133770687) / 287.9578083841517 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare_DifferentFirstChar - Duration of single invocation 49.91 ns 16.91 ns 0.34 0.20 False
Compare_DifferentFirstChar - Duration of single invocation 45.66 ns 15.86 ns 0.35 0.05 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringEquality*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (en-US, Ordinal))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.90584266936153 < 47.49237336367996.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 8.276575590362022 (T) = (0 -16.98871632068658) / Math.Sqrt((39.91381857646167 / (284)) + (0.3582726078986497 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.15907943745287803 = (20.202522185006742 - 16.98871632068658) / 20.202522185006742 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.862303448104017 < 43.28187875559823.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.5601372234247832 (T) = (0 -15.817384238822479) / Math.Sqrt((37.169369227676995 / (284)) + (0.009030907172189885 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.0753640064551665 = (17.106606653048846 - 15.817384238822479) / 17.106606653048846 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 977.24 μs 832.79 μs 0.85 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.JsonNet_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 832.7882951127821 < 925.3276035975797.
IsChangePoint: Marked as a change because one of 6/6/2022 3:15:19 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 25.699940864473675 (T) = (0 -843735.387850266) / Math.Sqrt((886250556.480992 / (284)) + (121822479.22391517 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.07257972205840041 = (909765.9474547274 - 843735.387850266) / 909765.9474547274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 68.01 ns 54.31 ns 0.80 0.06 False
ToString - Duration of single invocation 681.41 ns 405.33 ns 0.59 0.02 False
ToString - Duration of single invocation 342.83 ns 252.70 ns 0.74 0.02 False
ToString - Duration of single invocation 111.97 ns 77.31 ns 0.69 0.01 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_DateTimeOffset*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTimeOffset.ToString(format: "r")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.31404958803944 < 64.75478710114878.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 35.42484355110202 (T) = (0 -52.246086502920036) / Math.Sqrt((17.673628578394645 / (284)) + (0.5661988104899164 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1595654319495578 = (62.16556111454981 - 52.246086502920036) / 62.16556111454981 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTimeOffset.ToString(format: null)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 405.32957835508046 < 642.7711067617148.
IsChangePoint: Marked as a change because one of 6/1/2022 7:59:43 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 21.81855581832623 (T) = (0 -404.80599183868776) / Math.Sqrt((2431.0801116407492 / (284)) + (2.971667121635852 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.13679675288692755 = (468.9579113523209 - 404.80599183868776) / 468.9579113523209 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTimeOffset.ToString(format: "s")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 252.70334033557648 < 325.58543533365537.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 62.238596481378295 (T) = (0 -251.7337579680784) / Math.Sqrt((137.13473569267276 / (284)) + (0.5976157710832684 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.14880218401080977 = (295.7406060488239 - 251.7337579680784) / 295.7406060488239 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTimeOffset.ToString(format: "o")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 77.30709284137873 < 106.42938108565359.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 27.116588613854255 (T) = (0 -76.80453305720893) / Math.Sqrt((48.630307997056974 / (284)) + (0.4416570585620925 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.13148073741771055 = (88.43158277094855 - 76.80453305720893) / 88.43158277094855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLineAsync - Duration of single invocation 473.38 μs 351.80 μs 0.74 0.34 False
ReadLineAsync - Duration of single invocation 216.63 μs 150.43 μs 0.69 0.26 False
ReadLineAsync - Duration of single invocation 759.60 μs 491.45 μs 0.65 0.42 False
ReadLine - Duration of single invocation 44.73 μs 32.24 μs 0.72 0.03 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 1, 1])

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 351.80165438034186 < 447.06297334315116.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 24.080985846540784 (T) = (0 -339382.9890022104) / Math.Sqrt((2206829510.6080976 / (228)) + (240132854.57662728 / (35))) is greater than 1.9690947240420478 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (35) - 2, .975) and 0.22394631227419662 = (437318.9566262608 - 339382.9890022104) / 437318.9566262608 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 150.42906842456608 < 205.77997555415476.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 28.66573699540755 (T) = (0 -149913.9121536007) / Math.Sqrt((478159507.47811085 / (228)) + (22160052.391572684 / (35))) is greater than 1.9690947240420478 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (35) - 2, .975) and 0.24009724447194283 = (197280.39023812377 - 149913.9121536007) / 197280.39023812377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 491.4523558872768 < 745.480400645633.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 21.805712942589295 (T) = (0 -518886.86810293834) / Math.Sqrt((5428810512.099262 / (228)) + (768395082.7680514 / (35))) is greater than 1.9690947240420478 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (228) + (35) - 2, .975) and 0.2213602256835356 = (666401.6984727599 - 518886.86810293834) / 666401.6984727599 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.23698402033417 < 41.609661873094744.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 38.28108831547064 (T) = (0 -31883.358238280187) / Math.Sqrt((9419073.378087912 / (283)) + (55580.12218790462 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.18314568938697134 = (39031.87854190612 - 31883.358238280187) / 39031.87854190612 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Char_IsLower - Duration of single invocation 392.09 ns 31.12 ns 0.08 0.03 False
Char_IsUpper - Duration of single invocation 372.57 ns 28.35 ns 0.08 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Char.Char_IsLower(input: "Good afternoon, Constable!")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.12250542746434 < 403.1589338875387.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.031122553650638 (T) = (0 -31.292658939733258) / Math.Sqrt((5826.433413259923 / (284)) + (0.06448270504957157 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.36848988413411976 = (49.55211033607445 - 31.292658939733258) / 49.55211033607445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Char.Char_IsUpper(input: "Good afternoon, Constable!")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.354746537297636 < 373.02484963501655.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.020719924634611 (T) = (0 -27.888587116095266) / Math.Sqrt((5488.706002407317 / (284)) + (0.22182684963300037 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.3879693926048908 = (45.567307874998484 - 27.888587116095266) / 45.567307874998484 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseHex - Duration of single invocation 296.60 ns 32.35 ns 0.11 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParseHex(value: "FFFFFFFFFFFFFFFF")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.35272998315809 < 281.76385851081517.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.4658242962995462 (T) = (0 -32.18672232094927) / Math.Sqrt((2856.18968493124 / (283)) + (0.08116811639027631 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.25491080758820933 = (43.19848234110547 - 32.18672232094927) / 43.19848234110547 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 159.65 μs 46.63 μs 0.29 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.XmlSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.634915999503505 < 151.95277372708972.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.190049504651382 (T) = (0 -46462.77851827142) / Math.Sqrt((504432388.80344564 / (284)) + (230879.81345704081 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.12978907603311096 = (53392.547988790015 - 46462.77851827142) / 53392.547988790015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArraySetAll - Duration of single invocation 11.46 ns 5.51 ns 0.48 0.45 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 4)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.50883534213872 < 10.884423715206008.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.3349716483224023 (T) = (0 -5.482682531257852) / Math.Sqrt((1.5084091651477238 / (283)) + (0.19556947890574228 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.05975923594699178 = (5.831147447409283 - 5.482682531257852) / 5.831147447409283 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 5.30 μs 3.03 μs 0.57 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<Location>.XmlSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.034248182432905 < 5.113665552934357.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 37.68007706664736 (T) = (0 -3025.011281498323) / Math.Sqrt((318899.36399686075 / (284)) + (1291.9955181710113 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.2978540882925029 = (4308.237406299241 - 3025.011281498323) / 4308.237406299241 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Memory.Constructors<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SpanFromMemory - Duration of single invocation 12.06 ns 4.52 ns 0.37 0.26 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Constructors&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Constructors<String>.SpanFromMemory

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.516614579481816 < 11.372039604805833.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 2.467183597513558 (T) = (0 -4.399992866305958) / Math.Sqrt((2.608032415478688 / (284)) + (0.22590707118581616 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.06552452056693345 = (4.708516128187091 - 4.399992866305958) / 4.708516128187091 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseInt16 - Duration of single invocation 17.33 ns 6.02 ns 0.35 0.40 False
TryParseUInt16 - Duration of single invocation 17.90 ns 4.34 ns 0.24 0.40 False
TryParseUInt32Hex - Duration of single invocation 21.50 ns 3.62 ns 0.17 0.48 False
TryParseSByte - Duration of single invocation 16.04 ns 4.07 ns 0.25 0.37 False
TryParseInt16 - Duration of single invocation 14.95 ns 3.21 ns 0.21 0.53 False
TryParseUInt32Hex - Duration of single invocation 23.81 ns 6.70 ns 0.28 0.28 False
TryParseByte - Duration of single invocation 16.34 ns 2.63 ns 0.16 0.56 False
TryParseUInt16 - Duration of single invocation 14.36 ns 3.05 ns 0.21 0.49 False
TryParseUInt32Hex - Duration of single invocation 19.57 ns 1.89 ns 0.10 0.68 False
TryParseByte - Duration of single invocation 14.37 ns 3.20 ns 0.22 0.47 False
TryParseInt16 - Duration of single invocation 17.06 ns 5.51 ns 0.32 0.37 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: -32768)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.0194047631583505 < 16.475529425460376.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.6119769106727078 (T) = (0 -5.809141527767812) / Math.Sqrt((5.203012259925605 / (284)) + (0.02860099120974016 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.07920290377758604 = (6.308818252794145 - 5.809141527767812) / 6.308818252794145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.338669747678854 < 17.03105624978978.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.426272781670929 (T) = (0 -4.318389367794551) / Math.Sqrt((7.430900556735697 / (284)) + (0.030819160294260736 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.11542122695816873 = (4.881859591706862 - 4.318389367794551) / 4.881859591706862 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: 3039)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.6165712363694125 < 20.38161111765032.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.265256165502719 (T) = (0 -3.6066586604936366) / Math.Sqrt((12.995274190083565 / (283)) + (0.028331383955274993 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.20359535734342438 = (4.5286760866471925 - 3.6066586604936366) / 4.5286760866471925 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSByte(value: -128)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.072111770541697 < 15.470285249392264.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.9076591244452925 (T) = (0 -3.9772405715193386) / Math.Sqrt((6.166727512541839 / (284)) + (0.03307358404712394 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.12884009484560147 = (4.56545411237038 - 3.9772405715193386) / 4.56545411237038 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 0)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.207902625044429 < 14.20467731302812.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 2.51819200039436 (T) = (0 -3.173322060083919) / Math.Sqrt((5.9842088239768545 / (284)) + (0.035466239878135124 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.10548829996406824 = (3.5475467341080607 - 3.173322060083919) / 3.5475467341080607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: FFFFFFFFFFFFFFFF)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.696110466136905 < 22.62253203253618.
IsChangePoint: Marked as a change because one of 5/11/2022 7:18:06 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 6.6763371048496705 (T) = (0 -6.603345235191741) / Math.Sqrt((13.606332589354038 / (283)) + (0.030199796759726193 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.1827883858660481 = (8.08033699103714 - 6.603345235191741) / 8.08033699103714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseByte(value: 255)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.630023450301911 < 15.534194073719934.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.7743602373771026 (T) = (0 -2.9665629838621284) / Math.Sqrt((7.326424961218627 / (284)) + (0.04969872623223828 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.1734851923506877 = (3.5892436002439196 - 2.9665629838621284) / 3.5892436002439196 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 0)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.0498227814814896 < 13.588748309654463.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.037384649396947 (T) = (0 -3.0718313131306396) / Math.Sqrt((5.411634779539164 / (284)) + (0.038188038210716535 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.12307275072944822 = (3.5029488656965095 - 3.0718313131306396) / 3.5029488656965095 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: 0)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8859067009269805 < 18.5903275885372.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.0324206720533042 (T) = (0 -1.74676918613975) / Math.Sqrt((13.214687741830804 / (283)) + (0.03498292591175839 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.2749052054188047 = (2.4090218261029714 - 1.74676918613975) / 2.4090218261029714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseByte(value: 0)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1969253073730046 < 13.545829999293236.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 2.9665312181093237 (T) = (0 -3.0525645312620004) / Math.Sqrt((5.379202175842883 / (284)) + (0.04108359205004412 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.12113329563354834 = (3.4732963668961627 - 3.0525645312620004) / 3.4732963668961627 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 32767)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.5113855108650345 < 16.41129350656583.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 4.094800910520705 (T) = (0 -5.3236868858874065) / Math.Sqrt((5.822902037584398 / (284)) + (0.032484134437493296 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.10120566303094965 = (5.9231424441771106 - 5.3236868858874065) / 5.9231424441771106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetPathRoot - Duration of single invocation 13.21 ns 1.36 ns 0.10 0.55 False
IsPathRooted - Duration of single invocation 11.01 ns 0.15 ns 0.01 0.82 False
HasExtension - Duration of single invocation 15.09 ns 4.69 ns 0.31 0.36 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetPathRoot

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3604545752300965 < 12.527459092157313.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.8189767015155613 (T) = (0 -1.269175862356775) / Math.Sqrt((5.948229843831338 / (284)) + (0.0519521766789302 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.3106484378012458 = (1.8411155235633536 - 1.269175862356775) / 1.8411155235633536 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.IsPathRooted

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.15208128122698927 < 10.49359145612199.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.7948140611228083 (T) = (0 -0.08482919081253873) / Math.Sqrt((4.811037753356188 / (284)) + (0.004807251922124981 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.8539293268923303 = (0.5807407401348149 - 0.08482919081253873) / 0.5807407401348149 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.HasExtension

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.689368699880782 < 14.75262532548849.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.7919186846051196 (T) = (0 -4.188628202588373) / Math.Sqrt((5.219800831257169 / (284)) + (0.07084901386263742 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.11450612013882935 = (4.7302734641656405 - 4.188628202588373) / 4.7302734641656405 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromStream - Duration of single invocation 58.68 μs 48.10 μs 0.82 0.13 False
DeserializeFromString - Duration of single invocation 58.55 μs 45.74 μs 0.78 0.19 False
DeserializeFromUtf8Bytes - Duration of single invocation 53.40 μs 45.87 μs 0.86 0.16 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ArrayList&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<ArrayList>.DeserializeFromStream

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.09616527227576 < 55.253438848701634.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 24.04253439471575 (T) = (0 -45248.49339257689) / Math.Sqrt((10937396.40166521 / (283)) + (1730986.6102790784 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.13623034607585244 = (52384.90746579344 - 45248.49339257689) / 52384.90746579344 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;ArrayList&gt;.DeserializeFromString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.74183015070191 < 54.69855811666319.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 23.302153868602343 (T) = (0 -44054.10522795945) / Math.Sqrt((11703650.010262078 / (283)) + (1960377.1297231796 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.14166739603295084 = (51325.214752824024 - 44054.10522795945) / 51325.214752824024 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;ArrayList&gt;.DeserializeFromUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.868384331854045 < 53.68579590462958.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 24.916164788027928 (T) = (0 -44361.14443383196) / Math.Sqrt((8981012.619154397 / (283)) + (1258865.9056019876 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.12750933820789292 = (50844.26272564752 - 44361.14443383196) / 50844.26272564752 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromStream - Duration of single invocation 1.47 μs 1.03 μs 0.71 0.07 False
DeserializeFromUtf8Bytes - Duration of single invocation 876.70 ns 557.92 ns 0.64 0.11 False
DeserializeFromString - Duration of single invocation 1.05 μs 668.40 ns 0.64 0.18 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromStream

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0338711760588735 < 1.3704201047918694.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 18.199233430152262 (T) = (0 -1041.177342006289) / Math.Sqrt((5377.019808810913 / (284)) + (241.61015279496394 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.08159747954559905 = (1133.6830189568104 - 1041.177342006289) / 1133.6830189568104 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;LoginViewModel&gt;.DeserializeFromUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 557.9180416063857 < 820.1496463315689.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 15.94292159369403 (T) = (0 -561.0777040856891) / Math.Sqrt((3016.7820807682524 / (284)) + (114.25512403085087 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.09574947250605478 = (620.4892195536441 - 561.0777040856891) / 620.4892195536441 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;LoginViewModel&gt;.DeserializeFromString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 668.3959173783029 < 0.9699455232926771.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 14.729231630108679 (T) = (0 -645.6966526911564) / Math.Sqrt((4888.660196390255 / (284)) + (202.88948651943232 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.09863164480560124 = (716.3515880828749 - 645.6966526911564) / 716.3515880828749 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TrimStart - Duration of single invocation 84.25 ns 12.75 ns 0.15 0.15 False
TrimEnd - Duration of single invocation 97.76 ns 12.55 ns 0.13 0.21 False
LastIndexOf - Duration of single invocation 20.22 ns 3.84 ns 0.19 0.23 False
Trim - Duration of single invocation 185.40 ns 27.67 ns 0.15 0.02 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimStart

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.75373630152347 < 78.68116094057696.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.5494348548339043 (T) = (0 -12.596236778164844) / Math.Sqrt((198.1448150165848 / (283)) + (0.1773009227390237 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.19135502025777318 = (15.576967759300464 - 12.596236778164844) / 15.576967759300464 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimEnd

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.55116363634251 < 89.85679094447286.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.4730877391167776 (T) = (0 -12.42944175146572) / Math.Sqrt((288.4674273680593 / (284)) + (0.15336123309389987 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.22010275276505814 = (15.937281219459653 - 12.42944175146572) / 15.937281219459653 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.8406179336906585 < 19.304058472496575.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.7783346969214398 (T) = (0 -4.023154116020566) / Math.Sqrt((11.4642826795904 / (283)) + (1.2453254879879723 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.20575621987361567 = (5.065389514766336 - 4.023154116020566) / 5.065389514766336 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Trim

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.67486598276519 < 175.88058938058288.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 3.4836439886702446 (T) = (0 -27.781775955256823) / Math.Sqrt((1008.9713797576184 / (283)) + (0.11801203723744036 / (35))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (283) + (35) - 2, .975) and 0.19151300489879886 = (34.36267512476101 - 27.781775955256823) / 34.36267512476101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline a33b9d82d45edd2ca47db710ff06a87aa0eb9cbd
Compare 3cf8f49b111ebb809607555bba70465f67157533
Diff Diff

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 961.39 μs 403.96 μs 0.42 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.XmlSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 403.96354174757283 < 911.8054430729212.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/14/2022 8:56:55 AM falls between 7/5/2022 7:51:46 PM and 7/14/2022 8:56:55 AM.
IsImprovementStdDev: Marked as improvement because 5.863073247869298 (T) = (0 -406824.6990649004) / Math.Sqrt((11804093065.433044 / (284)) + (8118000.275898465 / (35))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (284) + (35) - 2, .975) and 0.08523053484523434 = (444729.2072610574 - 406824.6990649004) / 444729.2072610574 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added ampere arm64 PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Jul 14, 2022
@EgorBo
Copy link
Member

EgorBo commented Jul 14, 2022

dotnet/runtime#71673

@EgorBo EgorBo closed this as completed Jul 14, 2022
@EgorBo EgorBo removed the untriaged label Jul 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tannergooding tannergooding

Labels
ampere arch-arm64 branch-refs/heads/main kind-micro os-linux perf-improvement PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@EgorBo @tannergooding