Skip to content

[Perf] Windows/x64: 15 Improvements on 5/16/2023 7:34:13 AM #18214

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 63e677701d35c7e2dad4fe74a84eb935cd396155
Compare 6e91162e6ce882f476d4548aedf6a1efaa75e547
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFourValues - Duration of single invocation 7.02 ns 5.66 ns 0.81 0.21 False
SequenceEqual - Duration of single invocation 6.09 ns 2.99 ns 0.49 0.32 False
SequenceCompareTo - Duration of single invocation 29.18 ns 26.53 ns 0.91 0.02 False
SequenceCompareTo - Duration of single invocation 436.83 ns 355.38 ns 0.81 0.00 True
SequenceEqual - Duration of single invocation 7.68 ns 4.74 ns 0.62 0.35 False
Fill - Duration of single invocation 3.13 ns 1.82 ns 0.58 0.09 False
LastIndexOfValue - Duration of single invocation 3.06 ns 1.86 ns 0.61 0.04 True
SequenceEqual - Duration of single invocation 43.83 ns 37.05 ns 0.85 0.12 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 4)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.6599411782122075 < 6.621252992696526.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/19/2023 8:38:27 PM, 5/10/2023 6:05:58 AM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.04039010697924 (T) = (0 -5.628255931385413) / Math.Sqrt((0.2585540196310834 / (299)) + (0.002962508492719604 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09534345702376602 = (6.221428425056194 - 5.628255931385413) / 6.221428425056194 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceEqual(Size: 4)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.9940499166511283 < 5.902637192445108.
IsChangePoint: Marked as a change because one of 3/23/2023 12:23:24 PM, 3/29/2023 12:02:44 PM, 4/10/2023 4:31:44 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.618913271180089 (T) = (0 -3.3841069286153096) / Math.Sqrt((1.454867551249301 / (299)) + (0.1881910292619375 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.26984748938894876 = (4.634794620898054 - 3.3841069286153096) / 4.634794620898054 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceCompareTo(Size: 33)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.532030096407997 < 27.702661289869116.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/16/2023 5:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.773393589175345 (T) = (0 -27.138324113327958) / Math.Sqrt((0.8670642521524421 / (299)) + (0.1923162577690232 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12734810147116837 = (31.09868225701377 - 27.138324113327958) / 31.09868225701377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceCompareTo(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 355.3830688439129 < 414.9644648397839.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 623.8774369546351 (T) = (0 -354.929935680673) / Math.Sqrt((3.339692212105521 / (299)) + (0.07533088127270908 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1862913045259352 = (436.18795971436884 - 354.929935680673) / 436.18795971436884 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceEqual(Size: 33)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.738882333006298 < 7.315670990088212.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/29/2023 12:02:44 PM, 4/9/2023 11:04:55 PM, 5/16/2023 5:40:18 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.100504829079842 (T) = (0 -4.815711091208273) / Math.Sqrt((1.7419063506072818 / (299)) + (0.007932771814854286 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.19763703110672026 = (6.001910952907896 - 4.815711091208273) / 6.001910952907896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.Fill(Size: 4)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8197413716515651 < 3.007127463083304.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/9/2023 11:04:55 PM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.94594463383437 (T) = (0 -1.8133884874559056) / Math.Sqrt((0.1248544268406225 / (299)) + (0.001312429696164366 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36077696177442853 = (2.8368634717699117 - 1.8133884874559056) / 2.8368634717699117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.LastIndexOfValue(Size: 4)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8626378897463833 < 2.91042405726533.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 110.23734522591842 (T) = (0 -1.8623843694063198) / Math.Sqrt((0.031695072521193426 / (299)) + (2.6049796427602407E-07 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.37868201428576603 = (2.9974737770795774 - 1.8623843694063198) / 2.9974737770795774 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceEqual(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 37.0456129883448 < 41.28232492809811.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/29/2023 12:02:44 PM, 4/9/2023 11:04:55 PM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/16/2023 5:40:18 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 2.5058266492423407 (T) = (0 -34.488556594489104) / Math.Sqrt((22.040473759909975 / (299)) + (12.659432151070014 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.053331048288362874 = (36.43148592982966 - 34.488556594489104) / 36.43148592982966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 63e677701d35c7e2dad4fe74a84eb935cd396155
Compare 6e91162e6ce882f476d4548aedf6a1efaa75e547
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Append_NonEmptySpan - Duration of single invocation 308.59 ns 276.70 ns 0.90 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.Append_NonEmptySpan


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 276.70074044951184 < 287.10703446261266.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.563809427484543 (T) = (0 -284.94745317949145) / Math.Sqrt((61.60522537931382 / (299)) + (54.0508836397245 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.05140221531379828 = (300.3880651837625 - 284.94745317949145) / 300.3880651837625 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 63e677701d35c7e2dad4fe74a84eb935cd396155
Compare 6e91162e6ce882f476d4548aedf6a1efaa75e547
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedSet - Duration of single invocation 6.74 μs 5.63 μs 0.84 0.06 False
IList - Duration of single invocation 2.15 μs 2.01 μs 0.93 0.11 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<String>.ImmutableSortedSet(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.630817255187477 < 6.400514149959824.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/19/2023 8:38:27 PM, 5/1/2023 3:42:23 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.958115734773795 (T) = (0 -5648.741994888761) / Math.Sqrt((136292.3546218913 / (299)) + (7945.6199593077745 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10341652254979532 = (6300.296778782081 - 5648.741994888761) / 6300.296778782081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateFor<String>.IList(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.006534302999099 < 2.040789968120117.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 56.355833285894704 (T) = (0 -2006.6810074736572) / Math.Sqrt((2400.463541502011 / (299)) + (0.09517012134579354 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07374003064815411 = (2166.4339104256446 - 2006.6810074736572) / 2166.4339104256446 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 63e677701d35c7e2dad4fe74a84eb935cd396155
Compare 6e91162e6ce882f476d4548aedf6a1efaa75e547
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Char_IsUpper - Duration of single invocation 24.57 ns 16.71 ns 0.68 0.07 False
GetUnicodeCategory - Duration of single invocation 3.62 ns 1.69 ns 0.47 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Char.Char_IsUpper(input: "Good afternoon, Constable!")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.712411109571285 < 23.381415264526034.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.829916612325352 (T) = (0 -16.540938858575437) / Math.Sqrt((12.975144391443985 / (299)) + (0.3363831018423959 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.245219186674383 = (21.91489047753467 - 16.540938858575437) / 21.91489047753467 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.GetUnicodeCategory(c: 'א')


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.694038829009963 < 3.4391732424477808.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 109.07133739348707 (T) = (0 -1.6943231705076809) / Math.Sqrt((0.0908635030914469 / (299)) + (4.2954013478790214E-07 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5288062975215837 = (3.5958103039063682 - 1.6943231705076809) / 3.5958103039063682 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 63e677701d35c7e2dad4fe74a84eb935cd396155
Compare 6e91162e6ce882f476d4548aedf6a1efaa75e547
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.BenchE

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 319.59 ms 246.61 ms 0.77 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BenchE*'
Details

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.BenchE.Test


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 246.61142666666666 < 303.5918576071429.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/25/2023 3:46:39 AM, 5/2/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.313213616570712 (T) = (0 -254257748.0370225) / Math.Sqrt((724295281528950.5 / (299)) + (72203021271429.62 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09771728474328165 = (281793880.9397239 - 254257748.0370225) / 281793880.9397239 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 63e677701d35c7e2dad4fe74a84eb935cd396155
Compare 6e91162e6ce882f476d4548aedf6a1efaa75e547
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayRetrieve1D - Duration of single invocation 1.79 ms 1.54 ms 0.86 0.14 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.ArrayRetrieve1D


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.5442870192307692 < 1.6866743553240737.
IsChangePoint: Marked as a change because one of 3/18/2023 3:39:51 AM, 4/24/2023 11:08:13 AM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 13.846709231004123 (T) = (0 -1631359.351224393) / Math.Sqrt((8597491513.66714 / (299)) + (2124620790.628447 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08086608697613729 = (1774887.5632902898 - 1631359.351224393) / 1774887.5632902898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions