-
Notifications
You must be signed in to change notification settings - Fork 3
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 63e677701d35c7e2dad4fe74a84eb935cd396155 |
| Compare | 6e91162e6ce882f476d4548aedf6a1efaa75e547 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Memory.Span<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IndexOfAnyFourValues - Duration of single invocation | 7.02 ns | 5.66 ns | 0.81 | 0.21 | False | |||||
| SequenceEqual - Duration of single invocation | 6.09 ns | 2.99 ns | 0.49 | 0.32 | False | |||||
| SequenceCompareTo - Duration of single invocation | 29.18 ns | 26.53 ns | 0.91 | 0.02 | False | |||||
| SequenceCompareTo - Duration of single invocation | 436.83 ns | 355.38 ns | 0.81 | 0.00 | True | |||||
| SequenceEqual - Duration of single invocation | 7.68 ns | 4.74 ns | 0.62 | 0.35 | False | |||||
| Fill - Duration of single invocation | 3.13 ns | 1.82 ns | 0.58 | 0.09 | False | |||||
| LastIndexOfValue - Duration of single invocation | 3.06 ns | 1.86 ns | 0.61 | 0.04 | True | |||||
| SequenceEqual - Duration of single invocation | 43.83 ns | 37.05 ns | 0.85 | 0.12 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Int32>*'Details
Payloads
Histogram
System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 4)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.6599411782122075 < 6.621252992696526.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/19/2023 8:38:27 PM, 5/10/2023 6:05:58 AM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.04039010697924 (T) = (0 -5.628255931385413) / Math.Sqrt((0.2585540196310834 / (299)) + (0.002962508492719604 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09534345702376602 = (6.221428425056194 - 5.628255931385413) / 6.221428425056194 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Int32>.SequenceEqual(Size: 4)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.9940499166511283 < 5.902637192445108.
IsChangePoint: Marked as a change because one of 3/23/2023 12:23:24 PM, 3/29/2023 12:02:44 PM, 4/10/2023 4:31:44 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.618913271180089 (T) = (0 -3.3841069286153096) / Math.Sqrt((1.454867551249301 / (299)) + (0.1881910292619375 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.26984748938894876 = (4.634794620898054 - 3.3841069286153096) / 4.634794620898054 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Int32>.SequenceCompareTo(Size: 33)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.532030096407997 < 27.702661289869116.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/16/2023 5:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.773393589175345 (T) = (0 -27.138324113327958) / Math.Sqrt((0.8670642521524421 / (299)) + (0.1923162577690232 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12734810147116837 = (31.09868225701377 - 27.138324113327958) / 31.09868225701377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Int32>.SequenceCompareTo(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 355.3830688439129 < 414.9644648397839.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 623.8774369546351 (T) = (0 -354.929935680673) / Math.Sqrt((3.339692212105521 / (299)) + (0.07533088127270908 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1862913045259352 = (436.18795971436884 - 354.929935680673) / 436.18795971436884 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Int32>.SequenceEqual(Size: 33)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.738882333006298 < 7.315670990088212.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/29/2023 12:02:44 PM, 4/9/2023 11:04:55 PM, 5/16/2023 5:40:18 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.100504829079842 (T) = (0 -4.815711091208273) / Math.Sqrt((1.7419063506072818 / (299)) + (0.007932771814854286 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.19763703110672026 = (6.001910952907896 - 4.815711091208273) / 6.001910952907896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Int32>.Fill(Size: 4)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8197413716515651 < 3.007127463083304.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/9/2023 11:04:55 PM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.94594463383437 (T) = (0 -1.8133884874559056) / Math.Sqrt((0.1248544268406225 / (299)) + (0.001312429696164366 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36077696177442853 = (2.8368634717699117 - 1.8133884874559056) / 2.8368634717699117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Int32>.LastIndexOfValue(Size: 4)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8626378897463833 < 2.91042405726533.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 110.23734522591842 (T) = (0 -1.8623843694063198) / Math.Sqrt((0.031695072521193426 / (299)) + (2.6049796427602407E-07 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.37868201428576603 = (2.9974737770795774 - 1.8623843694063198) / 2.9974737770795774 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Int32>.SequenceEqual(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 37.0456129883448 < 41.28232492809811.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/29/2023 12:02:44 PM, 4/9/2023 11:04:55 PM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/16/2023 5:40:18 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 2.5058266492423407 (T) = (0 -34.488556594489104) / Math.Sqrt((22.040473759909975 / (299)) + (12.659432151070014 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.053331048288362874 = (36.43148592982966 - 34.488556594489104) / 36.43148592982966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 63e677701d35c7e2dad4fe74a84eb935cd396155 |
| Compare | 6e91162e6ce882f476d4548aedf6a1efaa75e547 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Tests.Perf_StringBuilder
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Append_NonEmptySpan - Duration of single invocation | 308.59 ns | 276.70 ns | 0.90 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'Details
Payloads
Histogram
System.Text.Tests.Perf_StringBuilder.Append_NonEmptySpan
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 276.70074044951184 < 287.10703446261266.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.563809427484543 (T) = (0 -284.94745317949145) / Math.Sqrt((61.60522537931382 / (299)) + (54.0508836397245 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.05140221531379828 = (300.3880651837625 - 284.94745317949145) / 300.3880651837625 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 63e677701d35c7e2dad4fe74a84eb935cd396155 |
| Compare | 6e91162e6ce882f476d4548aedf6a1efaa75e547 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.IterateFor<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ImmutableSortedSet - Duration of single invocation | 6.74 μs | 5.63 μs | 0.84 | 0.06 | False | |||||
| IList - Duration of single invocation | 2.15 μs | 2.01 μs | 0.93 | 0.11 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor<String>*'Details
Payloads
Histogram
System.Collections.IterateFor<String>.ImmutableSortedSet(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.630817255187477 < 6.400514149959824.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/19/2023 8:38:27 PM, 5/1/2023 3:42:23 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.958115734773795 (T) = (0 -5648.741994888761) / Math.Sqrt((136292.3546218913 / (299)) + (7945.6199593077745 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10341652254979532 = (6300.296778782081 - 5648.741994888761) / 6300.296778782081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.IterateFor<String>.IList(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.006534302999099 < 2.040789968120117.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 56.355833285894704 (T) = (0 -2006.6810074736572) / Math.Sqrt((2400.463541502011 / (299)) + (0.09517012134579354 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07374003064815411 = (2166.4339104256446 - 2006.6810074736572) / 2166.4339104256446 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 63e677701d35c7e2dad4fe74a84eb935cd396155 |
| Compare | 6e91162e6ce882f476d4548aedf6a1efaa75e547 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Char
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Char_IsUpper - Duration of single invocation | 24.57 ns | 16.71 ns | 0.68 | 0.07 | False | |||||
| GetUnicodeCategory - Duration of single invocation | 3.62 ns | 1.69 ns | 0.47 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'Details
Payloads
Histogram
System.Tests.Perf_Char.Char_IsUpper(input: "Good afternoon, Constable!")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.712411109571285 < 23.381415264526034.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.829916612325352 (T) = (0 -16.540938858575437) / Math.Sqrt((12.975144391443985 / (299)) + (0.3363831018423959 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.245219186674383 = (21.91489047753467 - 16.540938858575437) / 21.91489047753467 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Char.GetUnicodeCategory(c: 'א')
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.694038829009963 < 3.4391732424477808.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 109.07133739348707 (T) = (0 -1.6943231705076809) / Math.Sqrt((0.0908635030914469 / (299)) + (4.2954013478790214E-07 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5288062975215837 = (3.5958103039063682 - 1.6943231705076809) / 3.5958103039063682 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 63e677701d35c7e2dad4fe74a84eb935cd396155 |
| Compare | 6e91162e6ce882f476d4548aedf6a1efaa75e547 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in Benchstone.BenchI.BenchE
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Test - Duration of single invocation | 319.59 ms | 246.61 ms | 0.77 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BenchE*'Details
Payloads
Histogram
Benchstone.BenchI.BenchE.Test
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 246.61142666666666 < 303.5918576071429.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/25/2023 3:46:39 AM, 5/2/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.313213616570712 (T) = (0 -254257748.0370225) / Math.Sqrt((724295281528950.5 / (299)) + (72203021271429.62 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09771728474328165 = (281793880.9397239 - 254257748.0370225) / 281793880.9397239 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 63e677701d35c7e2dad4fe74a84eb935cd396155 |
| Compare | 6e91162e6ce882f476d4548aedf6a1efaa75e547 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Array
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ArrayRetrieve1D - Duration of single invocation | 1.79 ms | 1.54 ms | 0.86 | 0.14 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'Details
Payloads
Histogram
System.Tests.Perf_Array.ArrayRetrieve1D
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.5442870192307692 < 1.6866743553240737.
IsChangePoint: Marked as a change because one of 3/18/2023 3:39:51 AM, 4/24/2023 11:08:13 AM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 13.846709231004123 (T) = (0 -1631359.351224393) / Math.Sqrt((8597491513.66714 / (299)) + (2124620790.628447 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08086608697613729 = (1774887.5632902898 - 1631359.351224393) / 1774887.5632902898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository













