[Perf] Windows/x64: 3 Regressions on 5/19/2023 3:32:16 PM #18059
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 2f49fcff6df15a200ef01eea16b3ce7930f75c5c |
| Compare | 558345d16cf76525d0c7fdbafbfd3a2457142b39 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Text.Json.Tests.Perf_Get
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| GetUInt64 - Duration of single invocation | 614.84 ns | 766.25 ns | 1.25 | 0.01 | True | 7338.256133203437 | 9271.546625347795 | 1.2634536676086832) | Trace | Trace |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'Details
Payloads
Histogram
System.Text.Json.Tests.Perf_Get.GetUInt64
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 766.2534440344403 > 645.5402299868969.
IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/24/2023 7:12:11 AM falls between 5/15/2023 6:46:12 PM and 5/24/2023 7:12:11 AM.
IsRegressionStdDev: Marked as regression because -18.341096416029387 (T) = (0 -766.2598122571278) / Math.Sqrt((913.6933998153497 / (23)) + (1.17732008444234 / (19))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (19) - 2, .025) and -0.1778307697970317 = (650.5686826208256 - 766.2598122571278) / 650.5686826208256 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 2f49fcff6df15a200ef01eea16b3ce7930f75c5c |
| Compare | 558345d16cf76525d0c7fdbafbfd3a2457142b39 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in Benchstone.BenchI.BubbleSort2
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Test - Duration of single invocation | 32.35 ms | 40.84 ms | 1.26 | 0.02 | True | 269261111.1111111 | 298561904.7619048 | 1.1088192555170087) | Trace | Trace |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BubbleSort2*'Details
Payloads
Histogram
Benchstone.BenchI.BubbleSort2.Test
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.837305000000015 > 34.26345795833334.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/24/2023 7:12:11 AM falls between 5/15/2023 6:46:12 PM and 5/24/2023 7:12:11 AM.
IsRegressionStdDev: Marked as regression because -23.07603956640422 (T) = (0 -39371830.70464623) / Math.Sqrt((57088264505.41299 / (25)) + (1602914772638.8525 / (19))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (19) - 2, .025) and -0.2084966317363877 = (32579181.16666667 - 39371830.70464623) / 32579181.16666667 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 2f49fcff6df15a200ef01eea16b3ce7930f75c5c |
| Compare | 558345d16cf76525d0c7fdbafbfd3a2457142b39 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in LinqBenchmarks
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Count00ForX - Duration of single invocation | 122.84 ms | 164.00 ms | 1.34 | 0.02 | True | 1822333333.3333333 | 1748066666.6666667 | 0.9592463874154016) | Trace | Trace |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'Details
Payloads
Histogram
LinqBenchmarks.Count00ForX
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 164.00102142857142 > 128.978873625.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/24/2023 7:12:11 AM falls between 5/15/2023 6:46:12 PM and 5/24/2023 7:12:11 AM.
IsRegressionStdDev: Marked as regression because -361.33385203591956 (T) = (0 -163627211.76016963) / Math.Sqrt((211418211081.15118 / (23)) + (65157448937.04379 / (19))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (19) - 2, .025) and -0.3299446236898854 = (123033101.41304348 - 163627211.76016963) / 123033101.41304348 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository