I have been playing around with --launchCount and noticed this when looking at the data:
BenchmarkDotNet v0.13.7-nightly.20230717.35, Windows 11 (10.0.22621.2283/22H2/2022Update/SunValley2)
Intel Core i7-8700 CPU 3.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET SDK 8.0.100-preview.7.23376.3
[Host] : .NET 6.0.22 (6.0.2223.42425), X64 RyuJIT AVX2
Job-QYHTSN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
EnvironmentVariables=DOTNET_JitCSEHash=26ab038f,DOTNET_JitDisasmSummary=1,DOTNET_JitStdOutFile=D:\bugs\cse\base.4a.list,DOTNET_JitCSEMask=4 PowerPlanMode=00000000-0000-0000-0000-000000000000 Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true
Toolchain=CoreRun IterationTime=250.0000 ms LaunchCount=100
MaxIterationCount=20 MinIterationCount=15 WarmupCount=1
| Method |
Count |
Mean |
Error |
StdDev |
Median |
Min |
Max |
Gen0 |
Gen1 |
Allocated |
| Dictionary |
512 |
13.71 μs |
0.052 μs |
0.598 μs |
13.82 μs |
12.44 μs |
17.10 μs |
7.6273 |
0.7133 |
46.97 KB |
// * Hints *
Outliers
TryAddDefaultSize.Dictionary: EnvironmentVariables=DOTNET_JitCSEHash=26ab038f,DOTNET_JitDisasmSummary=1,DOTNET_JitStdOutFile=D:\bugs\cse\base.4a.list,DOTNET_JitCSEMask=4, PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, Toolchain=CoreRun, IterationTime=250.0000 ms, LaunchCount=100, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1 -> Something went wrong with outliers: Size(WorkloadActual) = 1548, Size(WorkloadActual/Outliers) = 34, Size(Result) = 1467), OutlierMode = RemoveUpper
This was running --filter "System.Collections.TryAddDefaultSize<String>.Dictionary" from dotnet/performance using their config setup and a locally built corerun. But I suspect it will repro with a normal build.
I have been playing around with
--launchCountand noticed this when looking at the data:BenchmarkDotNet v0.13.7-nightly.20230717.35, Windows 11 (10.0.22621.2283/22H2/2022Update/SunValley2)
Intel Core i7-8700 CPU 3.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET SDK 8.0.100-preview.7.23376.3
[Host] : .NET 6.0.22 (6.0.2223.42425), X64 RyuJIT AVX2
Job-QYHTSN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
EnvironmentVariables=DOTNET_JitCSEHash=26ab038f,DOTNET_JitDisasmSummary=1,DOTNET_JitStdOutFile=D:\bugs\cse\base.4a.list,DOTNET_JitCSEMask=4 PowerPlanMode=00000000-0000-0000-0000-000000000000 Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true
Toolchain=CoreRun IterationTime=250.0000 ms LaunchCount=100
MaxIterationCount=20 MinIterationCount=15 WarmupCount=1
// * Hints *
Outliers
TryAddDefaultSize.Dictionary: EnvironmentVariables=DOTNET_JitCSEHash=26ab038f,DOTNET_JitDisasmSummary=1,DOTNET_JitStdOutFile=D:\bugs\cse\base.4a.list,DOTNET_JitCSEMask=4, PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, Toolchain=CoreRun, IterationTime=250.0000 ms, LaunchCount=100, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1 -> Something went wrong with outliers: Size(WorkloadActual) = 1548, Size(WorkloadActual/Outliers) = 34, Size(Result) = 1467), OutlierMode = RemoveUpper
This was running
--filter "System.Collections.TryAddDefaultSize<String>.Dictionary"from dotnet/performance using their config setup and a locally built corerun. But I suspect it will repro with a normal build.