[Perf] Linux/arm64: 2182 Regressions on 2/20/2024 2:11:38 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 22.04
Queue	AmpereUbuntu
Baseline	eeadd653e1982d7037a93a9ab38129c07336e7db
Compare	9dc6ea62a4d195ae4559f4609a56933c61889756
Diff	Diff
Configs	CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
SubtractBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	10.75 ns	28.51 ns	2.65	0.05	True
InequalityOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.13 ns	23.08 ns	2.84	0.16	True
CreateTranslationFromVectorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.79 ns	13.38 ns	2.80	0.37	True
NegateBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	9.36 ns	16.10 ns	1.72	0.19	True
InvertBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	13.50 ns	45.24 ns	3.35	0.06	True
CreateScaleFromScalarBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.78 ns	18.38 ns	3.84	0.35	True
CreateRotationBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	26.70 ns	111.49 ns	4.18	0.42	True
MultiplyByScalarBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	9.79 ns	19.33 ns	1.97	0.30	True
GetDeterminantBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.85 ns	13.84 ns	2.86	0.20	True
AddOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	10.98 ns	28.66 ns	2.61	0.02	True
SubtractOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	11.20 ns	28.42 ns	2.54	0.04	True
CreateRotationWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	28.62 ns	116.79 ns	4.08	0.43	True
EqualityOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.12 ns	22.80 ns	2.81	0.10	True
NegationOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	9.28 ns	16.18 ns	1.74	0.20	True
AddBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	10.95 ns	28.89 ns	2.64	0.05	True
MultiplyByMatrixOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	18.66 ns	47.93 ns	2.57	0.03	True
MultiplyByMatrixBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	18.44 ns	40.79 ns	2.21	0.01	True
IsIdentityBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	8.22 ns	12.95 ns	1.58	0.29	True
CreateScaleFromScalarXYWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.28 ns	21.39 ns	5.00	0.29	True
CreateScaleFromScalarXYBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.80 ns	18.32 ns	3.82	0.32	True
CreateFromScalars - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.74 ns	15.83 ns	3.34	0.41	True
CreateScaleFromVectorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.69 ns	19.53 ns	4.16	0.27	True
EqualsBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	12.59 ns	58.23 ns	4.63	0.05	True
CreateScaleFromScalarWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.28 ns	19.95 ns	3.78	0.42	True
IdentityBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.20 ns	8.53 ns	1.64	0.44	True
LerpBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	10.98 ns	23.35 ns	2.13	0.02	True
CreateTranslationFromScalarXY - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.43 ns	16.47 ns	3.71	0.34	True
CreateScaleFromVectorWithCenterBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.52 ns	19.82 ns	4.39	0.31	True
MultiplyByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	9.57 ns	19.10 ns	2.00	0.25	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Matrix3x2* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Matrix3x2* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Details

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Matrix3x2.SubtractBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.InequalityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateTranslationFromVectorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.NegateBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.InvertBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateRotationBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.GetDeterminantBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.AddOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.SubtractOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateRotationWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.EqualityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.NegationOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.AddBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.MultiplyByMatrixOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.MultiplyByMatrixBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.IsIdentityBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarXYWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarXYBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateFromScalars

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromVectorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.EqualsBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.IdentityBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.LerpBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateTranslationFromScalarXY

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromVectorWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 22.04
Queue	AmpereUbuntu
Baseline	eeadd653e1982d7037a93a9ab38129c07336e7db
Compare	9dc6ea62a4d195ae4559f4609a56933c61889756
Diff	Diff
Configs	CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.MathBenchmarks.Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Pow - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	68.25 μs	137.54 μs	2.02	0.08	True
Sin - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	27.76 μs	99.72 μs	3.59	0.16	True
CosPi - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	53.91 μs	213.63 μs	3.96	0.01	True
SinPi - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	55.04 μs	211.82 μs	3.85	0.01	True
Log10P1 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	52.49 μs	104.06 μs	1.98	0.17	True
Log2 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	31.26 μs	64.98 μs	2.08	0.12	True
Hypot - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	41.63 μs	280.73 μs	6.74	0.01	True
Ceiling - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.13 μs	50.14 μs	9.77	0.06	True
SinCos - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	53.52 μs	205.83 μs	3.85	0.09	True
Log2P1 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	30.79 μs	114.74 μs	3.73	0.14	True
Min - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	6.71 μs	25.62 μs	3.82	0.01	True
Log10 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	52.64 μs	92.15 μs	1.75	0.13	True
SinCosPi - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	102.24 μs	415.24 μs	4.06	0.01	True
Exp10M1 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	68.43 μs	121.40 μs	1.77	0.12	True
FusedMultiplyAdd - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	7.30 μs	58.31 μs	7.99	0.06	True
Exp10 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	67.72 μs	120.28 μs	1.78	0.17	True
Floor - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.15 μs	46.84 μs	9.10	0.10	True
Sqrt - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.60 μs	55.28 μs	12.02	0.08	True
ILogB - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	20.05 μs	51.01 μs	2.54	0.04	True
ExpM1 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	36.90 μs	89.55 μs	2.43	0.14	True
CopySign - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	10.84 μs	41.47 μs	3.82	0.01	True
ScaleB - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	22.14 μs	45.68 μs	2.06	0.02	True
Max - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.45 μs	25.80 μs	4.73	0.01	True
Abs - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.87 μs	47.65 μs	9.79	0.01	True
Round - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	19.08 μs	116.36 μs	6.10	0.02	True
Exp2M1 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	34.45 μs	120.76 μs	3.51	0.16	True
Cos - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	43.80 μs	100.31 μs	2.29	0.21	True
Exp2 - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	34.37 μs	118.95 μs	3.46	0.15	True
TanPi - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	59.29 μs	249.21 μs	4.20	0.01	True
Exp - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	36.19 μs	72.17 μs	1.99	0.14	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.MathBenchmarks.Single*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.MathBenchmarks.Single* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.MathBenchmarks.Single*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.MathBenchmarks.Single* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Details

Payloads

Baseline
Compare

System.MathBenchmarks.Single.Pow

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Sin

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.CosPi

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.SinPi

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Log10P1

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Log2

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Hypot

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Ceiling

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.SinCos

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Log2P1

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Min

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Log10

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.SinCosPi

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Exp10M1

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.FusedMultiplyAdd

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Exp10

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Floor

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Sqrt

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.ILogB

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.ExpM1

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.CopySign

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.ScaleB

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Max

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Abs

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Round

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Exp2M1

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Cos

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Exp2

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.TanPi

ETL Files

Histogram

JIT Disasms

System.MathBenchmarks.Single.Exp

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 22.04
Queue	AmpereUbuntu
Baseline	eeadd653e1982d7037a93a9ab38129c07336e7db
Compare	9dc6ea62a4d195ae4559f4609a56933c61889756
Diff	Diff
Configs	CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.IO.Tests.Perf_Path

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
GetDirectoryName - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	137.16 ns	190.53 ns	1.39	0.47	False
GetFullPathForTypicalLongPath - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.80 μs	3.59 μs	2.00	0.01	True
GetFullPathNoRedundantSegments - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	391.68 ns	811.09 ns	2.07	0.01	True
HasExtension - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	10.35 ns	20.08 ns	1.94	0.33	True
Combine - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	4.97 ns	11.03 ns	2.22	0.42	True
GetFullPathForLegacyLength - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	698.70 ns	1.35 μs	1.93	0.01	True
GetFullPathWithRedundantSegments - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.65 μs	4.75 μs	1.30	0.51	False
GetFullPathForReallyLongPath - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.49 μs	6.77 μs	1.94	0.01	True
GetPathRoot - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.59 ns	14.22 ns	3.96	0.45	True
IsPathRooted - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.78 ns	10.90 ns	6.12	0.14	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.Perf_Path*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.IO.Tests.Perf_Path* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.Perf_Path*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.IO.Tests.Perf_Path* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Details

Payloads

Baseline
Compare

System.IO.Tests.Perf_Path.GetDirectoryName

ETL Files

Histogram

JIT Disasms

System.IO.Tests.Perf_Path.GetFullPathForTypicalLongPath

ETL Files

Histogram

JIT Disasms

System.IO.Tests.Perf_Path.GetFullPathNoRedundantSegments

ETL Files

Histogram

JIT Disasms

System.IO.Tests.Perf_Path.HasExtension

ETL Files

Histogram

JIT Disasms

System.IO.Tests.Perf_Path.Combine

ETL Files

Histogram

JIT Disasms

System.IO.Tests.Perf_Path.GetFullPathForLegacyLength

ETL Files

Histogram

JIT Disasms

System.IO.Tests.Perf_Path.GetFullPathWithRedundantSegments

ETL Files

Histogram

JIT Disasms

System.IO.Tests.Perf_Path.GetFullPathForReallyLongPath

ETL Files

Histogram

JIT Disasms

System.IO.Tests.Perf_Path.GetPathRoot

ETL Files

Histogram

JIT Disasms

System.IO.Tests.Perf_Path.IsPathRooted

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 22.04
Queue	AmpereUbuntu
Baseline	eeadd653e1982d7037a93a9ab38129c07336e7db
Compare	9dc6ea62a4d195ae4559f4609a56933c61889756
Diff	Diff
Configs	CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Text.Json.Tests.Perf_Base64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.26 μs	2.00 μs	1.59	0.02	True
WriteByteArrayAsBase64_NoEscaping - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	234.54 ns	409.22 ns	1.74	0.20	True
WriteByteArrayAsBase64_NoEscaping - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.26 μs	2.04 μs	1.62	0.02	True
WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	243.84 ns	410.48 ns	1.68	0.24	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Tests.Perf_Base64*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Tests.Perf_Base64* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Text.Json.Tests.Perf_Base64*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Text.Json.Tests.Perf_Base64* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Details

Payloads

Baseline
Compare

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 1000)

ETL Files

Histogram

JIT Disasms

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 100)

ETL Files

Histogram

JIT Disasms

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 1000)

ETL Files

Histogram

JIT Disasms

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 100)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 22.04
Queue	AmpereUbuntu
Baseline	eeadd653e1982d7037a93a9ab38129c07336e7db
Compare	9dc6ea62a4d195ae4559f4609a56933c61889756
Diff	Diff
Configs	CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Tests.Perf_Int128

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	267.91 ns	827.78 ns	3.09	0.10	True
TryFormat - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	56.74 ns	86.21 ns	1.52	0.01	True
TryParse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	520.42 ns	2.45 μs	4.71	0.06	True
ParseSpan - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	548.79 ns	2.55 μs	4.64	0.05	True
ParseSpan - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	531.18 ns	2.50 μs	4.70	0.05	True
ParseSpan - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	90.03 ns	327.17 ns	3.63	0.08	True
ToString - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	274.75 ns	871.34 ns	3.17	0.07	True
TryParse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	530.29 ns	2.46 μs	4.63	0.09	True
TryParse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	83.38 ns	313.27 ns	3.76	0.06	True
TryFormat - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	412.46 ns	843.95 ns	2.05	0.01	True
Parse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	91.35 ns	340.54 ns	3.73	0.15	True
TryParseSpan - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	518.62 ns	2.46 μs	4.73	0.05	True
TryParseSpan - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	528.04 ns	2.55 μs	4.83	0.05	True
Parse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	532.81 ns	2.46 μs	4.62	0.03	True
TryParseSpan - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	80.20 ns	321.96 ns	4.01	0.02	True
CopySign - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.76 ns	18.09 ns	3.14	0.03	True
Parse - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	544.35 ns	2.52 μs	4.62	0.05	True
TryFormat - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	476.19 ns	959.76 ns	2.02	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Int128*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Int128* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Int128*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Int128* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Details

Payloads

Baseline
Compare

System.Tests.Perf_Int128.ToString(value: 170141183460469231731687303715884105727)

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.TryFormat(value: 12345)

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.TryParse(value: "170141183460469231731687303715884105727")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.ParseSpan(value: "-170141183460469231731687303715884105728")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.ParseSpan(value: "170141183460469231731687303715884105727")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.ParseSpan(value: "12345")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.ToString(value: -170141183460469231731687303715884105728)

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.TryParse(value: "-170141183460469231731687303715884105728")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.TryParse(value: "12345")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.TryFormat(value: 170141183460469231731687303715884105727)

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.Parse(value: "12345")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.TryParseSpan(value: "170141183460469231731687303715884105727")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.TryParseSpan(value: "-170141183460469231731687303715884105728")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.Parse(value: "170141183460469231731687303715884105727")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.TryParseSpan(value: "12345")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.CopySign(value: 1, sign: -1)

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.Parse(value: "-170141183460469231731687303715884105728")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int128.TryFormat(value: -170141183460469231731687303715884105728)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 22.04
Queue	AmpereUbuntu
Baseline	eeadd653e1982d7037a93a9ab38129c07336e7db
Compare	9dc6ea62a4d195ae4559f4609a56933c61889756
Diff	Diff
Configs	CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Collections.IndexerSet<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
List - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	6.50 μs	8.85 μs	1.36	0.02	True
Dictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	28.29 μs	39.70 μs	1.40	0.09	True
SortedList - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	692.07 μs	841.04 μs	1.22	0.25	True
Span - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	351.20 ns	867.91 ns	2.47	0.01	True
SortedDictionary - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	804.33 μs	905.30 μs	1.13	0.40	False
Array - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	3.20 μs	4.12 μs	1.29	0.03	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.IndexerSet<String>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.IndexerSet<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Collections.IndexerSet<String>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Collections.IndexerSet<String>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Details

Payloads

Baseline
Compare

System.Collections.IndexerSet<String>.List(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Collections.IndexerSet<String>.Dictionary(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Collections.IndexerSet<String>.SortedList(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Collections.IndexerSet<String>.Span(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Collections.IndexerSet<String>.SortedDictionary(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Collections.IndexerSet<String>.Array(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 22.04
Queue	AmpereUbuntu
Baseline	eeadd653e1982d7037a93a9ab38129c07336e7db
Compare	9dc6ea62a4d195ae4559f4609a56933c61889756
Diff	Diff
Configs	CompilationMode:tiered, LLVM:true, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
AddOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.86 ns	3.57 ns	1.92	0.04	True
MultiplyFunctionBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.90 ns	3.46 ns	1.82	0.05	True
UnitYBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	0.38 ns	3.68 ns	9.63	0.26	True
DivideByVector2Benchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.82 ns	3.83 ns	2.10	0.08	True
SubtractOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.84 ns	3.78 ns	2.05	0.10	True
TransformByMatrix3x2Benchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.36 ns	24.54 ns	4.58	0.16	True
SquareRootBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.85 ns	3.91 ns	2.12	0.12	True
SubtractFunctionBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.81 ns	3.69 ns	2.03	0.11	True
DivideByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.85 ns	3.64 ns	1.97	0.09	True
TransformNormalByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.28 ns	32.91 ns	6.23	0.16	True
DistanceBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.78 ns	8.87 ns	4.98	0.21	True
AbsBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.88 ns	3.92 ns	2.09	0.40	True
MultiplyOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.83 ns	3.69 ns	2.01	0.12	True
LengthBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.81 ns	9.08 ns	5.01	0.09	True
MaxBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.86 ns	3.63 ns	1.95	0.09	True
AddFunctionBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.85 ns	3.77 ns	2.03	0.15	True
DivideByVector2OperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.81 ns	3.92 ns	2.16	0.07	True
NegateBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.84 ns	3.81 ns	2.07	0.25	True
ClampBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.89 ns	3.54 ns	1.88	0.25	True
OneBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	0.33 ns	3.66 ns	11.04	0.25	True
TransformByMatrix4x4Benchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	6.54 ns	33.63 ns	5.14	0.25	True
MultiplyByScalarOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.85 ns	3.72 ns	2.01	0.05	True
TransformByQuaternionBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.85 ns	21.48 ns	11.64	0.07	True
CreateFromScalarXYBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	0.33 ns	5.96 ns	18.26	0.23	True
TransformNormalByMatrix3x2Benchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	5.26 ns	23.52 ns	4.47	0.47	True
NegateOperatorBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.83 ns	3.70 ns	2.02	0.06	True
ZeroBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	0.34 ns	3.76 ns	11.21	0.26	True
LerpBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.78 ns	3.01 ns	1.69	0.30	True
ReflectBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.71 ns	10.04 ns	5.88	0.27	True
DivideByScalarBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.83 ns	3.83 ns	2.09	0.11	True
UnitXBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	0.34 ns	3.80 ns	11.16	0.14	True
CreateFromScalar - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	0.38 ns	3.01 ns	7.97	0.35	True
EqualsBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.94 ns	9.60 ns	4.95	0.30	True
NormalizeBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	2.17 ns	21.72 ns	10.00	0.13	True
MinBenchmark - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	1.85 ns	3.77 ns	2.03	0.09	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Vector2* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Vector2* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Details

Payloads

[