Skip to content

Commit 526736e

Browse files
thomhurstthomhurst
authored
chore: update benchmark results (#4674)
1 parent 078faf7 commit 526736e

18 files changed

+368
-368
lines changed

‎docs/docs/benchmarks/AsyncTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 2
77
# AsyncTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-05** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.102
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2026-02-05** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.12.125 | 523.5 ms | 522.1 ms | 3.92 ms |
20-
| NUnit | 4.4.0 | 677.5 ms | 677.1 ms | 4.09 ms |
21-
| MSTest | 4.1.0 | 616.5 ms | 615.2 ms | 4.05 ms |
22-
| xUnit3 | 3.2.2 | 745.7 ms | 746.1 ms | 7.58 ms |
23-
| **TUnit (AOT)** | 1.12.125 | 122.5 ms | 122.5 ms | 0.29 ms |
19+
| **TUnit** | 1.12.144 | 515.8 ms | 514.6 ms | 4.51 ms |
20+
| NUnit | 4.4.0 | 690.2 ms | 691.0 ms | 8.86 ms |
21+
| MSTest | 4.1.0 | 629.3 ms | 628.5 ms | 5.84 ms |
22+
| xUnit3 | 3.2.2 | 767.3 ms | 767.9 ms | 10.19 ms |
23+
| **TUnit (AOT)** | 1.12.144 | 122.8 ms | 122.8 ms | 0.26 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2026-02-05** from the latest CI
5858
xychart-beta
5959
title "AsyncTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 895
62-
bar [523.5, 677.5, 616.5, 745.7, 122.5]
61+
y-axis "Time (ms)" 0 --> 921
62+
bar [515.8, 690.2, 629.3, 767.3, 122.8]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2026-02-05T00:35:56.409Z*
75+
*Last generated: 2026-02-06T00:33:44.204Z*

‎docs/docs/benchmarks/BuildTime.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 8
77
# Build Performance Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-05** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.102
1313
:::
@@ -18,10 +18,10 @@ Compilation time comparison across frameworks:
1818

1919
| Framework | Version | Mean | Median | StdDev |
2020
|-----------|---------|------|--------|--------|
21-
| **TUnit** | 1.12.125 | 2.411 s | 2.413 s | 0.0393 s |
22-
| Build_NUnit | 4.4.0 | 1.814 s | 1.813 s | 0.0158 s |
23-
| Build_MSTest | 4.1.0 | 1.881 s | 1.883 s | 0.0173 s |
24-
| Build_xUnit3 | 3.2.2 | 1.802 s | 1.805 s | 0.0140 s |
21+
| **TUnit** | 1.12.144 | 2.285 s | 2.281 s | 0.0361 s |
22+
| Build_NUnit | 4.4.0 | 1.840 s | 1.845 s | 0.0273 s |
23+
| Build_MSTest | 4.1.0 | 1.916 s | 1.922 s | 0.0215 s |
24+
| Build_xUnit3 | 3.2.2 | 1.820 s | 1.815 s | 0.0194 s |
2525

2626
## 📈 Visual Comparison
2727

@@ -60,7 +60,7 @@ xychart-beta
6060
title "Build Time Comparison"
6161
x-axis ["Build_TUnit", "Build_NUnit", "Build_MSTest", "Build_xUnit3"]
6262
y-axis "Time (s)" 0 --> 3
63-
bar [2.411, 1.814, 1.881, 1.802]
63+
bar [2.285, 1.84, 1.916, 1.82]
6464
```
6565

6666
---
@@ -69,4 +69,4 @@ xychart-beta
6969
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7070
:::
7171

72-
*Last generated: 2026-02-05T00:35:56.411Z*
72+
*Last generated: 2026-02-06T00:33:44.206Z*

‎docs/docs/benchmarks/DataDrivenTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 3
77
# DataDrivenTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-05** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.102
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2026-02-05** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.12.125 | 447.94 ms | 446.93 ms | 4.066 ms |
20-
| NUnit | 4.4.0 | 546.19 ms | 546.50 ms | 7.371 ms |
21-
| MSTest | 4.1.0 | 541.63 ms | 547.17 ms | 16.359 ms |
22-
| xUnit3 | 3.2.2 | 599.93 ms | 601.29 ms | 7.899 ms |
23-
| **TUnit (AOT)** | 1.12.125 | 22.39 ms | 22.30 ms | 0.821 ms |
19+
| **TUnit** | 1.12.144 | 449.81 ms | 449.63 ms | 2.515 ms |
20+
| NUnit | 4.4.0 | 540.28 ms | 540.85 ms | 3.112 ms |
21+
| MSTest | 4.1.0 | 467.43 ms | 467.88 ms | 8.837 ms |
22+
| xUnit3 | 3.2.2 | 584.73 ms | 584.53 ms | 8.810 ms |
23+
| **TUnit (AOT)** | 1.12.144 | 22.53 ms | 22.39 ms | 0.391 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2026-02-05** from the latest CI
5858
xychart-beta
5959
title "DataDrivenTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 720
62-
bar [447.94, 546.19, 541.63, 599.93, 22.39]
61+
y-axis "Time (ms)" 0 --> 702
62+
bar [449.81, 540.28, 467.43, 584.73, 22.53]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2026-02-05T00:35:56.409Z*
75+
*Last generated: 2026-02-06T00:33:44.205Z*

‎docs/docs/benchmarks/MassiveParallelTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 4
77
# MassiveParallelTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-05** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.102
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2026-02-05** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.12.125 | 549.7 ms | 548.0 ms | 5.16 ms |
20-
| NUnit | 4.4.0 | 1,202.0 ms | 1,203.4 ms | 6.37 ms |
21-
| MSTest | 4.1.0 | 2,943.8 ms | 2,942.8 ms | 4.25 ms |
22-
| xUnit3 | 3.2.2 | 3,086.0 ms | 3,084.2 ms | 7.70 ms |
23-
| **TUnit (AOT)** | 1.12.125 | 127.1 ms | 127.1 ms | 0.41 ms |
19+
| **TUnit** | 1.12.144 | 529.3 ms | 530.1 ms | 6.21 ms |
20+
| NUnit | 4.4.0 | 1,217.6 ms | 1,213.7 ms | 12.73 ms |
21+
| MSTest | 4.1.0 | 2,976.5 ms | 2,975.1 ms | 7.53 ms |
22+
| xUnit3 | 3.2.2 | 3,096.9 ms | 3,094.0 ms | 9.16 ms |
23+
| **TUnit (AOT)** | 1.12.144 | 126.9 ms | 126.8 ms | 0.45 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2026-02-05** from the latest CI
5858
xychart-beta
5959
title "MassiveParallelTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 3704
62-
bar [549.7, 1202, 2943.8, 3086, 127.1]
61+
y-axis "Time (ms)" 0 --> 3717
62+
bar [529.3, 1217.6, 2976.5, 3096.9, 126.9]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2026-02-05T00:35:56.410Z*
75+
*Last generated: 2026-02-06T00:33:44.205Z*

‎docs/docs/benchmarks/MatrixTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 5
77
# MatrixTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-05** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.102
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2026-02-05** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.12.125 | 518.68 ms | 518.56 ms | 3.051 ms |
20-
| NUnit | 4.4.0 | 1,577.31 ms | 1,576.69 ms | 8.184 ms |
21-
| MSTest | 4.1.0 | 1,499.87 ms | 1,499.97 ms | 8.243 ms |
22-
| xUnit3 | 3.2.2 | 1,634.66 ms | 1,631.23 ms | 9.248 ms |
23-
| **TUnit (AOT)** | 1.12.125 | 76.08 ms | 76.14 ms | 0.201 ms |
19+
| **TUnit** | 1.12.144 | 502.27 ms | 504.58 ms | 10.487 ms |
20+
| NUnit | 4.4.0 | 1,594.77 ms | 1,599.82 ms | 9.496 ms |
21+
| MSTest | 4.1.0 | 1,509.57 ms | 1,507.88 ms | 7.804 ms |
22+
| xUnit3 | 3.2.2 | 1,631.96 ms | 1,632.83 ms | 6.993 ms |
23+
| **TUnit (AOT)** | 1.12.144 | 73.65 ms | 73.54 ms | 0.399 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2026-02-05** from the latest CI
5858
xychart-beta
5959
title "MatrixTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1962
62-
bar [518.68, 1577.31, 1499.87, 1634.66, 76.08]
61+
y-axis "Time (ms)" 0 --> 1959
62+
bar [502.27, 1594.77, 1509.57, 1631.96, 73.65]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2026-02-05T00:35:56.410Z*
75+
*Last generated: 2026-02-06T00:33:44.205Z*

‎docs/docs/benchmarks/ScaleTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 6
77
# ScaleTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-05** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.102
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2026-02-05** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.12.125 | 470.16 ms | 470.79 ms | 5.698 ms |
20-
| NUnit | 4.4.0 | 614.11 ms | 612.86 ms | 6.958 ms |
21-
| MSTest | 4.1.0 | 591.09 ms | 587.48 ms | 11.219 ms |
22-
| xUnit3 | 3.2.2 | 634.65 ms | 632.92 ms | 9.877 ms |
23-
| **TUnit (AOT)** | 1.12.125 | 32.07 ms | 31.94 ms | 1.209 ms |
19+
| **TUnit** | 1.12.144 | 458.47 ms | 458.88 ms | 3.489 ms |
20+
| NUnit | 4.4.0 | 593.89 ms | 591.41 ms | 10.141 ms |
21+
| MSTest | 4.1.0 | 545.46 ms | 546.66 ms | 15.522 ms |
22+
| xUnit3 | 3.2.2 | 621.32 ms | 624.47 ms | 8.320 ms |
23+
| **TUnit (AOT)** | 1.12.144 | 32.42 ms | 31.98 ms | 2.611 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2026-02-05** from the latest CI
5858
xychart-beta
5959
title "ScaleTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 762
62-
bar [470.16, 614.11, 591.09, 634.65, 32.07]
61+
y-axis "Time (ms)" 0 --> 746
62+
bar [458.47, 593.89, 545.46, 621.32, 32.42]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2026-02-05T00:35:56.410Z*
75+
*Last generated: 2026-02-06T00:33:44.206Z*

‎docs/docs/benchmarks/SetupTeardownTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 7
77
# SetupTeardownTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-05** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.102
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2026-02-05** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.12.125 | 559.9 ms | 559.2 ms | 9.00 ms |
20-
| NUnit | 4.4.0 | 1,211.2 ms | 1,211.7 ms | 7.55 ms |
21-
| MSTest | 4.1.0 | 1,153.3 ms | 1,153.7 ms | 6.13 ms |
22-
| xUnit3 | 3.2.2 | 1,259.7 ms | 1,257.7 ms | 7.82 ms |
23-
| **TUnit (AOT)** | 1.12.125 | NA | NA | NA |
19+
| **TUnit** | 1.12.144 | 544.0 ms | 545.1 ms | 4.30 ms |
20+
| NUnit | 4.4.0 | 1,193.5 ms | 1,192.7 ms | 8.52 ms |
21+
| MSTest | 4.1.0 | 1,132.1 ms | 1,134.6 ms | 6.76 ms |
22+
| xUnit3 | 3.2.2 | 1,266.1 ms | 1,264.9 ms | 12.67 ms |
23+
| **TUnit (AOT)** | 1.12.144 | NA | NA | NA |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2026-02-05** from the latest CI
5858
xychart-beta
5959
title "SetupTeardownTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1512
62-
bar [559.9, 1211.2, 1153.3, 1259.7, 0]
61+
y-axis "Time (ms)" 0 --> 1520
62+
bar [544, 1193.5, 1132.1, 1266.1, 0]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2026-02-05T00:35:56.410Z*
75+
*Last generated: 2026-02-06T00:33:44.206Z*

‎docs/docs/benchmarks/index.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 1
77
# Performance Benchmarks
88

99
:::info Last Updated
10-
These benchmarks were automatically generated on **2026-02-05** from the latest CI run.
10+
These benchmarks were automatically generated on **2026-02-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.102
1313
:::
@@ -37,7 +37,7 @@ These benchmarks compare TUnit against the most popular .NET testing frameworks:
3737

3838
| Framework | Version Tested |
3939
|-----------|----------------|
40-
| **TUnit** | 1.12.125 |
40+
| **TUnit** | 1.12.144 |
4141
| **xUnit v3** | 3.2.2 |
4242
| **NUnit** | 4.4.0 |
4343
| **MSTest** | 4.1.0 |
@@ -80,4 +80,4 @@ These benchmarks run automatically daily via [GitHub Actions](https://github.com
8080
Each benchmark runs multiple iterations with statistical analysis to ensure accuracy. Results may vary based on hardware and test characteristics.
8181
:::
8282

83-
*Last generated: 2026-02-05T00:35:56.411Z*
83+
*Last generated: 2026-02-06T00:33:44.207Z*

‎docs/static/benchmarks/AsyncTests.json‎

Lines changed: 23 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"timestamp": "2026-02-05T00:35:56.409Z",
2+
"timestamp": "2026-02-06T00:33:44.205Z",
33
"category": "AsyncTests",
44
"environment": {
55
"benchmarkDotNetVersion": "BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.3 LTS (Noble Numbat)",
@@ -9,43 +9,43 @@
99
"results": [
1010
{
1111
"Method": "TUnit",
12-
"Version": "1.12.125",
13-
"Mean": "523.5 ms",
14-
"Error": "4.19 ms",
15-
"StdDev": "3.92 ms",
16-
"Median": "522.1 ms"
12+
"Version": "1.12.144",
13+
"Mean": "515.8 ms",
14+
"Error": "4.82 ms",
15+
"StdDev": "4.51 ms",
16+
"Median": "514.6 ms"
1717
},
1818
{
1919
"Method": "NUnit",
2020
"Version": "4.4.0",
21-
"Mean": "677.5 ms",
22-
"Error": "4.61 ms",
23-
"StdDev": "4.09 ms",
24-
"Median": "677.1 ms"
21+
"Mean": "690.2 ms",
22+
"Error": "10.61 ms",
23+
"StdDev": "8.86 ms",
24+
"Median": "691.0 ms"
2525
},
2626
{
2727
"Method": "MSTest",
2828
"Version": "4.1.0",
29-
"Mean": "616.5 ms",
30-
"Error": "4.33 ms",
31-
"StdDev": "4.05 ms",
32-
"Median": "615.2 ms"
29+
"Mean": "629.3 ms",
30+
"Error": "7.48 ms",
31+
"StdDev": "5.84 ms",
32+
"Median": "628.5 ms"
3333
},
3434
{
3535
"Method": "xUnit3",
3636
"Version": "3.2.2",
37-
"Mean": "745.7 ms",
38-
"Error": "8.55 ms",
39-
"StdDev": "7.58 ms",
40-
"Median": "746.1 ms"
37+
"Mean": "767.3 ms",
38+
"Error": "10.89 ms",
39+
"StdDev": "10.19 ms",
40+
"Median": "767.9 ms"
4141
},
4242
{
4343
"Method": "TUnit_AOT",
44-
"Version": "1.12.125",
45-
"Mean": "122.5 ms",
46-
"Error": "0.32 ms",
47-
"StdDev": "0.29 ms",
48-
"Median": "122.5 ms"
44+
"Version": "1.12.144",
45+
"Mean": "122.8 ms",
46+
"Error": "0.28 ms",
47+
"StdDev": "0.26 ms",
48+
"Median": "122.8 ms"
4949
}
5050
]
5151
}

0 commit comments

Comments
 (0)