Skip to content

Commit dbe74b0

Browse files
thomhurstthomhurst
authored
chore: update benchmark results (#3841)
1 parent f98f903 commit dbe74b0

18 files changed

Lines changed: 368 additions & 364 deletions

‎docs/docs/benchmarks/AsyncTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 2
77
# AsyncTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-14** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-13** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.1.0 | 551.3 ms | 551.1 ms | 1.86 ms |
20-
| NUnit | 4.4.0 | 639.4 ms | 634.6 ms | 10.77 ms |
21-
| MSTest | 4.0.2 | 612.9 ms | 611.2 ms | 9.13 ms |
22-
| xUnit3 | 3.2.0 | 692.8 ms | 692.6 ms | 5.81 ms |
23-
| **TUnit (AOT)** | 1.1.0 | 122.7 ms | 122.8 ms | 0.41 ms |
19+
| **TUnit** | 1.1.10 | 565.6 ms | 564.2 ms | 3.94 ms |
20+
| NUnit | 4.4.0 | 681.8 ms | 682.0 ms | 4.42 ms |
21+
| MSTest | 4.0.2 | 654.5 ms | 653.4 ms | 6.20 ms |
22+
| xUnit3 | 3.2.0 | 734.5 ms | 736.6 ms | 8.35 ms |
23+
| **TUnit (AOT)** | 1.1.10 | 124.2 ms | 124.2 ms | 0.29 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-13** from the latest CI
5858
xychart-beta
5959
title "AsyncTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 832
62-
bar [551.3, 639.4, 612.9, 692.8, 122.7]
61+
y-axis "Time (ms)" 0 --> 882
62+
bar [565.6, 681.8, 654.5, 734.5, 124.2]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-13T00:28:35.380Z*
75+
*Last generated: 2025-11-14T00:28:50.466Z*

‎docs/docs/benchmarks/BuildTime.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 8
77
# Build Performance Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-14** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -18,10 +18,10 @@ Compilation time comparison across frameworks:
1818

1919
| Framework | Version | Mean | Median | StdDev |
2020
|-----------|---------|------|--------|--------|
21-
| **TUnit** | 1.1.0 | 2.100 s | 2.101 s | 0.0207 s |
22-
| Build_NUnit | 4.4.0 | 1.688 s | 1.689 s | 0.0214 s |
23-
| Build_MSTest | 4.0.2 | 1.731 s | 1.733 s | 0.0235 s |
24-
| Build_xUnit3 | 3.2.0 | 1.648 s | 1.639 s | 0.0283 s |
21+
| **TUnit** | 1.1.10 | 2.051 s | 2.047 s | 0.0286 s |
22+
| Build_NUnit | 4.4.0 | 1.637 s | 1.634 s | 0.0138 s |
23+
| Build_MSTest | 4.0.2 | 1.709 s | 1.706 s | 0.0086 s |
24+
| Build_xUnit3 | 3.2.0 | 1.618 s | 1.615 s | 0.0180 s |
2525

2626
## 📈 Visual Comparison
2727

@@ -60,7 +60,7 @@ xychart-beta
6060
title "Build Time Comparison"
6161
x-axis ["Build_TUnit", "Build_NUnit", "Build_MSTest", "Build_xUnit3"]
6262
y-axis "Time (s)" 0 --> 3
63-
bar [2.1, 1.688, 1.731, 1.648]
63+
bar [2.051, 1.637, 1.709, 1.618]
6464
```
6565

6666
---
@@ -69,4 +69,4 @@ xychart-beta
6969
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7070
:::
7171

72-
*Last generated: 2025-11-13T00:28:35.383Z*
72+
*Last generated: 2025-11-14T00:28:50.468Z*

‎docs/docs/benchmarks/DataDrivenTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 3
77
# DataDrivenTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-14** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-13** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.1.0 | 486.77 ms | 486.53 ms | 3.791 ms |
20-
| NUnit | 4.4.0 | 589.92 ms | 591.78 ms | 12.153 ms |
21-
| MSTest | 4.0.2 | 605.63 ms | 602.89 ms | 14.207 ms |
22-
| xUnit3 | 3.2.0 | 600.41 ms | 597.06 ms | 10.504 ms |
23-
| **TUnit (AOT)** | 1.1.0 | 24.05 ms | 24.05 ms | 0.239 ms |
19+
| **TUnit** | 1.1.10 | 486.00 ms | 486.93 ms | 2.911 ms |
20+
| NUnit | 4.4.0 | 587.97 ms | 586.01 ms | 10.141 ms |
21+
| MSTest | 4.0.2 | 606.98 ms | 607.45 ms | 10.647 ms |
22+
| xUnit3 | 3.2.0 | 599.37 ms | 599.31 ms | 9.447 ms |
23+
| **TUnit (AOT)** | 1.1.10 | 23.70 ms | 23.74 ms | 0.136 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-13** from the latest CI
5858
xychart-beta
5959
title "DataDrivenTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 727
62-
bar [486.77, 589.92, 605.63, 600.41, 24.05]
61+
y-axis "Time (ms)" 0 --> 729
62+
bar [486, 587.97, 606.98, 599.37, 23.7]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-13T00:28:35.381Z*
75+
*Last generated: 2025-11-14T00:28:50.466Z*

‎docs/docs/benchmarks/MassiveParallelTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 4
77
# MassiveParallelTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-14** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-13** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.1.0 | 601.9 ms | 602.5 ms | 2.97 ms |
20-
| NUnit | 4.4.0 | 1,181.6 ms | 1,177.9 ms | 13.31 ms |
21-
| MSTest | 4.0.2 | 2,965.3 ms | 2,966.7 ms | 6.96 ms |
22-
| xUnit3 | 3.2.0 | 3,066.6 ms | 3,066.7 ms | 6.78 ms |
23-
| **TUnit (AOT)** | 1.1.0 | 131.6 ms | 131.7 ms | 0.37 ms |
19+
| **TUnit** | 1.1.10 | 613.8 ms | 613.2 ms | 4.83 ms |
20+
| NUnit | 4.4.0 | 1,216.8 ms | 1,214.5 ms | 9.48 ms |
21+
| MSTest | 4.0.2 | 2,992.2 ms | 2,990.5 ms | 13.72 ms |
22+
| xUnit3 | 3.2.0 | 3,088.0 ms | 3,087.1 ms | 9.69 ms |
23+
| **TUnit (AOT)** | 1.1.10 | 131.8 ms | 131.9 ms | 0.45 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-13** from the latest CI
5858
xychart-beta
5959
title "MassiveParallelTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 3680
62-
bar [601.9, 1181.6, 2965.3, 3066.6, 131.6]
61+
y-axis "Time (ms)" 0 --> 3706
62+
bar [613.8, 1216.8, 2992.2, 3088, 131.8]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-13T00:28:35.381Z*
75+
*Last generated: 2025-11-14T00:28:50.466Z*

‎docs/docs/benchmarks/MatrixTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 5
77
# MatrixTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-14** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-13** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.1.0 | 570.36 ms | 570.43 ms | 4.980 ms |
20-
| NUnit | 4.4.0 | 1,565.27 ms | 1,565.69 ms | 5.876 ms |
21-
| MSTest | 4.0.2 | 1,523.48 ms | 1,521.53 ms | 8.270 ms |
22-
| xUnit3 | 3.2.0 | 1,605.46 ms | 1,605.92 ms | 7.776 ms |
23-
| **TUnit (AOT)** | 1.1.0 | 79.85 ms | 79.80 ms | 0.324 ms |
19+
| **TUnit** | 1.1.10 | 554.66 ms | 554.27 ms | 3.979 ms |
20+
| NUnit | 4.4.0 | 1,521.00 ms | 1,519.64 ms | 8.186 ms |
21+
| MSTest | 4.0.2 | 1,480.94 ms | 1,479.41 ms | 8.867 ms |
22+
| xUnit3 | 3.2.0 | 1,568.45 ms | 1,568.35 ms | 6.902 ms |
23+
| **TUnit (AOT)** | 1.1.10 | 77.76 ms | 77.77 ms | 0.199 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-13** from the latest CI
5858
xychart-beta
5959
title "MatrixTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1927
62-
bar [570.36, 1565.27, 1523.48, 1605.46, 79.85]
61+
y-axis "Time (ms)" 0 --> 1883
62+
bar [554.66, 1521, 1480.94, 1568.45, 77.76]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-13T00:28:35.381Z*
75+
*Last generated: 2025-11-14T00:28:50.467Z*

‎docs/docs/benchmarks/ScaleTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 6
77
# ScaleTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-14** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-13** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.1.0 | 516.15 ms | 515.58 ms | 7.373 ms |
20-
| NUnit | 4.4.0 | 582.62 ms | 580.89 ms | 8.879 ms |
21-
| MSTest | 4.0.2 | 564.54 ms | 562.45 ms | 7.780 ms |
22-
| xUnit3 | 3.2.0 | 578.98 ms | 576.98 ms | 9.057 ms |
23-
| **TUnit (AOT)** | 1.1.0 | 44.37 ms | 44.24 ms | 3.944 ms |
19+
| **TUnit** | 1.1.10 | 505.85 ms | 505.28 ms | 2.469 ms |
20+
| NUnit | 4.4.0 | 684.72 ms | 681.86 ms | 16.921 ms |
21+
| MSTest | 4.0.2 | 669.43 ms | 669.54 ms | 13.552 ms |
22+
| xUnit3 | 3.2.0 | 684.21 ms | 676.60 ms | 15.666 ms |
23+
| **TUnit (AOT)** | 1.1.10 | 44.88 ms | 44.70 ms | 3.904 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-13** from the latest CI
5858
xychart-beta
5959
title "ScaleTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 700
62-
bar [516.15, 582.62, 564.54, 578.98, 44.37]
61+
y-axis "Time (ms)" 0 --> 822
62+
bar [505.85, 684.72, 669.43, 684.21, 44.88]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-13T00:28:35.382Z*
75+
*Last generated: 2025-11-14T00:28:50.467Z*

‎docs/docs/benchmarks/SetupTeardownTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 7
77
# SetupTeardownTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-14** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-13** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.1.0 | 562.2 ms | 561.5 ms | 4.24 ms |
20-
| NUnit | 4.4.0 | 1,173.5 ms | 1,173.5 ms | 9.02 ms |
21-
| MSTest | 4.0.2 | 1,146.7 ms | 1,145.3 ms | 16.63 ms |
22-
| xUnit3 | 3.2.0 | 1,216.8 ms | 1,217.2 ms | 5.86 ms |
23-
| **TUnit (AOT)** | 1.1.0 | NA | NA | NA |
19+
| **TUnit** | 1.1.10 | 559.8 ms | 560.2 ms | 3.75 ms |
20+
| NUnit | 4.4.0 | 1,135.0 ms | 1,132.6 ms | 11.31 ms |
21+
| MSTest | 4.0.2 | 1,108.5 ms | 1,107.1 ms | 8.11 ms |
22+
| xUnit3 | 3.2.0 | 1,186.2 ms | 1,186.3 ms | 6.25 ms |
23+
| **TUnit (AOT)** | 1.1.10 | NA | NA | NA |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-13** from the latest CI
5858
xychart-beta
5959
title "SetupTeardownTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1461
62-
bar [562.2, 1173.5, 1146.7, 1216.8, 0]
61+
y-axis "Time (ms)" 0 --> 1424
62+
bar [559.8, 1135, 1108.5, 1186.2, 0]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-13T00:28:35.382Z*
75+
*Last generated: 2025-11-14T00:28:50.467Z*

‎docs/docs/benchmarks/index.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 1
77
# Performance Benchmarks
88

99
:::info Last Updated
10-
These benchmarks were automatically generated on **2025-11-13** from the latest CI run.
10+
These benchmarks were automatically generated on **2025-11-14** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -37,7 +37,7 @@ These benchmarks compare TUnit against the most popular .NET testing frameworks:
3737

3838
| Framework | Version Tested |
3939
|-----------|----------------|
40-
| **TUnit** | 1.1.0 |
40+
| **TUnit** | 1.1.10 |
4141
| **xUnit v3** | 3.2.0 |
4242
| **NUnit** | 4.4.0 |
4343
| **MSTest** | 4.0.2 |
@@ -80,4 +80,4 @@ These benchmarks run automatically daily via [GitHub Actions](https://github.com
8080
Each benchmark runs multiple iterations with statistical analysis to ensure accuracy. Results may vary based on hardware and test characteristics.
8181
:::
8282

83-
*Last generated: 2025-11-13T00:28:35.383Z*
83+
*Last generated: 2025-11-14T00:28:50.468Z*

‎docs/static/benchmarks/AsyncTests.json‎

Lines changed: 23 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"timestamp": "2025-11-13T00:28:35.381Z",
2+
"timestamp": "2025-11-14T00:28:50.466Z",
33
"category": "AsyncTests",
44
"environment": {
55
"benchmarkDotNetVersion": "BenchmarkDotNet v0.15.7, Linux Ubuntu 24.04.3 LTS (Noble Numbat)",
@@ -9,43 +9,43 @@
99
"results": [
1010
{
1111
"Method": "TUnit",
12-
"Version": "1.1.0",
13-
"Mean": "551.3 ms",
14-
"Error": "2.23 ms",
15-
"StdDev": "1.86 ms",
16-
"Median": "551.1 ms"
12+
"Version": "1.1.10",
13+
"Mean": "565.6 ms",
14+
"Error": "4.72 ms",
15+
"StdDev": "3.94 ms",
16+
"Median": "564.2 ms"
1717
},
1818
{
1919
"Method": "NUnit",
2020
"Version": "4.4.0",
21-
"Mean": "639.4 ms",
22-
"Error": "11.51 ms",
23-
"StdDev": "10.77 ms",
24-
"Median": "634.6 ms"
21+
"Mean": "681.8 ms",
22+
"Error": "5.30 ms",
23+
"StdDev": "4.42 ms",
24+
"Median": "682.0 ms"
2525
},
2626
{
2727
"Method": "MSTest",
2828
"Version": "4.0.2",
29-
"Mean": "612.9 ms",
30-
"Error": "10.30 ms",
31-
"StdDev": "9.13 ms",
32-
"Median": "611.2 ms"
29+
"Mean": "654.5 ms",
30+
"Error": "7.43 ms",
31+
"StdDev": "6.20 ms",
32+
"Median": "653.4 ms"
3333
},
3434
{
3535
"Method": "xUnit3",
3636
"Version": "3.2.0",
37-
"Mean": "692.8 ms",
38-
"Error": "6.56 ms",
39-
"StdDev": "5.81 ms",
40-
"Median": "692.6 ms"
37+
"Mean": "734.5 ms",
38+
"Error": "9.42 ms",
39+
"StdDev": "8.35 ms",
40+
"Median": "736.6 ms"
4141
},
4242
{
4343
"Method": "TUnit_AOT",
44-
"Version": "1.1.0",
45-
"Mean": "122.7 ms",
46-
"Error": "0.47 ms",
47-
"StdDev": "0.41 ms",
48-
"Median": "122.8 ms"
44+
"Version": "1.1.10",
45+
"Mean": "124.2 ms",
46+
"Error": "0.31 ms",
47+
"StdDev": "0.29 ms",
48+
"Median": "124.2 ms"
4949
}
5050
]
5151
}

0 commit comments

Comments
 (0)