Skip to content

[Perf] Changes at 6/20/2022 7:45:04 PM #6346

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToStream - Duration of single invocation 1.46 μs 883.86 ns 0.61 0.01 False
SerializeToUtf8Bytes - Duration of single invocation 1.40 μs 817.43 ns 0.58 0.01 False
SerializeToString - Duration of single invocation 1.74 μs 1.14 μs 0.66 0.01 False
SerializeObjectProperty - Duration of single invocation 2.03 μs 1.38 μs 0.68 0.02 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToStream


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 883.857165843224 < 1.3833268125103637.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 60.89621227893373 (T) = (0 -876.1515494432989) / Math.Sqrt((57969.32385303655 / (299)) + (158.7189346237011 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.5026936233069499 = (1761.794319368001 - 876.1515494432989) / 1761.794319368001 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;.SerializeToUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 817.4293255539347 < 1.333331239317457.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 65.80377891175843 (T) = (0 -829.5311740418587) / Math.Sqrt((46611.71141419696 / (299)) + (77.82109247689675 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.5036887498845598 = (1671.393049923637 - 829.5311740418587) / 1671.393049923637 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;.SerializeToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1404520294081908 < 1.6518461566246312.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 66.86854793944667 (T) = (0 -1149.4868890126104) / Math.Sqrt((44686.97535549383 / (299)) + (99.24705718449978 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.42343251862332376 = (1993.6727722971273 - 1149.4868890126104) / 1993.6727722971273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3840413381175984 < 1.9004342344900051.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 50.54349435241192 (T) = (0 -1422.6919312919147) / Math.Sqrt((48337.569921029295 / (299)) + (1252.7299324804344 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.375699942022038 = (2278.859200974375 - 1422.6919312919147) / 2278.859200974375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteByteArrayAsBase64_NoEscaping - Duration of single invocation 163.83 ns 108.37 ns 0.66 0.01 False
WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation 1.05 μs 473.97 ns 0.45 0.01 False
WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation 162.82 ns 106.44 ns 0.65 0.01 False
WriteByteArrayAsBase64_NoEscaping - Duration of single invocation 1.04 μs 477.79 ns 0.46 0.01 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Base64*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 100)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 108.36684464229948 < 155.07980627449516.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 63.98447900917435 (T) = (0 -108.58878728632578) / Math.Sqrt((492.90908746493767 / (299)) + (0.17792783854159552 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.4320187624165057 = (191.18375766833853 - 108.58878728632578) / 191.18375766833853 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 1000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 473.9677445522669 < 0.9921985444759072.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 67.01556886135363 (T) = (0 -476.23833294212375) / Math.Sqrt((46121.384638755204 / (299)) + (7.577015542245413 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.6366262392136861 = (1310.6018770083433 - 476.23833294212375) / 1310.6018770083433 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 106.43758991002588 < 154.8901704914402.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 61.88671612065644 (T) = (0 -108.34534500514994) / Math.Sqrt((489.5290681903971 / (299)) + (1.5322352856995118 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.4332050917283139 = (191.15440774781263 - 108.34534500514994) / 191.15440774781263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 1000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 477.78661996353884 < 0.9924306789724591.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 66.93502045736213 (T) = (0 -477.48554261895856) / Math.Sqrt((46013.804942921975 / (299)) + (8.9571348318835 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.6356511559406868 = (1310.5175174954788 - 477.48554261895856) / 1310.5175174954788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.Buffers.Text.Tests.Base64Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Base64Encode - Duration of single invocation 881.56 ns 314.21 ns 0.36 0.01 False
Base64EncodeDestinationTooSmall - Duration of single invocation 878.57 ns 320.48 ns 0.36 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Base64Tests.Base64Encode(NumberOfBytes: 1000)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 314.2116356221086 < 837.5219958614132.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 66.69610829381433 (T) = (0 -316.6646326220751) / Math.Sqrt((46494.290363698245 / (299)) + (1.0559457142302267 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.7243217073293606 = (1148.6745276691163 - 316.6646326220751) / 1148.6745276691163 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Base64Tests.Base64EncodeDestinationTooSmall(NumberOfBytes: 1000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 320.48052744398865 < 834.7421562527652.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 66.17870934756729 (T) = (0 -321.4121190572265) / Math.Sqrt((46408.29315684704 / (299)) + (0.44887072290911806 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.7195386182162227 = (1146.0120356428263 - 321.4121190572265) / 1146.0120356428263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.Security.Cryptography.Primitives.Tests.Performance.Perf_CryptoStream

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Base64EncodeAsync - Duration of single invocation 10.41 ms 4.34 ms 0.42 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_CryptoStream*'

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.Primitives.Tests.Performance.Perf_CryptoStream.Base64EncodeAsync


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.337096774193549 < 9.889730940476188.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 69.88888112518408 (T) = (0 -4345378.305785971) / Math.Sqrt((4679257342344.113 / (299)) + (129658770.29792878 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.6680892681113276 = (13092009.050323427 - 4345378.305785971) / 13092009.050323427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.Text.Json.Reader.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation 1.07 μs 885.39 ns 0.82 0.02 False
ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation 178.11 ns 143.17 ns 0.80 0.15 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 1000)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 885.3865158963844 < 1.0188946561235308.
IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 52.390398702396496 (T) = (0 -884.9523223220303) / Math.Sqrt((11067.881540533906 / (299)) + (9.07081032997818 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.26717047542538264 = (1207.5827906029238 - 884.9523223220303) / 1207.5827906029238 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 143.168082634513 < 167.25128424017848.
IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 50.998669826721276 (T) = (0 -143.66755753544726) / Math.Sqrt((166.1085303069004 / (299)) + (0.7188910647418901 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.2205551645447258 = (184.3203662405832 - 143.66755753544726) / 184.3203662405832 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.IO.Tests.Perf_FileStream

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LockUnlock - Duration of single invocation 42.25 μs 36.83 μs 0.87 0.47 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_FileStream*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_FileStream.LockUnlock(fileSize: 1024, options: Asynchronous)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.83300827423168 < 40.146437191150035.
IsChangePoint: Marked as a change because one of 3/2/2022 12:52:37 PM, 3/3/2022 4:26:18 PM, 3/9/2022 10:56:44 AM, 3/11/2022 10:05:40 PM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 4.080860358420332 (T) = (0 -36008.077403711715) / Math.Sqrt((5378018.658824573 / (299)) + (14618179.482487706 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.1268160912001915 = (41237.67861595712 - 36008.077403711715) / 41237.67861595712 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 1.56 μs 1.38 μs 0.89 0.01 False
DeserializeFromUtf8Bytes - Duration of single invocation 1.32 μs 1.12 μs 0.85 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;BinaryData&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromString


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3817041803004764 < 1.477783394712726.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 40.509887385569364 (T) = (0 -1369.4763464547552) / Math.Sqrt((12562.29381232156 / (299)) + (242.42428563675523 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1940558143533046 = (1699.2198353734354 - 1369.4763464547552) / 1699.2198353734354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;BinaryData&gt;.DeserializeFromUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1200287666307083 < 1.257796936502981.
IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 39.167370090437934 (T) = (0 -1132.7713449719854) / Math.Sqrt((11707.154134651015 / (299)) + (331.2281069644607 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.2271763582606389 = (1465.7565889450605 - 1132.7713449719854) / 1465.7565889450605 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions