Skip to content

fix: state serialization time #7109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
twoeths merged 4 commits into from
Sep 27, 2024
Merged

fix: state serialization time #7109

twoeths merged 4 commits into from
Sep 27, 2024

Conversation

@twoeths
Copy link
Contributor

@twoeths twoeths commented Sep 26, 2024

Motivation

There was a regression of v1.22 due to #7042 but it only affect n-historical-states configuration. This happens when we persist checkpoint state at every epoch

Screenshot 2024-09-26 at 13 13 41

state serialization time was spiked because:

  • BufferPool is not used correctly
  • the metrics are recorded after state bytes are persisted to db

Description

  • release Uint8Array back to BufferPool with "using" key word
  • record the state serialization time before persisting state bytes to db
  • also refactor the metric to be used for both archive_state and persistent_checkpoints_cache_state flow

will follow up with Dashboard fix when this is merged

a prerequisite for #7104

Testing

It now takes less than 300ms to serialize a mainnet state

Screenshot 2024-09-26 at 13 18 13

@twoeths twoeths mentioned this pull request Sep 26, 2024
Merged
twoeths
twoeths commented Sep 26, 2024
View reviewed changes
packages/beacon-node/src/chain/serializeState.ts
let stateBytes: Uint8Array | null = null;
if (bufferPool) {
const bufferWithKey = bufferPool.alloc(size, source);
using bufferWithKey = bufferPool.alloc(size, source);
Copy link
Contributor Author

@twoeths twoeths Sep 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the main fix

@github-actions
Copy link
Contributor

github-actions bot commented Sep 26, 2024

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 45e6bc8 Previous: bf60786 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.9289 ms/op 1.7725 ms/op 1.09
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 47.555 us/op 37.860 us/op 1.26
BLS verify - blst 840.58 us/op 893.40 us/op 0.94
BLS verifyMultipleSignatures 3 - blst 1.2474 ms/op 1.7107 ms/op 0.73
BLS verifyMultipleSignatures 8 - blst 2.5339 ms/op 1.9509 ms/op 1.30
BLS verifyMultipleSignatures 32 - blst 7.4848 ms/op 4.2600 ms/op 1.76
BLS verifyMultipleSignatures 64 - blst 10.971 ms/op 8.2356 ms/op 1.33
BLS verifyMultipleSignatures 128 - blst 16.819 ms/op 15.511 ms/op 1.08
BLS deserializing 10000 signatures 654.00 ms/op 593.58 ms/op 1.10
BLS deserializing 100000 signatures 6.5087 s/op 6.1357 s/op 1.06
BLS verifyMultipleSignatures - same message - 3 - blst 814.20 us/op 985.92 us/op 0.83
BLS verifyMultipleSignatures - same message - 8 - blst 952.99 us/op 1.0911 ms/op 0.87
BLS verifyMultipleSignatures - same message - 32 - blst 1.7320 ms/op 1.6556 ms/op 1.05
BLS verifyMultipleSignatures - same message - 64 - blst 2.5427 ms/op 2.4183 ms/op 1.05
BLS verifyMultipleSignatures - same message - 128 - blst 4.1676 ms/op 3.9903 ms/op 1.04
BLS aggregatePubkeys 32 - blst 19.052 us/op 17.660 us/op 1.08
BLS aggregatePubkeys 128 - blst 65.624 us/op 60.736 us/op 1.08
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 63.651 ms/op 64.202 ms/op 0.99
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 59.709 ms/op 43.586 ms/op 1.37
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 41.198 ms/op 32.628 ms/op 1.26
getSlashingsAndExits - default max 83.609 us/op 71.269 us/op 1.17
getSlashingsAndExits - 2k 253.10 us/op 243.81 us/op 1.04
proposeBlockBody type=full, size=empty 5.7867 ms/op 5.0919 ms/op 1.14
isKnown best case - 1 super set check 286.00 ns/op 501.00 ns/op 0.57
isKnown normal case - 2 super set checks 277.00 ns/op 461.00 ns/op 0.60
isKnown worse case - 16 super set checks 275.00 ns/op 468.00 ns/op 0.59
InMemoryCheckpointStateCache - add get delete 2.4630 us/op 2.8170 us/op 0.87
updateUnfinalizedPubkeys - updating 10 pubkeys 956.04 us/op 1.2399 ms/op 0.77
updateUnfinalizedPubkeys - updating 100 pubkeys 3.5973 ms/op 3.2239 ms/op 1.12
updateUnfinalizedPubkeys - updating 1000 pubkeys 47.583 ms/op 40.109 ms/op 1.19
validate api signedAggregateAndProof - struct 2.4479 ms/op 1.5448 ms/op 1.58
validate gossip signedAggregateAndProof - struct 2.6709 ms/op 1.7758 ms/op 1.50
validate gossip attestation - vc 640000 1.3289 ms/op 973.18 us/op 1.37
batch validate gossip attestation - vc 640000 - chunk 32 130.05 us/op 114.55 us/op 1.14
batch validate gossip attestation - vc 640000 - chunk 64 116.57 us/op 95.666 us/op 1.22
batch validate gossip attestation - vc 640000 - chunk 128 100.05 us/op 96.930 us/op 1.03
batch validate gossip attestation - vc 640000 - chunk 256 103.80 us/op 90.762 us/op 1.14
pickEth1Vote - no votes 994.21 us/op 800.58 us/op 1.24
pickEth1Vote - max votes 7.1567 ms/op 4.2318 ms/op 1.69
pickEth1Vote - Eth1Data hashTreeRoot value x2048 15.596 ms/op 9.2263 ms/op 1.69
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 26.150 ms/op 13.752 ms/op 1.90
pickEth1Vote - Eth1Data fastSerialize value x2048 461.02 us/op 329.91 us/op 1.40
pickEth1Vote - Eth1Data fastSerialize tree x2048 4.2992 ms/op 1.8613 ms/op 2.31
bytes32 toHexString 431.00 ns/op 563.00 ns/op 0.77
bytes32 Buffer.toString(hex) 255.00 ns/op 428.00 ns/op 0.60
bytes32 Buffer.toString(hex) from Uint8Array 361.00 ns/op 520.00 ns/op 0.69
bytes32 Buffer.toString(hex) + 0x 257.00 ns/op 423.00 ns/op 0.61
Object access 1 prop 0.13600 ns/op 0.30800 ns/op 0.44
Map access 1 prop 0.13300 ns/op 0.30500 ns/op 0.44
Object get x1000 5.8760 ns/op 4.7880 ns/op 1.23
Map get x1000 6.4900 ns/op 5.5950 ns/op 1.16
Object set x1000 31.171 ns/op 25.151 ns/op 1.24
Map set x1000 22.528 ns/op 18.667 ns/op 1.21
Return object 10000 times 0.28760 ns/op 0.28630 ns/op 1.00
Throw Error 10000 times 3.3248 us/op 2.5585 us/op 1.30
toHex 141.37 ns/op 100.89 ns/op 1.40
Buffer.from 132.66 ns/op 95.521 ns/op 1.39
shared Buffer 90.481 ns/op 63.900 ns/op 1.42
fastMsgIdFn sha256 / 200 bytes 2.1910 us/op 1.8810 us/op 1.16
fastMsgIdFn h32 xxhash / 200 bytes 225.00 ns/op 393.00 ns/op 0.57
fastMsgIdFn h64 xxhash / 200 bytes 267.00 ns/op 434.00 ns/op 0.62
fastMsgIdFn sha256 / 1000 bytes 7.3690 us/op 5.4930 us/op 1.34
fastMsgIdFn h32 xxhash / 1000 bytes 351.00 ns/op 511.00 ns/op 0.69
fastMsgIdFn h64 xxhash / 1000 bytes 345.00 ns/op 507.00 ns/op 0.68
fastMsgIdFn sha256 / 10000 bytes 63.985 us/op 46.280 us/op 1.38
fastMsgIdFn h32 xxhash / 10000 bytes 1.8410 us/op 1.8350 us/op 1.00
fastMsgIdFn h64 xxhash / 10000 bytes 1.2280 us/op 1.2850 us/op 0.96
send data - 1000 256B messages 12.392 ms/op 9.3224 ms/op 1.33
send data - 1000 512B messages 15.561 ms/op 13.147 ms/op 1.18
send data - 1000 1024B messages 26.079 ms/op 23.413 ms/op 1.11
send data - 1000 1200B messages 26.220 ms/op 22.890 ms/op 1.15
send data - 1000 2048B messages 32.132 ms/op 29.629 ms/op 1.08
send data - 1000 4096B messages 31.835 ms/op 25.649 ms/op 1.24
send data - 1000 16384B messages 72.005 ms/op 65.331 ms/op 1.10
send data - 1000 65536B messages 215.41 ms/op 238.55 ms/op 0.90
enrSubnets - fastDeserialize 64 bits 1.0940 us/op 1.1350 us/op 0.96
enrSubnets - ssz BitVector 64 bits 355.00 ns/op 530.00 ns/op 0.67
enrSubnets - fastDeserialize 4 bits 149.00 ns/op 349.00 ns/op 0.43
enrSubnets - ssz BitVector 4 bits 346.00 ns/op 534.00 ns/op 0.65
prioritizePeers score -10:0 att 32-0.1 sync 2-0 162.99 us/op 120.97 us/op 1.35
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 138.80 us/op 147.56 us/op 0.94
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 371.86 us/op 268.79 us/op 1.38
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 579.28 us/op 473.57 us/op 1.22
prioritizePeers score 0:0 att 64-1 sync 4-1 996.32 us/op 448.62 us/op 2.22
array of 16000 items push then shift 1.6277 us/op 1.2796 us/op 1.27
LinkedList of 16000 items push then shift 7.1280 ns/op 6.2920 ns/op 1.13
array of 16000 items push then pop 103.41 ns/op 75.958 ns/op 1.36
LinkedList of 16000 items push then pop 6.9600 ns/op 6.1310 ns/op 1.14
array of 24000 items push then shift 2.4039 us/op 1.8762 us/op 1.28
LinkedList of 24000 items push then shift 7.0650 ns/op 6.2660 ns/op 1.13
array of 24000 items push then pop 135.58 ns/op 106.50 ns/op 1.27
LinkedList of 24000 items push then pop 6.9040 ns/op 6.0300 ns/op 1.14
intersect bitArray bitLen 8 6.4500 ns/op 5.0860 ns/op 1.27
intersect array and set length 8 47.151 ns/op 36.012 ns/op 1.31
intersect bitArray bitLen 128 29.935 ns/op 24.801 ns/op 1.21
intersect array and set length 128 684.70 ns/op 533.86 ns/op 1.28
bitArray.getTrueBitIndexes() bitLen 128 2.9270 us/op 2.2920 us/op 1.28
bitArray.getTrueBitIndexes() bitLen 248 3.6550 us/op 3.2270 us/op 1.13
bitArray.getTrueBitIndexes() bitLen 512 8.3500 us/op 6.4240 us/op 1.30
Buffer.concat 32 items 997.00 ns/op 978.00 ns/op 1.02
Uint8Array.set 32 items 1.8640 us/op 1.7320 us/op 1.08
Buffer.copy 2.2690 us/op 1.8880 us/op 1.20
Uint8Array.set - with subarray 3.3680 us/op 2.4960 us/op 1.35
Uint8Array.set - without subarray 1.8280 us/op 1.6700 us/op 1.09
getUint32 - dataview 229.00 ns/op 412.00 ns/op 0.56
getUint32 - manual 148.00 ns/op 343.00 ns/op 0.43
Set add up to 64 items then delete first 2.1281 us/op 1.8455 us/op 1.15
OrderedSet add up to 64 items then delete first 3.1959 us/op 2.8647 us/op 1.12
Set add up to 64 items then delete last 2.4966 us/op 2.0953 us/op 1.19
OrderedSet add up to 64 items then delete last 3.8474 us/op 3.1607 us/op 1.22
Set add up to 64 items then delete middle 2.4926 us/op 2.1011 us/op 1.19
OrderedSet add up to 64 items then delete middle 5.2677 us/op 4.3454 us/op 1.21
Set add up to 128 items then delete first 4.9170 us/op 3.7705 us/op 1.30
OrderedSet add up to 128 items then delete first 7.4107 us/op 5.9257 us/op 1.25
Set add up to 128 items then delete last 4.9784 us/op 3.6443 us/op 1.37
OrderedSet add up to 128 items then delete last 7.6791 us/op 5.5940 us/op 1.37
Set add up to 128 items then delete middle 4.9070 us/op 3.6430 us/op 1.35
OrderedSet add up to 128 items then delete middle 13.523 us/op 11.109 us/op 1.22
Set add up to 256 items then delete first 10.286 us/op 7.4357 us/op 1.38
OrderedSet add up to 256 items then delete first 15.348 us/op 11.913 us/op 1.29
Set add up to 256 items then delete last 9.9575 us/op 7.2251 us/op 1.38
OrderedSet add up to 256 items then delete last 15.632 us/op 11.245 us/op 1.39
Set add up to 256 items then delete middle 9.9880 us/op 7.7943 us/op 1.28
OrderedSet add up to 256 items then delete middle 39.924 us/op 34.228 us/op 1.17
transfer serialized Status (84 B) 1.5160 us/op 1.3510 us/op 1.12
copy serialized Status (84 B) 1.3040 us/op 1.1550 us/op 1.13
transfer serialized SignedVoluntaryExit (112 B) 1.7570 us/op 1.5150 us/op 1.16
copy serialized SignedVoluntaryExit (112 B) 1.4180 us/op 1.1950 us/op 1.19
transfer serialized ProposerSlashing (416 B) 2.6650 us/op 2.2810 us/op 1.17
copy serialized ProposerSlashing (416 B) 2.7110 us/op 2.0300 us/op 1.34
transfer serialized Attestation (485 B) 2.9460 us/op 2.1000 us/op 1.40
copy serialized Attestation (485 B) 2.8650 us/op 1.9780 us/op 1.45
transfer serialized AttesterSlashing (33232 B) 2.9820 us/op 2.2920 us/op 1.30
copy serialized AttesterSlashing (33232 B) 6.3840 us/op 4.5220 us/op 1.41
transfer serialized Small SignedBeaconBlock (128000 B) 3.8660 us/op 2.7270 us/op 1.42
copy serialized Small SignedBeaconBlock (128000 B) 16.106 us/op 10.159 us/op 1.59
transfer serialized Avg SignedBeaconBlock (200000 B) 4.1440 us/op 3.2920 us/op 1.26
copy serialized Avg SignedBeaconBlock (200000 B) 23.386 us/op 14.058 us/op 1.66
transfer serialized BlobsSidecar (524380 B) 3.5780 us/op 3.2200 us/op 1.11
copy serialized BlobsSidecar (524380 B) 85.240 us/op 75.342 us/op 1.13
transfer serialized Big SignedBeaconBlock (1000000 B) 3.3440 us/op 3.4810 us/op 0.96
copy serialized Big SignedBeaconBlock (1000000 B) 153.17 us/op 135.16 us/op 1.13
pass gossip attestations to forkchoice per slot 2.8162 ms/op 2.2705 ms/op 1.24
forkChoice updateHead vc 100000 bc 64 eq 0 492.81 us/op 372.04 us/op 1.32
forkChoice updateHead vc 600000 bc 64 eq 0 3.1485 ms/op 2.5074 ms/op 1.26
forkChoice updateHead vc 1000000 bc 64 eq 0 5.2008 ms/op 3.9867 ms/op 1.30
forkChoice updateHead vc 600000 bc 320 eq 0 2.9529 ms/op 2.3261 ms/op 1.27
forkChoice updateHead vc 600000 bc 1200 eq 0 3.0065 ms/op 2.3320 ms/op 1.29
forkChoice updateHead vc 600000 bc 7200 eq 0 3.7604 ms/op 2.9664 ms/op 1.27
forkChoice updateHead vc 600000 bc 64 eq 1000 10.400 ms/op 8.9068 ms/op 1.17
forkChoice updateHead vc 600000 bc 64 eq 10000 10.752 ms/op 8.8146 ms/op 1.22
forkChoice updateHead vc 600000 bc 64 eq 300000 14.035 ms/op 11.360 ms/op 1.24
computeDeltas 500000 validators 300 proto nodes 3.4181 ms/op 2.9554 ms/op 1.16
computeDeltas 500000 validators 1200 proto nodes 3.4374 ms/op 2.9110 ms/op 1.18
computeDeltas 500000 validators 7200 proto nodes 3.4675 ms/op 2.8986 ms/op 1.20
computeDeltas 750000 validators 300 proto nodes 5.3053 ms/op 4.3639 ms/op 1.22
computeDeltas 750000 validators 1200 proto nodes 5.1213 ms/op 4.3820 ms/op 1.17
computeDeltas 750000 validators 7200 proto nodes 5.2056 ms/op 4.3820 ms/op 1.19
computeDeltas 1400000 validators 300 proto nodes 9.7725 ms/op 8.0802 ms/op 1.21
computeDeltas 1400000 validators 1200 proto nodes 10.217 ms/op 8.0391 ms/op 1.27
computeDeltas 1400000 validators 7200 proto nodes 10.059 ms/op 8.2811 ms/op 1.21
computeDeltas 2100000 validators 300 proto nodes 15.100 ms/op 12.164 ms/op 1.24
computeDeltas 2100000 validators 1200 proto nodes 14.993 ms/op 12.203 ms/op 1.23
computeDeltas 2100000 validators 7200 proto nodes 14.728 ms/op 12.658 ms/op 1.16
altair processAttestation - 250000 vs - 7PWei normalcase 1.6758 ms/op 1.4604 ms/op 1.15
altair processAttestation - 250000 vs - 7PWei worstcase 2.5391 ms/op 2.1987 ms/op 1.15
altair processAttestation - setStatus - 1/6 committees join 88.338 us/op 72.164 us/op 1.22
altair processAttestation - setStatus - 1/3 committees join 169.90 us/op 140.18 us/op 1.21
altair processAttestation - setStatus - 1/2 committees join 242.00 us/op 201.80 us/op 1.20
altair processAttestation - setStatus - 2/3 committees join 316.42 us/op 270.17 us/op 1.17
altair processAttestation - setStatus - 4/5 committees join 487.43 us/op 393.25 us/op 1.24
altair processAttestation - setStatus - 100% committees join 537.33 us/op 478.96 us/op 1.12
altair processBlock - 250000 vs - 7PWei normalcase 5.2281 ms/op 3.4566 ms/op 1.51
altair processBlock - 250000 vs - 7PWei normalcase hashState 28.738 ms/op 22.045 ms/op 1.30
altair processBlock - 250000 vs - 7PWei worstcase 36.114 ms/op 31.650 ms/op 1.14
altair processBlock - 250000 vs - 7PWei worstcase hashState 75.267 ms/op 65.917 ms/op 1.14
phase0 processBlock - 250000 vs - 7PWei normalcase 2.3096 ms/op 1.6852 ms/op 1.37
phase0 processBlock - 250000 vs - 7PWei worstcase 25.158 ms/op 21.956 ms/op 1.15
altair processEth1Data - 250000 vs - 7PWei normalcase 304.88 us/op 239.48 us/op 1.27
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 6.7150 us/op 5.3580 us/op 1.25
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 39.162 us/op 29.010 us/op 1.35
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 9.5120 us/op 8.2160 us/op 1.16
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 5.6990 us/op 3.3570 us/op 1.70
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 180.81 us/op 130.59 us/op 1.38
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.9167 ms/op 796.80 us/op 2.41
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.6572 ms/op 1.0540 ms/op 2.52
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 2.6645 ms/op 1.2295 ms/op 2.17
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.5340 ms/op 2.8773 ms/op 1.23
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.9035 ms/op 1.0999 ms/op 2.64
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.6864 ms/op 2.8862 ms/op 1.28
Tree 40 250000 create 207.13 ms/op 181.91 ms/op 1.14
Tree 40 250000 get(125000) 141.84 ns/op 105.40 ns/op 1.35
Tree 40 250000 set(125000) 650.70 ns/op 546.82 ns/op 1.19
Tree 40 250000 toArray() 15.203 ms/op 12.549 ms/op 1.21
Tree 40 250000 iterate all - toArray() + loop 15.323 ms/op 13.414 ms/op 1.14
Tree 40 250000 iterate all - get(i) 51.627 ms/op 40.266 ms/op 1.28
Array 250000 create 2.8849 ms/op 2.2797 ms/op 1.27
Array 250000 clone - spread 1.4484 ms/op 1.0724 ms/op 1.35
Array 250000 get(125000) 0.42500 ns/op 0.57100 ns/op 0.74
Array 250000 set(125000) 0.45000 ns/op 0.57500 ns/op 0.78
Array 250000 iterate all - loop 110.60 us/op 72.179 us/op 1.53
phase0 afterProcessEpoch - 250000 vs - 7PWei 88.416 ms/op 75.667 ms/op 1.17
Array.fill - length 1000000 3.6249 ms/op 2.3876 ms/op 1.52
Array push - length 1000000 17.628 ms/op 15.229 ms/op 1.16
Array.get 0.28991 ns/op 0.25526 ns/op 1.14
Uint8Array.get 0.44471 ns/op 0.34187 ns/op 1.30
phase0 beforeProcessEpoch - 250000 vs - 7PWei 19.087 ms/op 12.622 ms/op 1.51
altair processEpoch - mainnet_e81889 313.01 ms/op 314.54 ms/op 1.00
mainnet_e81889 - altair beforeProcessEpoch 19.382 ms/op 16.895 ms/op 1.15
mainnet_e81889 - altair processJustificationAndFinalization 16.116 us/op 12.113 us/op 1.33
mainnet_e81889 - altair processInactivityUpdates 5.7933 ms/op 4.1838 ms/op 1.38
mainnet_e81889 - altair processRewardsAndPenalties 55.724 ms/op 51.486 ms/op 1.08
mainnet_e81889 - altair processRegistryUpdates 2.4460 us/op 1.6040 us/op 1.52
mainnet_e81889 - altair processSlashings 452.00 ns/op 650.00 ns/op 0.70
mainnet_e81889 - altair processEth1DataReset 617.00 ns/op 681.00 ns/op 0.91
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.1945 ms/op 1.0471 ms/op 2.10
mainnet_e81889 - altair processSlashingsReset 5.2390 us/op 1.2690 us/op 4.13
mainnet_e81889 - altair processRandaoMixesReset 5.6780 us/op 1.5590 us/op 3.64
mainnet_e81889 - altair processHistoricalRootsUpdate 778.00 ns/op 406.00 ns/op 1.92
mainnet_e81889 - altair processParticipationFlagUpdates 2.9580 us/op 3.0360 us/op 0.97
mainnet_e81889 - altair processSyncCommitteeUpdates 668.00 ns/op 751.00 ns/op 0.89
mainnet_e81889 - altair afterProcessEpoch 93.295 ms/op 72.847 ms/op 1.28
capella processEpoch - mainnet_e217614 1.2807 s/op 1.0752 s/op 1.19
mainnet_e217614 - capella beforeProcessEpoch 64.354 ms/op 62.126 ms/op 1.04
mainnet_e217614 - capella processJustificationAndFinalization 23.631 us/op 6.5530 us/op 3.61
mainnet_e217614 - capella processInactivityUpdates 17.653 ms/op 14.947 ms/op 1.18
mainnet_e217614 - capella processRewardsAndPenalties 236.00 ms/op 266.30 ms/op 0.89
mainnet_e217614 - capella processRegistryUpdates 19.125 us/op 6.8980 us/op 2.77
mainnet_e217614 - capella processSlashings 658.00 ns/op 437.00 ns/op 1.51
mainnet_e217614 - capella processEth1DataReset 625.00 ns/op 430.00 ns/op 1.45
mainnet_e217614 - capella processEffectiveBalanceUpdates 8.9570 ms/op 5.7529 ms/op 1.56
mainnet_e217614 - capella processSlashingsReset 9.9680 us/op 2.5450 us/op 3.92
mainnet_e217614 - capella processRandaoMixesReset 6.2530 us/op 3.0930 us/op 2.02
mainnet_e217614 - capella processHistoricalRootsUpdate 755.00 ns/op 678.00 ns/op 1.11
mainnet_e217614 - capella processParticipationFlagUpdates 2.7730 us/op 1.3880 us/op 2.00
mainnet_e217614 - capella afterProcessEpoch 241.50 ms/op 181.99 ms/op 1.33
phase0 processEpoch - mainnet_e58758 424.59 ms/op 333.05 ms/op 1.27
mainnet_e58758 - phase0 beforeProcessEpoch 93.635 ms/op 76.500 ms/op 1.22
mainnet_e58758 - phase0 processJustificationAndFinalization 16.912 us/op 11.884 us/op 1.42
mainnet_e58758 - phase0 processRewardsAndPenalties 39.683 ms/op 35.906 ms/op 1.11
mainnet_e58758 - phase0 processRegistryUpdates 13.061 us/op 6.3220 us/op 2.07
mainnet_e58758 - phase0 processSlashings 504.00 ns/op 717.00 ns/op 0.70
mainnet_e58758 - phase0 processEth1DataReset 619.00 ns/op 734.00 ns/op 0.84
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.1243 ms/op 858.57 us/op 2.47
mainnet_e58758 - phase0 processSlashingsReset 3.5270 us/op 2.2260 us/op 1.58
mainnet_e58758 - phase0 processRandaoMixesReset 4.7960 us/op 3.3230 us/op 1.44
mainnet_e58758 - phase0 processHistoricalRootsUpdate 357.00 ns/op 671.00 ns/op 0.53
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.3190 us/op 2.7810 us/op 1.19
mainnet_e58758 - phase0 afterProcessEpoch 79.984 ms/op 58.701 ms/op 1.36
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.9375 ms/op 950.60 us/op 2.04
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.6623 ms/op 1.6748 ms/op 1.59
altair processInactivityUpdates - 250000 normalcase 18.484 ms/op 18.625 ms/op 0.99
altair processInactivityUpdates - 250000 worstcase 17.911 ms/op 18.318 ms/op 0.98
phase0 processRegistryUpdates - 250000 normalcase 8.1880 us/op 5.8140 us/op 1.41
phase0 processRegistryUpdates - 250000 badcase_full_deposits 372.75 us/op 295.22 us/op 1.26
phase0 processRegistryUpdates - 250000 worstcase 0.5 130.76 ms/op 116.89 ms/op 1.12
altair processRewardsAndPenalties - 250000 normalcase 47.318 ms/op 45.762 ms/op 1.03
altair processRewardsAndPenalties - 250000 worstcase 35.629 ms/op 41.212 ms/op 0.86
phase0 getAttestationDeltas - 250000 normalcase 9.2747 ms/op 6.1615 ms/op 1.51
phase0 getAttestationDeltas - 250000 worstcase 7.3136 ms/op 5.6895 ms/op 1.29
phase0 processSlashings - 250000 worstcase 102.88 us/op 51.070 us/op 2.01
altair processSyncCommitteeUpdates - 250000 133.72 ms/op 96.301 ms/op 1.39
BeaconState.hashTreeRoot - No change 243.00 ns/op 428.00 ns/op 0.57
BeaconState.hashTreeRoot - 1 full validator 121.33 us/op 133.73 us/op 0.91
BeaconState.hashTreeRoot - 32 full validator 1.0777 ms/op 1.4531 ms/op 0.74
BeaconState.hashTreeRoot - 512 full validator 14.481 ms/op 13.092 ms/op 1.11
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 158.22 us/op 143.73 us/op 1.10
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.2003 ms/op 1.7458 ms/op 1.26
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 32.029 ms/op 24.714 ms/op 1.30
BeaconState.hashTreeRoot - 1 balances 152.34 us/op 122.44 us/op 1.24
BeaconState.hashTreeRoot - 32 balances 1.3105 ms/op 1.1800 ms/op 1.11
BeaconState.hashTreeRoot - 512 balances 12.736 ms/op 7.1832 ms/op 1.77
BeaconState.hashTreeRoot - 250000 balances 217.05 ms/op 185.34 ms/op 1.17
aggregationBits - 2048 els - zipIndexesInBitList 37.232 us/op 19.551 us/op 1.90
byteArrayEquals 32 55.655 ns/op 44.929 ns/op 1.24
Buffer.compare 32 17.836 ns/op 14.435 ns/op 1.24
byteArrayEquals 1024 1.6468 us/op 1.2545 us/op 1.31
Buffer.compare 1024 25.986 ns/op 23.149 ns/op 1.12
byteArrayEquals 16384 25.889 us/op 19.910 us/op 1.30
Buffer.compare 16384 210.82 ns/op 187.22 ns/op 1.13
byteArrayEquals 123687377 198.83 ms/op 146.65 ms/op 1.36
Buffer.compare 123687377 8.0234 ms/op 4.1867 ms/op 1.92
byteArrayEquals 32 - diff last byte 53.260 ns/op 46.269 ns/op 1.15
Buffer.compare 32 - diff last byte 17.324 ns/op 16.602 ns/op 1.04
byteArrayEquals 1024 - diff last byte 1.6046 us/op 1.2553 us/op 1.28
Buffer.compare 1024 - diff last byte 26.962 ns/op 24.952 ns/op 1.08
byteArrayEquals 16384 - diff last byte 26.792 us/op 19.972 us/op 1.34
Buffer.compare 16384 - diff last byte 211.36 ns/op 193.10 ns/op 1.09
byteArrayEquals 123687377 - diff last byte 196.42 ms/op 150.10 ms/op 1.31
Buffer.compare 123687377 - diff last byte 8.5431 ms/op 4.2286 ms/op 2.02
byteArrayEquals 32 - random bytes 5.3780 ns/op 4.9450 ns/op 1.09
Buffer.compare 32 - random bytes 18.507 ns/op 16.500 ns/op 1.12
byteArrayEquals 1024 - random bytes 5.4620 ns/op 4.8520 ns/op 1.13
Buffer.compare 1024 - random bytes 18.477 ns/op 16.787 ns/op 1.10
byteArrayEquals 16384 - random bytes 5.4170 ns/op 4.8380 ns/op 1.12
Buffer.compare 16384 - random bytes 18.323 ns/op 15.961 ns/op 1.15
byteArrayEquals 123687377 - random bytes 6.8200 ns/op 7.7000 ns/op 0.89
Buffer.compare 123687377 - random bytes 19.770 ns/op 19.710 ns/op 1.00
regular array get 100000 times 42.377 us/op 30.236 us/op 1.40
wrappedArray get 100000 times 34.613 us/op 30.597 us/op 1.13
arrayWithProxy get 100000 times 13.626 ms/op 10.432 ms/op 1.31
ssz.Root.equals 47.675 ns/op 42.806 ns/op 1.11
byteArrayEquals 46.961 ns/op 36.742 ns/op 1.28
Buffer.compare 10.833 ns/op 8.4780 ns/op 1.28
shuffle list - 16384 els 6.5573 ms/op 5.4716 ms/op 1.20
shuffle list - 250000 els 93.976 ms/op 80.935 ms/op 1.16
processSlot - 1 slots 15.067 us/op 9.6150 us/op 1.57
processSlot - 32 slots 3.5228 ms/op 1.7932 ms/op 1.96
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 40.131 ms/op 34.521 ms/op 1.16
getCommitteeAssignments - req 1 vs - 250000 vc 2.2667 ms/op 1.7383 ms/op 1.30
getCommitteeAssignments - req 100 vs - 250000 vc 4.4122 ms/op 3.3761 ms/op 1.31
getCommitteeAssignments - req 1000 vs - 250000 vc 4.7421 ms/op 3.6541 ms/op 1.30
findModifiedValidators - 10000 modified validators 265.10 ms/op 241.71 ms/op 1.10
findModifiedValidators - 1000 modified validators 216.60 ms/op 146.18 ms/op 1.48
findModifiedValidators - 100 modified validators 165.09 ms/op 139.44 ms/op 1.18
findModifiedValidators - 10 modified validators 166.79 ms/op 130.59 ms/op 1.28
findModifiedValidators - 1 modified validators 138.91 ms/op 117.05 ms/op 1.19
findModifiedValidators - no difference 190.54 ms/op 129.12 ms/op 1.48
compare ViewDUs 3.2867 s/op 3.2021 s/op 1.03
compare each validator Uint8Array 1.9368 s/op 1.5893 s/op 1.22
compare ViewDU to Uint8Array 1.2100 s/op 629.44 ms/op 1.92
migrate state 1000000 validators, 24 modified, 0 new 940.30 ms/op 796.46 ms/op 1.18
migrate state 1000000 validators, 1700 modified, 1000 new 1.1774 s/op 1.0315 s/op 1.14
migrate state 1000000 validators, 3400 modified, 2000 new 1.3004 s/op 1.2737 s/op 1.02
migrate state 1500000 validators, 24 modified, 0 new 913.41 ms/op 822.76 ms/op 1.11
migrate state 1500000 validators, 1700 modified, 1000 new 1.1908 s/op 1.0823 s/op 1.10
migrate state 1500000 validators, 3400 modified, 2000 new 1.3566 s/op 1.2984 s/op 1.04
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.6800 ns/op 5.8800 ns/op 0.80
state getBlockRootAtSlot - 250000 vs - 7PWei 510.57 ns/op 957.95 ns/op 0.53
computeProposers - vc 250000 6.9839 ms/op 6.0749 ms/op 1.15
computeEpochShuffling - vc 250000 94.359 ms/op 79.464 ms/op 1.19
getNextSyncCommittee - vc 250000 126.48 ms/op 104.25 ms/op 1.21
computeSigningRoot for AttestationData 26.918 us/op 25.480 us/op 1.06
hash AttestationData serialized data then Buffer.toString(base64) 1.5952 us/op 1.1911 us/op 1.34
toHexString serialized data 875.65 ns/op 795.37 ns/op 1.10
Buffer.toString(base64) 180.77 ns/op 147.12 ns/op 1.23
nodejs block root to RootHex using toHex 146.41 ns/op 115.50 ns/op 1.27
nodejs block root to RootHex using toRootHex 102.73 ns/op 76.278 ns/op 1.35
browser block root to RootHex using the deprecated toHexString 223.73 ns/op 212.63 ns/op 1.05
browser block root to RootHex using toHex 182.66 ns/op 165.91 ns/op 1.10
browser block root to RootHex using toRootHex 161.18 ns/op 149.85 ns/op 1.08

by benchmarkbot/action

@twoeths twoeths marked this pull request as ready for review September 26, 2024 07:33
@twoeths twoeths requested a review from a team as a code owner September 26, 2024 07:33
@twoeths twoeths merged commit 77006ea into unstable Sep 27, 2024
@twoeths twoeths deleted the te/fix_buffer_pool_usage branch September 27, 2024 01:15
@codecov
Copy link

codecov bot commented Sep 27, 2024

Codecov Report

Attention: Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.

Project coverage is 50.82%. Comparing base (bf60786) to head (c400e7c).
Report is 3 commits behind head on unstable.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #7109      +/-   ##
============================================
- Coverage     50.83%   50.82%   -0.01%     
============================================
  Files           597      597              
  Lines         39789    39790       +1     
  Branches       2065     2057       -8     
============================================
- Hits          20226    20224       -2     
- Misses        19563    19566       +3

@twoeths twoeths mentioned this pull request Sep 27, 2024
Merged
philknows pushed a commit that referenced this pull request Sep 27, 2024
@twoeths @philknows
fix: state serialization time (#7109)
dd51441 
* fix: release memory to BufferPool

* fix: correct the serializeState metric

* fix: check types

* chore: fix lint
@wemeetagain
Copy link
Member

wemeetagain commented Nov 11, 2024

🎉 This PR is included in v1.23.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@twoeths @wemeetagain