eip6110: Queue deposit requests and apply them during epoch processing #3818

mkalinin · 2024-06-26T08:16:01Z

Description

Introduces majority of things described in #3689 (comment), specifically:

Queues full deposit request objects and process them on the epoch processing, rate limits a number of deposit signature verifications per epoch by the max number of deposit requests allowed to be processed (MAX_PENDING_DEPOSITS_PER_EPOCH)
Finalizes deposit request position in the queue before applying it to the state. The outcome is that no new validator is created before the corresponding deposit is finalized hence no fork aware (pubkey, index) cache is required.
Eth1 bridge deposits are processed as before this change, the corresponding entry with the entire balance is created in the pending_deposits queue to ensure the deposit will pass through the activation churn.
No deposit request is processed until Eth1 bridge covers the gap during the transition period. This prevents the scenario when deposit request can easily front-run the Eth1 bridge deposit because of the large follow distance and create validator with the same pubkey but malicious withdrawal credentials.
Refactors process_pending_deposits by moving deposit applying to apply_pending_deposit

Other than the above, the logic of the previous process_pending_balance_deposits function remains.

~~Additionally, moves switch_to_compounding_validator to process_deposit_request~~. Removes switch_to_compounding_validator from the deposit flow in favour of #3918.

Points of discussion and open questions

Due to refactor that facilitates spec reading there are two places where validatorIndex is looked up by the pubkey (process_pending_deposits and apply_pending_deposit functions), this happens for each pending deposit that is being processed. There is a less readable workaround where validatorIndex is resolved once, but I am not sure if it is necessary because client implementations are likely to use cache in both cases.
MAX_PENDING_DEPOSITS_PER_EPOCH_PROCESSING is set to 16, should it be more? The activation churn is limited by 256 ETH which would mean 256 signature verifications in the worst case. The most optimal scenario of an attack resulting in 256 sig verifications would entail creation of 128 1-ETH validators (the first deposit would be to create a validator with eth1 creds, the second would be to switch the validator to compounding creds); it seems that employing such an attack is quite discouraging because it requires decent amount of ETH with no substantial gain. Should we even have this limitation if the churn could do the limit?
This PR does not explore an optimisation with two queues that was discussed previously, it is planned to be a part of a separate PR once this PR gets merged.

Testing

Many thanks to @james-prysm for helping to write and adjust the tests!

apply_pending_deposits checks
Deposit transition period checks -- the logic has changed and part of it is in process_pending_deposit
Repeat test_process_deposit_requests scenarious in application to process_pending_deposit
Finalized deposit position check
MAX_PENDING_DEPOSITS_PER_EPOCH_PROCESSING check
Revisit test_process_pending_deposit (is_deposit_applied, is_churn_limit_reached etc)

Link to a more comprehensive test plan: https://hackmd.io/mqV7TaFyTq--vEMcB5msJw

Supersedes #3689

dapplion · 2024-06-26T12:01:24Z

specs/electra/beacon-chain.md

+    else:
+        validator_index = ValidatorIndex(validator_pubkeys.index(deposit.pubkey))
+        validator = state.validators[validator_index]
+        # Validator is exiting, do not consume the churn


From @mkalinin this line is motivated by

Handle top-ups to exiting/exited validators #3776

Why doesn't this condition validator.exit_epoch < FAR_FUTURE_EPOCH match

consensus-specs/specs/electra/beacon-chain.md

Lines 870 to 871 in 43e7344

if get_current_epoch(state) <= validator.withdrawable_epoch:

return False

The idea is to postpone deposit processing for an exiting validator until it has fully withdrawn. And then do the top-up without affecting WS period.

james-prysm · 2024-06-26T15:39:27Z

Due to refactor that facilitates spec reading there are two places where validatorIndex is looked up by the pubkey (get_activation_churn_consumption and apply_pending_deposit functions), this happens for each pending deposit that is being processed. There is a less readable workaround where validatorIndex is resolved once, but I am not sure if it is necessary because client implementations are likely to use cache in both cases.

~~does this mean that the cache should continue to be used as opposed to the previous pr that looked for its removal in #3689~~

reading through I think my question is answered. we need it based on this PR

specs/electra/beacon-chain.md

…process_pending_deposits.py Co-authored-by: Mikhail Kalinin <[email protected]>

…ikhail into deposit-queue

specs/electra/beacon-chain.md

ppopth · 2024-10-02T11:52:25Z

specs/electra/beacon-chain.md

+
+```python
+def get_validator_from_deposit(pubkey: BLSPubkey, withdrawal_credentials: Bytes32, amount: uint64) -> Validator:
+    if is_compounding_withdrawal_credential(withdrawal_credentials):


Can you use get_max_effective_balance instead? You can create an instance of Validator first and then call get_max_effective_balance.

ppopth · 2024-10-02T16:55:59Z

specs/electra/beacon-chain.md

+            withdrawal_credentials=validator.withdrawal_credentials,
+            amount=excess_balance,
+            signature=bls.G2_POINT_AT_INFINITY,
+            slot=GENESIS_SLOT,


What is the consequence if I set this to state.slot instead?

Then the excess balance will wait for when state.slot is finalized before being applied even in the case when there is enough activation churn to let the balance in. The GENESIS_SLOT is used to bypass that finality check as that slot is always finalized

ppopth · 2024-10-02T17:46:24Z

specs/electra/beacon-chain.md

+            signature=bls.G2_POINT_AT_INFINITY,
+            slot=GENESIS_SLOT,


I think we should define meaningful constants for both bls.G2_POINT_AT_INFINITY and GENESIS_SLOT. How about aliasing them as

UNSET_SIGNATURE and

UNSET_DEPOSIT_SLOT

By seeing only the point at infinity and the genesis slot, the readers wouldn't know what they mean.

I have put comments instead of aliasing those two. It is a bit difficult to incorporate the semantics into these constants in both cases

jtraglia · 2024-10-03T16:50:36Z

AIUI we would merge in #3918 which will simplify this PR a bit further
does this align with your understanding @mkalinin ?

yes, the plan is to wait for that PR and then merge this one

@mkalinin We've merged #3918. Any additional changes you'd like to make?

jtraglia · 2024-10-03T19:55:59Z

After merging in dev, the test_pending_consolidation_future_epoch test fails. I probably messed something up but it's not clear to me what is wrong. Do you know?

mkalinin · 2024-10-04T06:31:50Z

After merging in dev, the test_pending_consolidation_future_epoch test fails. I probably messed something up but it's not clear to me what is wrong. Do you know?

Fixed now (the change of this test from the dev wasn’t applied to this branch)

@mkalinin We've merged #3918. Any additional changes you'd like to make?

The corresponding changes were already made, so the PR is ready for review and merging into dev

tests/core/pyspec/eth2spec/test/electra/epoch_processing/test_apply_pending_deposit.py

hwwhww · 2024-10-04T10:57:02Z

tests/core/pyspec/eth2spec/test/helpers/deposits.py

      - post-state ('post').
-    If ``valid == False``, run expecting ``AssertionError``
    """
+    assert is_post_electra(spec)


is it needed?

Might be not as these methods are called from post electra tests only

hwwhww · 2024-10-04T10:57:31Z

tests/core/pyspec/eth2spec/test/helpers/deposits.py

+      - pre-state ('pre')
+      - post-state ('post').
+    """
+    assert is_post_electra(spec)


is it needed?

tests/generators/epoch_processing/main.py

mkalinin added 6 commits June 20, 2024 15:48

Add pending_deposits queue and queue deposit requests

fd75470

Refactor pending deposit processing

2a202c8

Limit number of pending deposits to be processed per epoch

0b7fef3

Process deposit requests after Eth1 bridge deposits are applied

5c3a51a

Finalize deposit requests before processing

179ee8e

Refactor process_pending_deposits

43e7344

This was referenced Jun 26, 2024

Finalize deposit before applying it to a state #3689

Closed

Consensus-layer Call 136 ethereum/pm#1084

Closed

dapplion reviewed Jun 26, 2024

View reviewed changes

james-prysm mentioned this pull request Jun 26, 2024

Finalized validator index cache saving OffchainLabs/prysm#14146

Merged

james-prysm reviewed Jun 26, 2024

View reviewed changes

specs/electra/beacon-chain.md Outdated Show resolved Hide resolved

mkalinin requested review from fradamt and hwwhww June 28, 2024 06:16

Merge remote-tracking branch 'origin/dev' into deposit-queue

8f6b1e0

james-prysm reviewed Jul 1, 2024

View reviewed changes

specs/electra/beacon-chain.md Outdated Show resolved Hide resolved

james-prysm and others added 13 commits July 1, 2024 15:26

added some simple tests

e1a4d8b

Fix off-by-one found by @james-prysm

5c0a8d3

Update tests/core/pyspec/eth2spec/test/electra/epoch_processing/test_…

2af84d8

…process_pending_deposits.py Co-authored-by: Mikhail Kalinin <[email protected]>

merging upstream

8099f39

adding new test

750f7a1

Merge branch 'deposit-queue' of github.com:james-prysm/eth2.0-specs-m…

077edf8

…ikhail into deposit-queue

apply pending deposit tests

b31fa63

fixing comment

b2cd3ce

preston's feedback

03071df

Merge remote-tracking branch 'origin/dev' into deposit-queue

5a2f887

Refactor postponed deposit processing

ee4da7b

Fix electra fork tests

6ed15b2

Merge remote-tracking branch 'upstream/deposit-queue' into deposit-queue

093590e

james-prysm reviewed Jul 9, 2024

View reviewed changes

specs/electra/beacon-chain.md Outdated Show resolved Hide resolved

mkalinin added 2 commits October 2, 2024 13:08

Merge branch 'dev' into deposit-queue

4ee4105

Fix fork logic

a04e4c3

ppopth reviewed Oct 2, 2024

View reviewed changes

james-prysm mentioned this pull request Oct 2, 2024

Remove finalized validator index to pubkey cache OffchainLabs/prysm#14497

Merged

3 tasks

ppopth reviewed Oct 2, 2024

View reviewed changes

james-prysm mentioned this pull request Oct 2, 2024

Execution API: execution requests serialization OffchainLabs/prysm#14498

Closed

3 tasks

lucassaldanha mentioned this pull request Oct 3, 2024

Consensus-layer Call 143 ethereum/pm#1158

Closed

hwwhww mentioned this pull request Oct 3, 2024

Release v1.5.0-alpha.7 #3954

Merged

6 tasks

Apply suggestions by @ppopth

beb2718

jtraglia and others added 2 commits October 3, 2024 13:49

Merge branch 'dev' into deposit-queue

358378d

Fix two tests by replacing old PendingBalanceDeposit

82d2bc0

Fix consolidation test

7705984

hwwhww reviewed Oct 4, 2024

View reviewed changes

Rename test_apply_pending_deposit to test_process_pending_deposit

77d2865

jtraglia approved these changes Oct 4, 2024

View reviewed changes

jtraglia added 2 commits October 4, 2024 10:57

Rename test_apply_pending_deposit* to test_process_pending_deposit*

3befc42

Use pending_deposits format for both test files

ad42273

jtraglia merged commit 7df1ce3 into ethereum:dev Oct 4, 2024

pk910 mentioned this pull request Oct 4, 2024

update electra specs to v1.5.0-alpha.7 attestantio/go-eth2-client#167

Merged

1 task

This was referenced Oct 4, 2024

Creating PendingDeposit container Consensys/teku#8678

Merged

EIP-6110: Queue deposit requests and apply them during epoch processing Consensys/teku#8681

Merged

ensi321 mentioned this pull request Oct 7, 2024

feat: process deposit requests at epoch processing ChainSafe/lodestar#7131

Merged

tersec mentioned this pull request Oct 8, 2024

[WIP] epbs (eip-7732) status-im/nimbus-eth2#6443

Closed

pawanjay176 mentioned this pull request Oct 16, 2024

Electra alpha8 spec updates sigp/lighthouse#6496

Merged

mkalinin mentioned this pull request Oct 18, 2024

Simplify validator activation flow #3983

Closed

1 task

ensi321 mentioned this pull request Nov 17, 2024

feat: remove unfinalized pubkey cache ChainSafe/lodestar#7230

Merged

	if get_current_epoch(state) <= validator.withdrawable_epoch:
	return False

eip6110: Queue deposit requests and apply them during epoch processing #3818

eip6110: Queue deposit requests and apply them during epoch processing #3818

Uh oh!

Conversation

mkalinin commented Jun 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Points of discussion and open questions

Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

james-prysm commented Jun 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jtraglia commented Oct 3, 2024

Uh oh!

jtraglia commented Oct 3, 2024

Uh oh!

mkalinin commented Oct 4, 2024

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

mkalinin commented Jun 26, 2024 •

edited

Loading

james-prysm commented Jun 26, 2024 •

edited

Loading