[NPoS] Paging reward payouts in order to scale rewardable nominators by Ank4n · Pull Request #1189 · paritytech/polkadot-sdk

Ank4n · 2023-08-26T18:26:39Z

helps #439.
closes #473.

PR link in the older substrate repository: paritytech/substrate#13498.

Context

Rewards payout is processed today in a single block and limited to MaxNominatorRewardedPerValidator. This number is currently 512 on both Kusama and Polkadot.

This PR tries to scale the nominators payout to an unlimited count in a multi-block fashion. Exposures are stored in pages, with each page capped to a certain number (MaxExposurePageSize). Starting out, this number would be the same as MaxNominatorRewardedPerValidator, but eventually, this number can be lowered through new runtime upgrades to limit the rewardeable nominators per dispatched call instruction.

The changes in the PR are backward compatible.

How payouts would work like after this change

Staking exposes two calls, 1) the existing payout_stakers and 2) payout_stakers_by_page.

payout_stakers

This remains backward compatible with no signature change. If for a given era a validator has multiple pages, they can call payout_stakers multiple times. The pages are executed in an ascending sequence and the runtime takes care of preventing double claims.

payout_stakers_by_page

Very similar to payout_stakers but also accepts an extra param page_index. An account can choose to payout rewards only for an explicitly passed page_index.

Lets look at an example scenario
Given an active validator on Kusama had 1100 nominators, MaxExposurePageSize set to 512 for Era e. In order to pay out rewards to all nominators, the caller would need to call payout_stakers 3 times.

payout_stakers(origin, stash, e) => will pay the first 512 nominators.
payout_stakers(origin, stash, e) => will pay the second set of 512 nominators.
payout_stakers(origin, stash, e) => will pay the last set of 76 nominators.
...
payout_stakers(origin, stash, e) => calling it the 4th time would return an error InvalidPage.

The above calls can also be replaced by payout_stakers_by_page and passing a page_index explicitly.

Commission note

Validator commission is paid out in chunks across all the pages where each commission chunk is proportional to the total stake of the current page. This implies higher the total stake of a page, higher will be the commission. If all the pages of a validator's single era are paid out, the sum of commission paid to the validator across all pages should be equal to what the commission would have been if we had a non-paged exposure.

Migration Note

Strictly speaking, we did not need to bump our storage version since there is no migration of storage in this PR. But it is still useful to mark a storage upgrade for the following reasons:

New storage items are introduced in this PR while some older storage items are deprecated.
For the next HistoryDepth eras, the exposure would be incrementally migrated to its corresponding paged storage item.
Runtimes using staking pallet would strictly need to wait at least HistoryDepth eras with current upgraded version (14) for the migration to complete. At some era E such that E > era_at_which_V14_gets_into_effect + HistoryDepth, we will upgrade to version X which will remove the deprecated storage items.
In other words, it is a strict requirement that E_x - E₁₄ > HistoryDepth, where
E_x = Era at which deprecated storages are removed from runtime,
E₁₄ = Era at which runtime is upgraded to version 14.
For Polkadot and Kusama, there is a tracker ticket to clean up the deprecated storage items.

Storage Changes

Added

ErasStakersOverview
ClaimedRewards
ErasStakersPaged

Deprecated

The following can be cleaned up after 84 eras which is tracked here.

ErasStakers.
ErasStakersClipped.
StakingLedger.claimed_rewards, renamed to StakingLedger.legacy_claimed_rewards.

Config Changes

Renamed MaxNominatorRewardedPerValidator to MaxExposurePageSize.

TODO

Followup issues

Runtime api for deprecated ErasStakers storage item

franciscoaguirre · 2023-09-05T18:36:11Z

substrate/frame/staking/src/lib.rs

+	/// Returns the number of pages of exposure a validator has for the given era.
+	///
+	/// For eras where paged exposure does not exist, this returns 1 to keep backward compatibility.
+	pub(crate) fn get_page_count(era: EraIndex, validator: &T::AccountId) -> PageIndex {


This shouldn't return PageIndex, just usize

Its a good point and I have thought about it too. But isn't page count always going to have the same type as PageIndex?

Yeah it is, it works perfectly fine and will probably never change, it's just a minor naming thing. By reading PageIndex, you would expect you get back an actual page index, not the number of pages. Even though they are both numbers, it's better to be expressive with type names.

Yeah agree about the readability but this is stored in storage and we can't use usize for that, so am keeping it as same type as PageIndex. It also is used to compare against PageIndex so it makes sense to have them as same type.

What do you think about renaming the type to just call it Page?

You can still use u32, which is the type aliased to PageIndex, that'd be better in my opinion

franciscoaguirre · 2023-09-05T18:39:41Z

substrate/frame/staking/src/lib.rs

+/// Wrapper struct for Era related information. It is not a pure encapsulation as these storage
+/// items can be accessed directly but nevertheless, its recommended to use `EraInfo` where we
+/// can and add more functions to it as needed.
+pub(crate) struct EraInfo<T>(sp_std::marker::PhantomData<T>);


This could go on another file so as to not keep on bloating the lib.rs

substrate/frame/staking/src/pallet/mod.rs

substrate/primitives/staking/src/lib.rs

Ank4n · 2023-09-06T08:47:53Z

polkadot/runtime/kusama/src/lib.rs


 	type MaxAuthorities = MaxAuthorities;
-	type MaxNominators = MaxNominatorRewardedPerValidator;
+	type MaxNominators = MaxExposurePageSize;


https://github.com/paritytech/substrate/pull/13498/files/29f800bf4eaa0d4d9a837591461e7ee6e438679f..ec0694bfb69c5625ba6d2fe76da8e0446dcd71d7#r1299430753

https://github.com/paritytech-secops/srlabs_findings/issues/2#issuecomment-1685860874

Isn't this a pretty bad name? I belive what pallet_babe wants to convey here is MaxNominatorsPerValidator?

Also, indeed, the correct value for this should be something like max_pages * page_size, but I trust that this will come in a follow-up.

Its terrible but its just more apparent now :(. MaxNominatorRewardedPerValidator was also wrong and exactly same thing.

I will add a comment about this in the code. Not sure what else I can do in the scope of current PR.

kianenigma · 2023-09-06T10:08:24Z

substrate/frame/grandpa/src/mock.rs

 	type SessionInterface = Self;
 	type UnixTime = pallet_timestamp::Pallet<Test>;
 	type EraPayout = pallet_staking::ConvertCurve<RewardCurve>;
-	type MaxNominatorRewardedPerValidator = ConstU32<64>;


can't wait to use derive_impl for staking as well, it is one of those pallet that is used in a couple of mocks.

…aritytech#1189) helps paritytech#439. closes paritytech#473. PR link in the older substrate repository: paritytech/substrate#13498. # Context Rewards payout is processed today in a single block and limited to `MaxNominatorRewardedPerValidator`. This number is currently 512 on both Kusama and Polkadot. This PR tries to scale the nominators payout to an unlimited count in a multi-block fashion. Exposures are stored in pages, with each page capped to a certain number (`MaxExposurePageSize`). Starting out, this number would be the same as `MaxNominatorRewardedPerValidator`, but eventually, this number can be lowered through new runtime upgrades to limit the rewardeable nominators per dispatched call instruction. The changes in the PR are backward compatible. ## How payouts would work like after this change Staking exposes two calls, 1) the existing `payout_stakers` and 2) `payout_stakers_by_page`. ### payout_stakers This remains backward compatible with no signature change. If for a given era a validator has multiple pages, they can call `payout_stakers` multiple times. The pages are executed in an ascending sequence and the runtime takes care of preventing double claims. ### payout_stakers_by_page Very similar to `payout_stakers` but also accepts an extra param `page_index`. An account can choose to payout rewards only for an explicitly passed `page_index`. **Lets look at an example scenario** Given an active validator on Kusama had 1100 nominators, `MaxExposurePageSize` set to 512 for Era e. In order to pay out rewards to all nominators, the caller would need to call `payout_stakers` 3 times. - `payout_stakers(origin, stash, e)` => will pay the first 512 nominators. - `payout_stakers(origin, stash, e)` => will pay the second set of 512 nominators. - `payout_stakers(origin, stash, e)` => will pay the last set of 76 nominators. ... - `payout_stakers(origin, stash, e)` => calling it the 4th time would return an error `InvalidPage`. The above calls can also be replaced by `payout_stakers_by_page` and passing a `page_index` explicitly. ## Commission note Validator commission is paid out in chunks across all the pages where each commission chunk is proportional to the total stake of the current page. This implies higher the total stake of a page, higher will be the commission. If all the pages of a validator's single era are paid out, the sum of commission paid to the validator across all pages should be equal to what the commission would have been if we had a non-paged exposure. ### Migration Note Strictly speaking, we did not need to bump our storage version since there is no migration of storage in this PR. But it is still useful to mark a storage upgrade for the following reasons: - New storage items are introduced in this PR while some older storage items are deprecated. - For the next `HistoryDepth` eras, the exposure would be incrementally migrated to its corresponding paged storage item. - Runtimes using staking pallet would strictly need to wait at least `HistoryDepth` eras with current upgraded version (14) for the migration to complete. At some era `E` such that `E > era_at_which_V14_gets_into_effect + HistoryDepth`, we will upgrade to version X which will remove the deprecated storage items. In other words, it is a strict requirement that Ex - E14 > `HistoryDepth`, where Ex = Era at which deprecated storages are removed from runtime, E14 = Era at which runtime is upgraded to version 14. - For Polkadot and Kusama, there is a [tracker ticket](paritytech#433) to clean up the deprecated storage items. ### Storage Changes #### Added - ErasStakersOverview - ClaimedRewards - ErasStakersPaged #### Deprecated The following can be cleaned up after 84 eras which is tracked [here](paritytech#433). - ErasStakers. - ErasStakersClipped. - StakingLedger.claimed_rewards, renamed to StakingLedger.legacy_claimed_rewards. ### Config Changes - Renamed MaxNominatorRewardedPerValidator to MaxExposurePageSize. ### TODO - [x] Tracker ticket for cleaning up the old code after 84 eras. - [x] Add companion. - [x] Redo benchmarks before merge. - [x] Add Changelog for pallet_staking. - [x] Pallet should be configurable to enable/disable paged rewards. - [x] Commission payouts are distributed across pages. - [x] Review documentation thoroughly. - [x] Rename `MaxNominatorRewardedPerValidator` -> `MaxExposurePageSize`. - [x] NMap for `ErasStakersPaged`. - [x] Deprecate ErasStakers. - [x] Integrity tests. ### Followup issues [Runtime api for deprecated ErasStakers storage item](paritytech#426) --------- Co-authored-by: Javier Viola <[email protected]> Co-authored-by: Ross Bulat <[email protected]> Co-authored-by: command-bot <>

…aritytech#2369) Addresses a bug caused by paritytech#1189. The changes are still not released yet, so would like to push the fix soon so it can go together with the release of the above PR. `fast_unstake` checks if a staker is exposed in an era. However, this fn is still returning whether the staker is exposed based on the old storage item. This PR fixes that by looking in both old and new exposure storages. Also adds some integrity tests for paged exposures.

…tytech#2459) The FastUnstake pallet tests were previously directly modifying storage items in the pallet-staking to alter exposure. However, due to the introduction of the [new paged exposure feature](paritytech#1189) these tests were not testing against correct storage items. This issue resulted in a bug that I didn't catch, which has been addressed in [this fix](paritytech#2369). This PR introduces a modification to how the pallet-fast-unstake handles exposure. It now utilizes `pallet-staking::EraInfo` to set or mutate Exposures.

squashed paged reward changes

6827b66

Ank4n self-assigned this Aug 26, 2023

paritytech-ci requested review from a team August 26, 2023 18:28

Ank4n added A3-backport Pull request is already reviewed well in another branch. D2-substantial Can be fixed by an experienced coder with a working knowledge of the codebase. labels Aug 26, 2023

Ank4n requested review from a team, gpestana, kianenigma and rossbulat August 26, 2023 18:29

Ank4n and others added 3 commits August 26, 2023 20:32

fix use different param

4738d9d

Merge branch 'master' into ankan/paged-rewards

913e945

Merge branch 'master' into ankan/paged-rewards

89dd5cf

rossbulat mentioned this pull request Aug 30, 2023

Nesting support for staking.payoutStakers Zondax/ledger-polkadot#199

Closed

Merge branch 'master' into ankan/paged-rewards

382f07c

franciscoaguirre reviewed Sep 5, 2023

View reviewed changes

Ank4n added 3 commits September 6, 2023 10:04

Merge branch 'master' into ankan/paged-rewards

dc9eb32

remove years from license

dbef906

Merge branch 'master' into ankan/paged-rewards

b3c12f8

Ank4n added T1-FRAME This PR/Issue is related to core FRAME, the framework. T8-polkadot This PR/Issue is related to/affects the Polkadot network. labels Sep 6, 2023

Ank4n added 2 commits September 6, 2023 10:23

format readme

03c9024

zepter format

95fc852

Ank4n commented Sep 6, 2023

View reviewed changes

kianenigma reviewed Sep 6, 2023

View reviewed changes

kianenigma approved these changes Sep 6, 2023

View reviewed changes

rename PageIndex to Page

4896403

rossbulat approved these changes Sep 14, 2023

View reviewed changes

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 9, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

cd9dafd

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 9, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

322526e

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 9, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

f756000

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 9, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

c31abba

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 10, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

1b86df5

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 10, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

975ce80

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 10, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

ccbfbb8

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 10, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

a71367a

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 10, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

fc05091

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 11, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

a735e66

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 12, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

6f202ae

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 12, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

63c61a4

henrikdent mentioned this pull request Jan 15, 2024

December update (polkadot-v1.4.0) dentnet/dentnet-node#39

Merged

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 22, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

c9b4182

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 23, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

aef83c9

claravanstaden pushed a commit to Snowfork/runtimes that referenced this pull request Feb 5, 2024

[pallet_staking] adapt paritytech/polkadot-sdk#1189

8b11c6e

github-actions bot mentioned this pull request Feb 19, 2024

Update substrate/polkadot/cumulus from v1.3.0 to v1.6.0 moondance-labs/tanssi#419

Closed

github-actions bot mentioned this pull request Mar 13, 2024

Update polkadot-sdk from v1.3.0 to v1.7.2 moonbeam-foundation/moonbeam#2703

Closed

tien mentioned this pull request Apr 23, 2024

fix: outdated active validators TalismanSociety/talisman-web#1097

Merged

Imod7 mentioned this pull request Apr 24, 2024

fix: add new staking storage items paritytech/substrate-api-sidecar#1432

Merged

ironoa mentioned this pull request Apr 24, 2024

stakerRewards bug | Payouts polkadot-js/api#5859

Closed

Imod7 mentioned this pull request Apr 24, 2024

/accounts/{accountId}/staking-info The return of this interface is changed (The key claimedRewards that returns the result becomes legacyClaimedRewards), why is this? paritytech/substrate-api-sidecar#1433

Closed

Ank4n mentioned this pull request Apr 25, 2024

[Staking] Runtime api if era rewards are pending to be claimed #4301

Merged

ukint-vs mentioned this pull request May 1, 2024

feat: Update substrate v1.4.0 gear-tech/gear#3934

Merged

This was referenced Jun 5, 2024

Update polkadot-sdk from v1.7.0 to v1.11.0 moondance-labs/tanssi#573

Closed

Update polkadot-sdk from v1.10.0 to v1.11.0 moondance-labs/tanssi#577

Closed

Aideepakchaudhary mentioned this pull request Oct 11, 2024

Feat: Polkadot v1.2. to v1.4 upgrade Cerebellum-Network/blockchain-node#437

Closed

Conversation

Ank4n commented Aug 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

How payouts would work like after this change

payout_stakers

payout_stakers_by_page

Commission note

Migration Note

Storage Changes

Added

Deprecated

Config Changes

TODO

Followup issues

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ank4n Sep 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Ank4n commented Aug 26, 2023 •

edited

Loading

Ank4n Sep 6, 2023 •

edited

Loading