fix: add new staking storage items

[Imod7]

Rel: #1411

Related Endpoint /accounts/{accountId}/staking-payouts

The changes in this PR are affecting the responses returned by /accounts/{accountId}/staking-payouts endpoint.

Description

After the runtime upgrade of Polkadot to v1.2.0, the new storage items erasStakersPaged and erasStakersOverview are used instead of erasStakersClipped.
More information regarding this change can be found:

• [NPoS] Paging reward payouts in order to scale rewardable nominators  polkadot-sdk#1189
• Support Paged Reward Payouts polkadot-js/apps#10004

Hence, as mentioned here, to get the rewards:

• in Kusama after era 6514 (Apr 18) and
• in Polkadot after era 1420 (Apr 21)

we need to make use of the new storage items instead of relying on erasStakersClipped, as it currently returns empty results for eras > than the eras mentioned previously. We still use erasStakersClipped for eras before the runtime upgrade until it is removed completely which we would need to also remove.

Implementation Notes

• Since now the information is split into 2 storage items (link), validatorsOverview was added in IAdjustedDeriveEraExposure so we can store and use the information from the additional call.

• Sidecar's response item totalValidatorExposure is retrieved from query.staking.erasStakersOverview > total

  • which is aligned with the old logic but using the new call.

  IMPORTANT NOTE
  There is also available query.staking.erasStakersPaged > pageTotal but based on manual testing and quoting this part of the forum post :

  ErasStakers Changes: total and value in Overview, others in Paged

  I believe the more complete information is found in ...Overview - total since there are cases when these 2 fields differ.

• Sidecar's response item nominatorExposure when the validator is also the nominator, is retrieved from query.staking.erasStakersOverview > own

  • which is aligned with the old logic but using the new call.

Sample Responses

Sidecar endpoint while connected to Polkadot chain :

http://127.0.0.1:8080/accounts/15CosmEmAfQAhnxwan18e5TueAe6bDzrqqxg13dToDWr7A8M/staking-payouts?depth=1&era=1420&unclaimedOnly=false

returns

{
  "at": {
    "height": "20480919",
    "hash": "0xc163f6e3182542699f89e78c95298579fd86168cc73370814f6dce6628747f6b"
  },
  "erasPayouts": [
    {
      "era": "1420",
      "totalEraRewardPoints": "27466000",
      "totalEraPayout": "3579344121756224",
      "payouts": [
        {
          "validatorId": "15CosmEmAfQAhnxwan18e5TueAe6bDzrqqxg13dToDWr7A8M",
          "nominatorStakingPayout": "633147762128",
          "claimed": false,
          "totalValidatorRewardPoints": "93760",
          "validatorCommission": "50000000",
          "totalValidatorExposure": "26454213023677507",
          "nominatorExposure": "50621054502286"
        }
      ]
    }
  ]
}

Cross checked results

• totalValidatorRewardPoints with subscan
• totalValidatorExposure with pjs-apps (staking.erasStakersOverview - total for the specified account and erra)
• nominatorExposure with pjs-apps (staking.erasStakersOverview - own for the specified account and erra)
• The current version of public sidecar does not return any results.

[marshacb]

Since the same logic around nominatorId is repeated here in both cases perhaps we can do something like

				const individualExposure = exposure.others ? exposure.others : (exposure as unknown as Option<SpStakingExposurePage>).unwrap().others;
				individualExposure.forEach(({ who }, validatorIndex): void => {
					const nominatorId = who.toString();

					nominators[nominatorId] = nominators[nominatorId] || [];
					nominators[nominatorId].push({ validatorId, validatorIndex });
				});

[Imod7]

Awesome! Changed it! Thank you for the suggestion! 💯

[marshacb]

Accounting for Tarik's suggestion below this could look something like

const individualExposure = exposure.others ? exposure.others : 
(exposure as unknown as Option<SpStakingExposurePage>).isSome ?
(exposure as unknown as Option<SpStakingExposurePage>).unwrap().others :
undefined;

if (individualExposure) {
	individualExposure.forEach(({ who }, validatorIndex): void => {
		const nominatorId = who.toString();

		nominators[nominatorId] = nominators[nominatorId] || [];
		nominators[nominatorId].push({ validatorId, validatorIndex });
	});
}

[Imod7]

I already changed this.

I did it like this

const individualExposure = exposure.others
  ? exposure.others
  : (exposure as unknown as Option<SpStakingExposurePage>).isSome
	  ? (exposure as unknown as Option<SpStakingExposurePage>).unwrap().others
	  : [];
  individualExposure.forEach(({ who }, validatorIndex): void => {
  const nominatorId = who.toString();
  
  nominators[nominatorId] = nominators[nominatorId] || [];
  nominators[nominatorId].push({ validatorId, validatorIndex });
  });

it looks the same as yours but instead of undefined i set an empty array. Then forEach in an empty array will just not do anything.
Is it recommended to set undefined and then check if (individualExposure) or I can leave it like that ?

[marshacb]

nope, this is totally cool! great job

[marshacb]

PR lgtm! Just had one minor nit/suggestion to consider

[TarikGul]

Might be safe to also check isSome on the Option before unwrapping it.

[Imod7]

Added checks with isSome in the code part mentioned here, as well as in 3 other parts of the code (in the extractExposure function) where I am also unwrapping. The cases though in the extractExposure is not adding much value since unwrap is also called later on these variables (here and here) but I prefer not to refactor that part right now if that is ok.