[Manta] Auto-evict low-performing collators each session

[Garandor]

Description

This PR modifies the manta-collator-selection pallet by adding three new storage items and corresponding setter extrinsics, that - if set to anything other than their default - automatically remove collators at the end of a session that significantly ( x% ) underperformed the y-th percentile of the total collator set over the current session.

A good initial value (and we intend to set this on Dolphin) is

• x = 10 ( default value on genesis: 100)
• y = 80 ( default value on genesis: 0)

Main changes in: pallets/collator-selection/src/lib.rs

• Substitutes the kick_threshold and last_block_authored storage items with two new storage items, defining the percentile and the percentage above
• Adds a setter extrinsic for each of the new items
• Adds a storage map that records the number of blocks produced by each collator in the current session

closes: #335

---

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

• Targeted PR against correct branch (manta or dolphin) with right title (start with [Manta] or [Dolphin]),
• Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• Wrote unit tests.
• Updated relevant documentation in the code.
• Re-reviewed Files changed in the Github PR explorer.
• If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. If this number is updated, then the spec_version must also be updated
• [ ] If needed, notify the committer this is a draft-release and a tag is needed after merging the PR.
• Verify benchmarks & weights have been updated for any modified runtime logics
• [ ] If needed, bump version for every crate.
• If import a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
• [ ] If needed, update our Javascript/Typescript APIs. These APIs are offcially used by exchanges or community developers.
• [ ] If we're going to issue a new release, freeze the code one week early(it depends, but usually it's one week), ensure we have enough time for related testing.

[Garandor]

This PR is functional but not fully ready - i have added comments where I still have open questions myself - please have a look and advise how to best resolve them

[Garandor]

Only one unresolved open question left and that one is strictly optional to implement.

This is now ready for final review

P.S. I'll see on the side if i can get rid of the pulled in ValidatorSet trait as an optional refactor

[ghzlatarev]

@Garandor The CI is failing.

[Garandor]

Reworking this to add extrinsics to change the percentile and threshold.
Ended the day yesterday with benchmarking problems

[Garandor]

Kicking config is now updateable and disabled by default on genesis.

The verify step of the benchmark is still failing (https://github.com/Manta-Network/Manta/runs/5211310990?check_suite_focus=true#step:9:1687), looking into it, but the code should now be fairly stable.

@ghzlatarev @bhgomes

[Garandor]

Do Not Merge until resolution of

https://substrate.stackexchange.com/questions/504/reclaim-used-space-of-a-removed-storageitem-on-a-live-chain

Resolution came quick - A migration to kill the old storage item is needed

[Garandor]

@stechu Worked in your comments.
As for adding #transactional, I think the two set_* extrinsics really don't have the complexity to warrant adding it.
Parity also hasnt marked set_candidacy_bond transactional: https://paritytech.github.io/cumulus/src/pallet_collator_selection/lib.rs.html#335

This stil is not ready to merge as it needs a storage migration to clear the removed storage item

[Garandor]

Tentatively final version of this PR:

• Calamari is set in Cargo.toml to use the last pre-merge version of manta_collator_selection on manta branch.
  No migration is needed as it continues to use the old KickThreshold-based eviction mechamism.

NOTE: This will go well for a while, until we upgrade substrate dependencies to v0.9.17, as this hard-set rev of manta_collator_selection can't easily have its deps updated along with the rest of the code.
When we do decide to upgrade calamari to the new mechanism, a migration is needed to remove the following storage items from the chain.

1. LastAuthoredBlock
2. KickThreshold

• Dolphin is set to use the pallet from local filesystem
  No migration is needed as the chain/testnet has not yet been launched

[Garandor]

updated weights from CI benchmark missing, don't merge yet

[Garandor]

Do not merge until resoution of #437

After #437 is on manta:

• Retarget collator selection to the merge commit of Bump upstream to v0.9.17 #437 in https://github.com/Manta-Network/Manta/pull/358/files#diff-c27ffa7e0d850a31b480f6aefffae5ad060b4022c5c9bc2cc1e2550bb7be2082R85 (not upgrading as waiting for v0.9.18)
• Retest on polkadot-launch
• Merge

[Garandor]

Finished refactoring the test code @stechu
It should be easier to follow along now.

LGTM