Move skip cert to per-slot pool and fix epoch stakes in cert pool

[wen-coding]

• Move skip cert into per-slot pool, use Arc so we don't do too many copies
• Fix stake calculation by caching epoch_stakes_map and epoch_schedule
• Moved the original tests in skip_pool to certificate_pool and changed API accordingly
• This should now accept multiple skips per pubkey

[AshwinSekar]

We should be able to accept any ordering of votes.
e.g. catching up from snapshot but you receive an all-to-all skip vote from tip

[wen-coding]

We are able to accept votes in any order, but it's good to have definitive order of which one wins if votes can arrive in arbitrary order.
If someone voted (3, 5), then later (4, 7), previously we reject (4, 7), now we say for slot 3 we keep (3, 5) vote in the skip certificate, but for slot 4 and 5 we have two choices:
A. keep (3, 5) which arrives first
B. replace (3, 5) with (4, 7)
The problem of A is that we decide which one to keep by arriving order, so someone could keep sending us (3, 5) and (4, 7) and the transaction in slot 4 and 5 will keep changing. Not end of the world, it's still correct, but we have lots of unnecessary replacements.
So this code uses B, we decide an order, (4, 7) is always better than (3, 5) because 4 is greater than 3 and 7 is greater than 5.
But you are right in that it's a bit arbitrary why (4, 6) is worse than (3, 7). All we need is a deterministic order to reduce needless replacements, the code is still correct in that no one can un-vote skip.
So how about this order:
Because normal validators almost always extend the end, let's compare end first then compare start and leave the one with hopefully wider range so we can hopefully have fewer transactions in the cert:

1. new_end > old_end: (4, 8) > (3, 5), (2, 8) > (3, 5), (3, 8) > (3, 5)
2. new_end == old_end, check if max(new_start, local_root) < max(old_start, local_root): if local_root is 6, reject (2, 8) when (3, 8) is available. But (7, 9) > (8, 9) when local_root is 6.
3. new_end < old_end: reject

[wen-coding]

Actually, thought about it more, changed to:
(new_end > old_end) || (new_end == old_end && new_start > old_start)

We can argue both ways and replace it with:
(new_end > old_end) || (new_end == old_end && new_start < old_start)

The algorithm just needs a deterministic order to be efficient.

[carllin]

might be able to simplify this to:

fn set_highest_slot(&mut self, certificate_type: CertificateType, slot: Slot) -> bool {
    let current = self.highest_slot_map.entry(certificate_type)
        .and_modify(|s| {
            if slot > *s {
                *s = slot;
            }
        })
        .or_insert(slot);

    *current == slot
}

[wen-coding]

Actually I want return value to be true only if the entry didn't exist or the original value is smaller to the new value, changed.

[carllin]

Isn't it possible we would need multiple disjoint skip votes for a specific validator? For instance a validator could vote (1, 4), and then (6, 9), and we might need both votes to make the skip certificate for (1, 9)

[wen-coding]

Yes it's possible, which is why the key to tx_set is (pubkey, skip_range). So we would have
(key1, (1, 4)), tx1
(key1, (6, 9)), tx2
So both tx1 and tx2 would be added to the certificate.

[carllin]

Overlapping skip votes should be slashable and we should reject them

[wen-coding]

Now that we use per-slot skip pool, And we allow multiple overlapping skip votes per pubkey, the skip range in skip votes are basically abbreviation. What are we trying to achieve by checking for overlapping skip votes? What damage can it do? Do we care?
If we allow overlapping skip votes, then users can safely discard any slot older than local root, even though those slots might have been skipped by the current user. Otherwise users have to remember what he sent for last skip vote, otherwise his new votes might all get rejected. This is not impossible, but it's a bit annoying.

[AshwinSekar]

lgtm #157 will handle moving skip to a single slot instead of a range