Skip to content

PoT parameters change and sync#1929

Merged
nazar-pc merged 19 commits intomainfrom
pot-parameters-change-and-sync
Sep 11, 2023
Merged

PoT parameters change and sync#1929
nazar-pc merged 19 commits intomainfrom
pot-parameters-change-and-sync

Conversation

@nazar-pc
Copy link
Copy Markdown
Member

@nazar-pc nazar-pc commented Sep 5, 2023

This builds on top of #1922 and extends implementation with two basic capabilities:

  • support for PoT parameters change
  • PoT sync between nodes on the network

Parameters change refers to both entropy injection (where seed is derived from both parent proof and blockchain entropy, which is a special case change from default behavior) and change for number of iterations. Both should coincide, so when we will want to update number of iterations, the actual change will be deferred until next entropy injection. Combining them together simplifies implementation and we should not need to update number of iterations very frequently anyway.

PoT sync is achieved in the following way:

When new proof arrives from one of 3 sources (timekeeper, gossip, block import), verifier will be queried for proofs at higher slot until no such proofs can be found. This ensures that we get to the tip when proofs arrive out of order or when we catch-up block sync and have a gap between best proof in the latest block and the oldest block we have received via gossip.

Proof of time state was introduced, which is a small piece of tricky logic manages information about inputs for the upcoming slot proof. Timekeeper will always use this information as the source of truth and will build proofs on top. This in turn means that syncing proofs to the tip updates the state and re-syncs Timekeeper with the rest of the network.

Code finally starts to take the eventual shape.

There is still TODO for low-latency timekeeper sync (when we receive proof that timekeeper is still working on we should start another timekeeper even before proof is verified to make sure we close the gap in case proof ends up being valid).

Also checkpoints are not sent alongside blocks, which in some cases will decrease verification performance.

Also gossip needs to be tuned and punishment for bad proofs (or proofs that don't contribute to canonical PoT chain) must be implemented.

But overall it is in decent shape.


The next major step is to implement entropy injection and then re-sync implementation and what I have in my head with the spec and have a discussion with Dariia.

Code contributor checklist:

dariolina
dariolina previously approved these changes Sep 5, 2023
Copy link
Copy Markdown
Contributor

@dariolina dariolina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Postponing iteration change to the closest entropy injection point is ok. Injection is quite frequent and iteration adjustment will be quite rare, even if/when we make it automatic.
Are checkpoints not included until it's possible to include them in justifications or are you planning to not include them ever?

@nazar-pc
Copy link
Copy Markdown
Member Author

nazar-pc commented Sep 5, 2023

Are checkpoints not included until it's possible to include them in justifications

This, once we update Substrate to a fork that includes paritytech/polkadot-sdk#1211 I'll add them and add corresponding logic. Logic will be a bit different from the current implmentation because we'll be able to check more in verification rather than block import, but both code paths would still have to exist, so current implementation is still useful. Optimistic verification for sync from genesis will be done at some point in the future.

Base automatically changed from basic-pot-gossip to main September 6, 2023 07:39
@nazar-pc nazar-pc dismissed dariolina’s stale review September 6, 2023 07:39

The base branch was changed.

@nazar-pc nazar-pc marked this pull request as draft September 6, 2023 07:39
Copy link
Copy Markdown
Contributor

@shamil-gadelshin shamil-gadelshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Feature-gated" code is hard to follow and it makes sense to join such blocks where possible.

}

#[cfg(feature = "pot")]
let slot_iterations;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put these three variables and two following if-statements under the same cfg-block?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did that in some cases, here doing let (slot_iterations, pot_seed, next_slot) = {} would be hardly more readable.

slot = slot + Slot::from(1);
}

#[cfg(feature = "pot")]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be probably joined with the previous cfg-block.

Copy link
Copy Markdown
Member Author

@nazar-pc nazar-pc Sep 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then another level of indentation would have to be added and I wanted to avoid that.

Copy link
Copy Markdown
Member Author

@nazar-pc nazar-pc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also fixed one verification bug that I found while testing entropy injection locally

}

#[cfg(feature = "pot")]
let slot_iterations;
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did that in some cases, here doing let (slot_iterations, pot_seed, next_slot) = {} would be hardly more readable.

slot = slot + Slot::from(1);
}

#[cfg(feature = "pot")]
Copy link
Copy Markdown
Member Author

@nazar-pc nazar-pc Sep 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then another level of indentation would have to be added and I wanted to avoid that.

@nazar-pc nazar-pc marked this pull request as ready for review September 6, 2023 09:59
dariolina
dariolina previously approved these changes Sep 6, 2023
rahulksnv
rahulksnv previously approved these changes Sep 6, 2023
@nazar-pc
Copy link
Copy Markdown
Member Author

nazar-pc commented Sep 8, 2023

I found some issues with implementation and fixed them in last commits. Entropy injection PR will be separate, but this should be all that is necessary to support it. I was trying to find a way to not verify PoT in slot worker, but eventually wasn't able to.

@nazar-pc nazar-pc merged commit 19dc65f into main Sep 11, 2023
@nazar-pc nazar-pc deleted the pot-parameters-change-and-sync branch September 11, 2023 16:00
@nazar-pc nazar-pc mentioned this pull request Sep 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants