Skip to content

PVF: drop backing jobs if it is too late#5616

Merged
AndreiEres merged 123 commits intomasterfrom
AndreiEres/issue5530
Nov 7, 2024
Merged

PVF: drop backing jobs if it is too late#5616
AndreiEres merged 123 commits intomasterfrom
AndreiEres/issue5530

Conversation

@AndreiEres
Copy link
Copy Markdown
Contributor

@AndreiEres AndreiEres commented Sep 6, 2024

Fixes #5530

This PR introduces the removal of backing jobs that have been back pressured for longer than allowedAncestryLen, as these candidates are no longer viable.

It is reasonable to expect a result for a backing job execution within allowedAncestryLen blocks. Therefore, we set the job TTL as a relay block number and synchronize the validation host by sending activated leaves.

Copy link
Copy Markdown
Contributor

@sandreim sandreim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! 🚀

Comment on lines +331 to +335
if in_active_fork {
None
} else {
Some(index)
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could directly filter on the queue, so if in active fork we send ValidationError::ExecutionDeadline and return true otherwise false.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't send the result inside the closure, that why I didn't use retain

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not? the send function is not async and we ignore the result

Copy link
Copy Markdown
Contributor

@alindima alindima left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Just a couple of suggestions

@AndreiEres AndreiEres added this pull request to the merge queue Nov 7, 2024
Merged via the queue into master with commit 6c8a347 Nov 7, 2024
@AndreiEres AndreiEres deleted the AndreiEres/issue5530 branch November 7, 2024 13:46
paritytech-ci pushed a commit that referenced this pull request Nov 7, 2024
Fixes #5530

This PR introduces the removal of backing jobs that have been back
pressured for longer than `allowedAncestryLen`, as these candidates are
no longer viable.

It is reasonable to expect a result for a backing job execution within
`allowedAncestryLen` blocks. Therefore, we set the job TTL as a relay
block number and synchronize the validation host by sending activated
leaves.

---------

Co-authored-by: Andrei Sandu <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>
Comment on lines +42 to +46
/// The execution deadline of allowed_ancestry_len + 1 has been reached. Jobs like backing have
/// a limited time to execute. Once the deadline is reached, the current candidate cannot be
/// backed, regardless of its validity.
#[error("candidate validation: execution deadline has been reached.")]
ExecutionDeadline,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice PR! Would this have worked as an InternalValidationError? If there is back pressure, it's technically not a problem with the PVF itself, so you could have nodes abstain from voting instead of voting against. The only reason I'd be hesitant with that, is that if one node is experiencing back pressure, it's possible that the rest of the network is as well. If too many nodes abstain then IIRC it could cause a finality stall.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OMG, that's one epic comeback!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrcnski have we met before?
We have backing back pressure that should free workers to do approval job, so the effect is supposed to be opposite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

I5-enhancement An additional feature request. T0-node This PR/Issue is related to the topic “node”. T8-polkadot This PR/Issue is related to/affects the Polkadot network.

Projects

Development

Successfully merging this pull request may close these issues.

PVF: drop backing jobs if it is too late