Parallel vps' gas accounting rework #1835
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe your changes
Currently, during the parallel execution of vps (more specifically in the
try_foldfunction we use thesetmethod ofVpsGasto set the gas cost of the current Vp. This function only updates themaxfield as it assumes theVpsGasinstance to have been just initialized and, therefore, be empty. From the rayon documentation though (https://docs.rs/rayon/latest/rayon/iter/trait.ParallelIterator.html#method.fold), it is stated that the sequence might be subdivided before being folded. In this case, more than on VP would be aggregated in one of these chunks and all of the VP following the first one would see a non-defaultVpsGasinstance, i.e. an instance with themaxfield containing the gas cost of the previous vp. Given that the current implementation ofsetonly overwrites themaxfield, we would end up under assessing the gas cost of the transaction. The correct logic, in this case, would be to update the maximum (if needed) and push the cheaper vp cost to therestfield.I have tested this thing and it actually looks like rayon never produces subsequences with more than one element, i.e. all of the vps are allocated to their own subsequence and therefore do not incur in this problem. From some tests I saw that to trigger the misbehavior the collection needs to contain at least 8 elements, but this number is variable (depends from run to run) and also depends on the number of logical cpus allocated to rayon (which for Namada is non-constant, since we use half of the available logical cores). IMPORTANT: We might never come to such number of vps for a single transaction.
In the
setmethod we also have a couple ofdebug_assertsverifying that theVpsGasinstance is indeed empty (and therefore the current logic is correct).My concern is that if the internals of rayon change (which might even go unnoticed by SemVer), or if we start triggering more vps per transaction than we currently do, we might start to experience this issue on some machines. If we don't catch this with the
debug_assertsand the code gets shipped we might end up with a non-consistent gas accounting which would eventually lead to a break in consensus.So this PR carries a small rework that ensures that the
identityargument oftry_foldis indeed an identity function and would guarantee the correct execution logic regardless of the rayon internals/execution context.Indicate on which release or other PRs this topic is based on
v0.22.0
Checklist before merging to
draft