Skip to content

Make the sender parallel#70

Merged
bredamatt merged 13 commits intomainfrom
bredamatt/sender-parallelism
Jun 20, 2023
Merged

Make the sender parallel#70
bredamatt merged 13 commits intomainfrom
bredamatt/sender-parallelism

Conversation

@bredamatt
Copy link
Copy Markdown
Contributor

@bredamatt bredamatt commented Jun 20, 2023

Closes #71

This PR makes the sender support parallel execution. Normal usage is still supported, however.

Parallelism is supported by leveraging the same logic, but by replacing the sender_index with a thread index, given that the number of threads passed as an argument to the sender is > 1.

Because there was a mix of I/O bound and CPU bound tasks related to sending transactions using subxt, I split the CPU bound task into a spawn_blocking section, and used a JoinSet for the pre-checks which are I/O bound.
All the transactions are submitted after all them are signed.

There also had to be changes made to the subxt client (Api type here), which was done using the OnlineClient<PolkadotConfig>::from_rpc() method, taking an Arc<RpcClient> as an argument. This is because the default OnlineClient does not support the number of concurrent requests required when the sender in parallel submits all the transactions. Therefore, the client has to be built with the WsClientBuilder from jsonrpsee to set the max_concurrent_requests parameter in stead. Currently, the max_concurrent_requests is set to u32::MAX.

@bredamatt bredamatt self-assigned this Jun 20, 2023
@bredamatt bredamatt added the enhancement New feature or request label Jun 20, 2023
@bredamatt bredamatt requested a review from pepoviola June 20, 2023 13:16
@bredamatt bredamatt requested a review from pepoviola June 20, 2023 15:05
@bredamatt
Copy link
Copy Markdown
Contributor Author

Thanks for the comments @pepoviola ! All of them should be addressed by now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

Make the sender parallel

2 participants