Skip to content

Conversation

@lean-apple
Copy link

@lean-apple lean-apple commented Nov 1, 2024

Issue Addressed

Evaluate and benchmark the performance of newly implemented axum endpoints, specifically comparing them to the existing Warp implementation

Proposed Changes

  • Add performance tests for a simple endpoint /eth/v1/beacon/genesis
  • Measure response times to identify any discrepancies or optimizations needed, especially in cases where Axum endpoints with and without task spawners show performance differences

Additional Info

The following results were obtained by running:

cargo test --release axum_warp_tests  -- --nocapture  

gives us the following :

Performance Comparison (100 requests per implementation):
----------------------------------------
Framework & Configuration    | Total Time | Avg Time/Request | vs. Warp
----------------------------------------
Warp                        |  68.417µs |          684ns |  baseline
Axum (no task spawner)      | 103.292µs |        1.032µs |  1.51x slower
Axum (with task spawner)    |  823.75µs |        8.237µs |  12.04x slower

Next Steps

To further investigate, additional endpoints should be tested to determine if performance differences with the task spawner can be reduced.

@lean-apple
Copy link
Author

@pawanjay176 @AgeManning could you have a look ? :)

@AgeManning
Copy link

Oh yeah, seems strange Axum is that little bit slower.

I do expect the task spawner to take more time for individual small endpoints because it does more things.

However, we do want to keep the task spawner because it spawns the processing into our beacon_processor. This thing manages all the work of the node and prioritizes what work should be done.

For example, we don't want someone to be hitting the API to retrieve lots of blocks and then have that consume all the CPU threads such that we can no longer follow the chain.

By using the spawner, we can still prioritize important chain duties over HTTP API requests.

Having it be a little bit slower per endpoint is worth this trade-off I think.

@lean-apple
Copy link
Author

lean-apple commented Nov 4, 2024

Oh yeah, seems strange Axum is that little bit slower.

I do expect the task spawner to take more time for individual small endpoints because it does more things.

However, we do want to keep the task spawner because it spawns the processing into our beacon_processor. This thing manages all the work of the node and prioritizes what work should be done.

For example, we don't want someone to be hitting the API to retrieve lots of blocks and then have that consume all the CPU threads such that we can no longer follow the chain.

By using the spawner, we can still prioritize important chain duties over HTTP API requests.

Having it be a little bit slower per endpoint is worth this trade-off I think.

Yes, I was a bit surprised to see axum performing slower.

The reason for testing it without the task spawner was purely out of curiosity :
I wanted to see if the endpoint would still be faster or slower without it in comparison with warp.

I'd like to investigate further to determine if there's something in the code causing this slowdown or if it is only linked to the framework.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants