feat(l1): add import-bench command for benchmarking block importing #5215
Conversation
Lines of code reportTotal lines added: Detailed view |
|
We should print the actual execution time based without taking into account the background thread. Then we should print how much we're waiting. Print as "stall on background task" |
This reverts commit e4f895e.
tooling/import_benchmark/README.md
Outdated
| This tool is used to benchmark the performance of **ethrex**. | ||
| We aim to execute the same set of blocks on the same hardware to ensure consistent | ||
| performance comparisons. Doing this on a running node is difficult because of variations | ||
| in hardware, peer count, and system load. |
There was a problem hiding this comment.
Mention block content, which is the biggest motivation for this tool.
tooling/import_benchmark/README.md
Outdated
| # Parameters: | ||
| # - BENCH_ID: Identifier for the log file, saved as bench-BENCH_ID.log | ||
| # - NETWORK: Network to access (e.g., hoodi, mainnet) |
There was a problem hiding this comment.
Maybe bring the parameter list outside the code block
| ) | ||
| .await?; | ||
| } | ||
| Subcommand::ImportBench { path, removedb, l2 } => { |
There was a problem hiding this comment.
I think we should try to merge this with the import subcommand in the future.
| 3. Restart the node and let it advance by *X* additional blocks. | ||
| 4. Stop the node again and run: | ||
| ```bash | ||
| ethrex export --first <block_num> --last <block_num + X> ~/.local/share/ethrex_NETWORK_bench/chain.rlp |
There was a problem hiding this comment.
Unrelated to this PR, but I got an error "Too many open files" when exporting the chain. We probably need to add the "ulimit fix" to the export command.
Edit: nvm, I also got the error when running make run-bench. We should add that fix when opening the DB. This can be changed in another PR.
| ## View Output | ||
|
|
||
| You can view and compare benchmark results with: | ||
| `python3 parse_bench.py <bench_num_1> <bench_num_2>` |
There was a problem hiding this comment.
Running this, I got a ModuleNotFoundError.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
There was a problem hiding this comment.
| `python3 parse_bench.py <bench_num_1> <bench_num_2>` | |
| `python3 parse_bench.py <bench_id_start> <bench_id_end>` |
We should also add an example, like we have in the PR description:
For example:
```text
# here we compare bench runs from 10 to 13 (non-inclusive)
$ python3 parse_bench.py 10 13
Blocks tested 1065
Mean ggas accross multiple runs: 0.14995899843505422
Mean ggas in run: 10 0.15196807511737107
Mean ggas in run: 11 0.15213427230046964
Mean ggas in run: 12 0.14577464788732383
Mean ggas spread across blocks: 0.004736776212832552
```
Motivation
This pr aims to add a consistent command we can use to reproducible benchmark the speed of ethrex, without p2p and other concurrency problems enabled.
Description
Example