Skip to content

Commit 3ba9c2c

Browse files
authored
Add benchmarks for wasmi builtin metering (#56)
* Upgraded to newest wasmi * Refactored benchmarks * Two new benchmark strategies (`no_metering` and `wasmi_builtin`) We can now benchmark the execution of modules using our two instrumentation strategies in addition to no metering (as a baseline) and wasmi's builtin metering. We can learn from the following results (ran on my M1) that the builtin metering decisively outperforms the instrumentation on every single fixture. cc @Robbepop ``` coremark/no_metering [15.586 s 15.588 s 15.589 s] coremark/wasmi_builtin [16.403 s 16.414 s 16.434 s] coremark/host_function [18.245 s 18.248 s 18.252 s] coremark/mutable_global [20.476 s 20.486 s 20.505 s] recursive_ok/no_metering [111.32 µs 111.33 µs 111.34 µs] recursive_ok/wasmi_builtin [138.64 µs 138.65 µs 138.66 µs] recursive_ok/host_function [495.55 µs 495.64 µs 495.78 µs] recursive_ok/mutable_global [514.07 µs 514.09 µs 514.11 µs] fibonacci_recursive/no_metering [3.9098 µs 3.9102 µs 3.9108 µs] fibonacci_recursive/wasmi_builtin [4.3242 µs 4.3246 µs 4.3250 µs] fibonacci_recursive/host_function [12.913 µs 12.914 µs 12.915 µs] fibonacci_recursive/mutable_global [13.202 µs 13.208 µs 13.212 µs] factorial_recursive/no_metering [530.72 ns 530.84 ns 530.91 ns] factorial_recursive/wasmi_builtin [619.17 ns 619.30 ns 619.44 ns] factorial_recursive/host_function [1.7656 µs 1.7657 µs 1.7659 µs] factorial_recursive/mutable_global [1.8783 µs 1.8786 µs 1.8788 µs] count_until/no_metering [1.2422 ms 1.2423 ms 1.2424 ms] count_until/wasmi_builtin [1.3976 ms 1.3978 ms 1.3981 ms] count_until/host_function [4.8074 ms 4.8106 ms 4.8125 ms] count_until/mutable_global [5.9161 ms 5.9169 ms 5.9182 ms] memory_vec_add/no_metering [4.1630 ms 4.1638 ms 4.1648 ms] memory_vec_add/wasmi_builtin [4.3913 ms 4.3925 ms 4.3930 ms] memory_vec_add/host_function [8.2925 ms 8.2949 ms 8.2967 ms] memory_vec_add/mutable_global [9.1124 ms 9.1152 ms 9.1163 ms] wasm_kernel::tiny_keccak/no_metering [613.21 µs 613.42 µs 613.58 µs] wasm_kernel::tiny_keccak/wasmi_builtin [617.04 µs 617.46 µs 617.81 µs] wasm_kernel::tiny_keccak/host_function [817.24 µs 817.44 µs 817.89 µs] wasm_kernel::tiny_keccak/mutable_global [873.42 µs 873.90 µs 874.65 µs] global_bump/no_metering [1.4597 ms 1.4598 ms 1.4600 ms] global_bump/wasmi_builtin [1.6151 ms 1.6152 ms 1.6153 ms] global_bump/host_function [5.5393 ms 5.5418 ms 5.5435 ms] global_bump/mutable_global [6.9446 ms 6.9454 ms 6.9461 ms] ```
1 parent 16297ab commit 3ba9c2c

4 files changed

Lines changed: 503 additions & 577 deletions

File tree

Cargo.toml

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,14 @@ repository = "https://github.com/paritytech/wasm-instrument"
1212
include = ["src/**/*", "LICENSE-*", "README.md"]
1313

1414
[[bench]]
15-
name = "benches"
15+
name = "instrumentation"
1616
harness = false
17-
path = "benches/benches.rs"
17+
path = "benches/instrumentation.rs"
18+
19+
[[bench]]
20+
name = "execution"
21+
harness = false
22+
path = "benches/execution.rs"
1823

1924
[profile.bench]
2025
lto = "fat"
@@ -32,7 +37,7 @@ rand = "0.8"
3237
wat = "1"
3338
wasmparser = "0.101"
3439
wasmprinter = "0.2"
35-
wasmi = "0.22"
40+
wasmi = "0.28"
3641

3742
[features]
3843
default = ["std"]

0 commit comments

Comments
 (0)