Bounded interpreter cache support · Pull Request #316 · paritytech/polkavm

ghost · 2025-07-16T14:10:01Z

Add support for bounded interpreter cache and automatic cache reset when quota is exceeded.

Instead of adding a check in the instruction execution loop, we cleverly replace the first instruction of the new basic block with reset cache micro-instruction. This keeps the runner loop optimized. In addition, we make sure capacity of the interpreter cache never gets too big.

As you may notice, max_cache_size is not a tight upper bound, and transiently we could allocate more than max_cache_size. This is needed to keep the implementation optimized and complexity at bay.

Finally, we are also adding support to reset FlatMap and allocate its memory on first use.

Testing

running 1 test
[DEBUG polkavm::api] Selected backend: 'interpreter'
[TRACE polkavm::api] Creating new module from a 32-bit program blob
[TRACE polkavm::api] Checking imports...
[TRACE polkavm::api] Checking jump table...
[TRACE polkavm::api] Parsing exports...
[TRACE polkavm::api]   Export at 0: 'main'
[TRACE polkavm::api] Processing finished!
[DEBUG polkavm::api] Backend used: 'interpreted'
[DEBUG polkavm::api]   Memory map: RO data: 0x00010000..0x00010000 (0/0 bytes, non-zero until 0x00010000)
[DEBUG polkavm::api]   Memory map: RW data: 0x00020000..0x00020000 (0/0 bytes, non-zero until 0x00020000)
[DEBUG polkavm::api]   Memory map:   Stack: 0xfffe0000..0xfffe0000 (0/0 bytes)
[DEBUG polkavm::api]   Memory map:     Aux: 0xffff0000..0xffff0000 (0/0 bytes requested)
[TRACE polkavm::interpreter] Resolving arbitrary jump: 0
[TRACE polkavm::interpreter]   -> Found start of a basic block at: 0
[DEBUG polkavm::interpreter] Compiling block:
[DEBUG polkavm::interpreter]   [2]: 0: a0 = 0x1
[DEBUG polkavm::interpreter]   [3]: 3: a1 = 0x2
[DEBUG polkavm::interpreter]   [4]: 6: a2 = 0x3
[DEBUG polkavm::interpreter]   [5]: 9: jump 11
[DEBUG polkavm::interpreter] Starting execution at: 0 [2]
[TRACE polkavm::interpreter::raw_handlers] [2]: a0 = 0x1
[TRACE polkavm::interpreter]   set: a0 = 0x1
[TRACE polkavm::interpreter::raw_handlers] [3]: a1 = 0x2
[TRACE polkavm::interpreter]   set: a1 = 0x2
[TRACE polkavm::interpreter::raw_handlers] [4]: a2 = 0x3
[TRACE polkavm::interpreter]   set: a2 = 0x3
[TRACE polkavm::interpreter::raw_handlers] [5]: unresolved jump 11
[DEBUG polkavm::interpreter] Compiling block:
[DEBUG polkavm::interpreter]   [6]: 11: a3 = 0x4
[DEBUG polkavm::interpreter]   [7]: 14: a4 = 0x5
[DEBUG polkavm::interpreter]   [8]: 17: jump 19
[DEBUG polkavm::interpreter] Compiled handlers cache size exceeded at 6: 9 > 6; will reset the cache
[TRACE polkavm::interpreter::raw_handlers]   -> resolved to fallthrough
[TRACE polkavm::interpreter::raw_handlers] [6]: reset_cache
[DEBUG polkavm::interpreter] Compiling block:
[DEBUG polkavm::interpreter]   [2]: 11: a3 = 0x4
[DEBUG polkavm::interpreter]   [3]: 14: a4 = 0x5
[DEBUG polkavm::interpreter]   [4]: 17: jump 19
[TRACE polkavm::interpreter::raw_handlers] [2]: a3 = 0x4
[TRACE polkavm::interpreter]   set: a3 = 0x4
[TRACE polkavm::interpreter::raw_handlers] [3]: a4 = 0x5
[TRACE polkavm::interpreter]   set: a4 = 0x5
[TRACE polkavm::interpreter::raw_handlers] [4]: unresolved jump 19
[DEBUG polkavm::interpreter] Compiling block:
[DEBUG polkavm::interpreter]   [5]: 19: a5 = 0x6
[DEBUG polkavm::interpreter]   [6]: 22: ret
[DEBUG polkavm::interpreter] Compiled handlers cache size exceeded at 5: 7 > 6; will reset the cache
[TRACE polkavm::interpreter::raw_handlers]   -> resolved to fallthrough
[TRACE polkavm::interpreter::raw_handlers] [5]: reset_cache
[DEBUG polkavm::interpreter] Compiling block:
[DEBUG polkavm::interpreter]   [2]: 19: a5 = 0x6
[DEBUG polkavm::interpreter]   [3]: 22: ret
[TRACE polkavm::interpreter::raw_handlers] [2]: a5 = 0x6
[TRACE polkavm::interpreter]   set: a5 = 0x6
[TRACE polkavm::interpreter::raw_handlers] [3]: ret
[TRACE polkavm::interpreter]   get: ra = 0xffff0000
[TRACE polkavm::api] Interrupted: Finished
test tests::interpreter_bounded_interpreter_cache ... ok

Fixes: #315

athei · 2025-07-16T15:35:30Z

Finally, we are also adding support to reset FlatMap and allocate its memory on first use.

Is it reseted using the already existing reset_interpreter_cache function?

ghost · 2025-07-16T16:09:04Z

Finally, we are also adding support to reset FlatMap and allocate its memory on first use.

Is it reseted using the already existing reset_interpreter_cache function?

Yes.

This will allow us to free FlatMap memory and allocate it lazily when needed. Therefore, optimizing run time memory usage. Signed-off-by: Aman <aman@parity.io>

...and automatic cache reset when quota is exceeded. Instead of adding a check in the instruction execution loop, we cleverly replace the first instruction of the new basic block with reset cache micro instruction. This keeps the runner loop optimized. In addition, we make sure capacity of the interpreter cache never gets too big. Signed-off-by: Aman <aman@parity.io>

crates/polkavm/src/api.rs

ghost · 2025-07-22T11:15:49Z

Latest commit adds support for #309

koute

Initial review done.

crates/polkavm/src/utils.rs

crates/polkavm/src/interpreter.rs

crates/polkavm-common/src/program.rs

crates/polkavm/src/interpreter.rs

crates/polkavm-common/src/program.rs

crates/polkavm/src/interpreter.rs

athei

AFAIK this is also missing the memory consumed by the flatmap.

crates/polkavm-common/src/program.rs

koute

Took a brief look; I'll finish reviewing later.

crates/polkavm/src/api.rs

crates/polkavm/src/interpreter.rs

ghost · 2025-07-31T11:02:49Z

Thanks @koute, I've addressed your comments in the latest revision.

crates/polkavm-common/src/program.rs

athei

API looks good to me now. But lets wait to Jan's approval.

athei

Both the new structs should be exported from polkavm::program? Just asking. I don't mind anyways because I am importing polkavm-common anyways in order to make use of the assembler.

athei · 2025-08-01T16:49:50Z

Updated my PR to make use of your calculation: paritytech/polkadot-sdk#9267

Seems to work as expected.

crates/polkavm/src/interpreter.rs

crates/polkavm-common/src/program.rs

crates/polkavm/src/interpreter.rs

crates/polkavm-common/src/program.rs

koute · 2025-08-05T06:14:28Z

@athei Is your PR the final version? You're using EstimateInterpreterMemoryUsageArgs::BoundedCache, but you're not using set_interpreter_cache_size_limit?

koute · 2025-08-05T11:09:10Z

Should also do #318 and #319 (can be in a separate PR)

athei · 2025-08-05T11:13:44Z

@athei Is your PR the final version? You're using EstimateInterpreterMemoryUsageArgs::BoundedCache, but you're not using set_interpreter_cache_size_limit?

Oops yeah. I forgot to set the limit. I am just resetting the cache on every sub call. Will add the limit.

athei · 2025-08-05T12:14:28Z

Updated: paritytech/polkadot-sdk@032e005

crates/polkavm-common/src/utils.rs

crates/polkavm-common/src/program.rs

crates/polkavm/src/interpreter.rs

crates/polkavm-common/src/program.rs

crates/polkavm/src/api.rs

crates/polkavm-common/src/program.rs

koute · 2025-08-06T12:05:19Z

Okay, so the number of corner cases that I had to point out in the review makes my head hurt a little, which is a sign that we should let the machine figure out whether this is correct or not. :P

So, I don't want to delay this PR any longer, but once we merge this let's do #320

crates/polkavm-common/src/program.rs

crates/polkavm-common/src/utils.rs

crates/polkavm/src/interpreter.rs

koute · 2025-08-07T10:57:30Z

crates/polkavm/src/tests.rs

    assert_eq!(instance.reg(Reg::A2), 0x4567);
 }

+fn bounded_interpreter_cache(config: Config) {


This test can be slightly improved I think.

Change every basic block to do something different, otherwise executing from the first block and from the fourth block is indistinguishable (every block should have a different effect); e.g. you could just have every basic block set a different single register.

Make the jumps non-monotonic (i.e. have them jump to blocks which are not the next block; as-is those jumps could have as well be replaced with a fallthrough)

Call into the program multiple times (using the same instance), into randomly selected basic blocks (where "random" here is just some arbitrary order you pick.)

Add an assert that max_cache_size_bytes returns an error when some small value is passed to it, e.g. 1 or something like that.

Add an assert that max_cache_size_bytes doesn't return an error when some huge value is passed to it, e.g. 100000 or something like that.

Test with the minimum possible max_cache_size_bytes (easiest way to get minimum without hardcoding: just increment it until set_interpreter_cache_size_limit doesn't return an error)

Done. Thanks for the recommendations!

koute

Feel free to merge.

Reminder that we also need #318, #319 and #320 as a follow-up next. (:

crates/polkavm-common/src/utils.rs

...and also return memory info, such as baseline memory usage and purgeable memory size. In addition, we are making max_cache_size_bytes an upper bound on the cache size. Signed-off-by: Aman <aman@parity.io>

ghost changed the title ~~Polkavm memory~~ Bounded interpreter cache support Jul 16, 2025

Aman added 2 commits July 16, 2025 17:10

utils: Add deferred allocation to FlatMap

87d34cd

This will allow us to free FlatMap memory and allocate it lazily when needed. Therefore, optimizing run time memory usage. Signed-off-by: Aman <aman@parity.io>

athei mentioned this pull request Jul 17, 2025

Add a memory consumption query function for the ProgramBlob #309

Closed

athei reviewed Jul 21, 2025

View reviewed changes

crates/polkavm/src/api.rs Outdated Show resolved Hide resolved

koute reviewed Jul 24, 2025

View reviewed changes

athei reviewed Jul 29, 2025

View reviewed changes

crates/polkavm-common/src/program.rs Outdated Show resolved Hide resolved

crates/polkavm-common/src/program.rs Outdated Show resolved Hide resolved

crates/polkavm/src/interpreter.rs Outdated Show resolved Hide resolved

athei reviewed Jul 30, 2025

View reviewed changes

crates/polkavm-common/src/program.rs Outdated Show resolved Hide resolved

koute reviewed Jul 30, 2025

View reviewed changes

athei mentioned this pull request Jul 30, 2025

pallet-revive: Raise contract size limit to one megabyte and raise call depth to 25 paritytech/polkadot-sdk#9267

Merged

athei reviewed Aug 1, 2025

View reviewed changes

crates/polkavm-common/src/program.rs Outdated Show resolved Hide resolved

athei approved these changes Aug 1, 2025

View reviewed changes

athei reviewed Aug 1, 2025

View reviewed changes

koute reviewed Aug 5, 2025

View reviewed changes

koute reviewed Aug 6, 2025

View reviewed changes

crates/polkavm-common/src/program.rs Outdated Show resolved Hide resolved

koute reviewed Aug 7, 2025

View reviewed changes

koute approved these changes Aug 7, 2025

View reviewed changes

crates/polkavm-common/src/utils.rs Outdated Show resolved Hide resolved

interpreter: Add a verifier function to validate instructions

ec8b637

...and also return memory info, such as baseline memory usage and purgeable memory size. In addition, we are making max_cache_size_bytes an upper bound on the cache size. Signed-off-by: Aman <aman@parity.io>

ghost enabled auto-merge (rebase) August 8, 2025 09:33

ghost merged commit 0ef6ec1 into paritytech:master Aug 8, 2025
12 checks passed

This pull request was closed.

Conversation

ghost commented Jul 16, 2025 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

athei commented Jul 16, 2025

Uh oh!

ghost commented Jul 16, 2025

Uh oh!

Uh oh!

ghost commented Jul 22, 2025

Uh oh!

koute left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

athei left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

koute left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ghost commented Jul 31, 2025

Uh oh!

Uh oh!

athei left a comment

Choose a reason for hiding this comment

Uh oh!

athei left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

athei commented Aug 1, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

koute commented Aug 5, 2025

Uh oh!

koute commented Aug 5, 2025

Uh oh!

athei commented Aug 5, 2025

Uh oh!

athei commented Aug 5, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

koute commented Aug 6, 2025

ghost commented Jul 16, 2025 •

edited by ghost

Loading

athei left a comment •

edited

Loading