Skip to content

Add a PolkaVM-based executor#3458

Merged
koute merged 9 commits intoparitytech:masterfrom
koute:master_polkavm_executor
Mar 12, 2024
Merged

Add a PolkaVM-based executor#3458
koute merged 9 commits intoparitytech:masterfrom
koute:master_polkavm_executor

Conversation

@koute
Copy link
Contributor

@koute koute commented Feb 23, 2024

This PR adds a new PolkaVM-based executor to Substrate.

  • The executor can now be used to actually run a PolkaVM-based runtime, and successfully produces blocks.
  • The executor is always compiled-in, but is disabled by default.
  • The SUBSTRATE_ENABLE_POLKAVM environment variable must be set to 1 to enable the executor, in which case the node will accept both WASM and PolkaVM program blobs (otherwise it'll default to WASM-only). This is deliberately undocumented and not explicitly exposed anywhere (e.g. in the command line arguments, or in the API) to disincentivize anyone from enabling it in production. If/when we'll move this into production usage I'll remove the environment variable and do it "properly".
  • I did not use our legacy runtime allocator for the PolkaVM executor, so currently every allocation inside of the runtime will leak guest memory until that particular instance is destroyed. The idea here is that I will work on the Remove unnecessary host functions allocator usage polkadot-fellows/RFCs#4 which will remove the need for the legacy allocator under WASM, and that will also allow us to use a proper non-leaking allocator under PolkaVM.
  • I also did some minor cleanups of the WASM executor and deleted some dead code.

No prdocs included since this is not intended to be an end-user feature, but an unofficial experiment, and shouldn't affect any current production user. Once this is production-ready a full Polkadot Fellowship RFC will be necessary anyway.

@koute koute added the R0-no-crate-publish-required The change does not require any crates to be re-published. label Feb 23, 2024
@koute koute requested a review from a team February 23, 2024 12:27
@michalkucharczyk michalkucharczyk requested a review from a team March 8, 2024 13:48
michalkucharczyk
michalkucharczyk reviewed Mar 8, 2024
View reviewed changes
davxy
davxy approved these changes Mar 8, 2024
View reviewed changes
Copy link
Member

@davxy davxy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I suppose that at some point there will be a follow up to rename WasmError type, WasmInstance trait, ...

@koute koute added this pull request to the merge queue Mar 12, 2024
Merged via the queue into paritytech:master with commit b0f34e4 Mar 12, 2024
@koute koute deleted the master_polkavm_executor branch March 12, 2024 06:01
ordian added a commit that referenced this pull request Mar 16, 2024
@ordian
Merge branch 'master' into ao-send-parent-head-data
227ff93 
* master: (65 commits)
  collator protocol changes for elastic scaling (validator side) (#3302)
  Contracts use polkavm workspace deps (#3715)
  Add elastic scaling support in ParaInherent BenchBuilder  (#3690)
  Removes `as [disambiguation_path]` from `derive_impl` usage (#3652)
  fix(paseo-spec): New Paseo Bootnodes (#3674)
  Improve Penpal runtime + emulated tests (#3543)
  Staking ledger bonding fixes (#3639)
  DescribeAllTerminal for HashedDescription (#3349)
  Increase timeout for assertions (#3680)
  Add subsystems regression tests to CI (#3527)
  Always print connectivity report (#3677)
  Revert "FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280)" (#3665)
  authority-discovery: Add log for debugging DHT authority records (#3668)
  Construct Runtime v2  (#1378)
  Support for `keyring` in runtimes (#2044)
  Add api-name in `cannot query the runtime API version` warning (#3653)
  Add a PolkaVM-based executor (#3458)
  Adds default config for assets pallet (#3637)
  Bump handlebars from 4.3.7 to 5.1.0 (#3248)
  [Collator Selection] Fix weight refund for `set_candidacy_bond` (#3643)
  ...
dharjeezy pushed a commit to dharjeezy/polkadot-sdk that referenced this pull request Mar 24, 2024
@koute @dharjeezy
Add a PolkaVM-based executor (paritytech#3458)
36c417b 
This PR adds a new PolkaVM-based executor to Substrate.

- The executor can now be used to actually run a PolkaVM-based runtime,
and successfully produces blocks.
- The executor is always compiled-in, but is disabled by default.
- The `SUBSTRATE_ENABLE_POLKAVM` environment variable must be set to `1`
to enable the executor, in which case the node will accept both WASM and
PolkaVM program blobs (otherwise it'll default to WASM-only). This is
deliberately undocumented and not explicitly exposed anywhere (e.g. in
the command line arguments, or in the API) to disincentivize anyone from
enabling it in production. If/when we'll move this into production usage
I'll remove the environment variable and do it "properly".
- I did not use our legacy runtime allocator for the PolkaVM executor,
so currently every allocation inside of the runtime will leak guest
memory until that particular instance is destroyed. The idea here is
that I will work on the polkadot-fellows/RFCs#4
which will remove the need for the legacy allocator under WASM, and that
will also allow us to use a proper non-leaking allocator under PolkaVM.
- I also did some minor cleanups of the WASM executor and deleted some
dead code.

No prdocs included since this is not intended to be an end-user feature,
but an unofficial experiment, and shouldn't affect any current
production user. Once this is production-ready a full Polkadot
Fellowship RFC will be necessary anyway.
bgallois pushed a commit to duniter/duniter-polkadot-sdk that referenced this pull request Mar 25, 2024
@koute
Add a PolkaVM-based executor (paritytech#3458)
688c25b 
This PR adds a new PolkaVM-based executor to Substrate.

- The executor can now be used to actually run a PolkaVM-based runtime,
and successfully produces blocks.
- The executor is always compiled-in, but is disabled by default.
- The `SUBSTRATE_ENABLE_POLKAVM` environment variable must be set to `1`
to enable the executor, in which case the node will accept both WASM and
PolkaVM program blobs (otherwise it'll default to WASM-only). This is
deliberately undocumented and not explicitly exposed anywhere (e.g. in
the command line arguments, or in the API) to disincentivize anyone from
enabling it in production. If/when we'll move this into production usage
I'll remove the environment variable and do it "properly".
- I did not use our legacy runtime allocator for the PolkaVM executor,
so currently every allocation inside of the runtime will leak guest
memory until that particular instance is destroyed. The idea here is
that I will work on the polkadot-fellows/RFCs#4
which will remove the need for the legacy allocator under WASM, and that
will also allow us to use a proper non-leaking allocator under PolkaVM.
- I also did some minor cleanups of the WASM executor and deleted some
dead code.

No prdocs included since this is not intended to be an end-user feature,
but an unofficial experiment, and shouldn't affect any current
production user. Once this is production-ready a full Polkadot
Fellowship RFC will be necessary anyway.
@athei athei mentioned this pull request Apr 8, 2024
Closed
5 tasks
@nazar-pc nazar-pc mentioned this pull request Apr 15, 2024
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@michalkucharczyk michalkucharczyk michalkucharczyk approved these changes

@dmitry-markin dmitry-markin dmitry-markin approved these changes

@davxy davxy davxy approved these changes

Assignees

No one assigned

Labels

R0-no-crate-publish-required The change does not require any crates to be re-published.

Projects

SDK Node
Status: done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@koute @michalkucharczyk @dmitry-markin @davxy