-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Redesign function configuration in bindgen!
#11328
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Redesign function configuration in bindgen!
#11328
Conversation
This commit is a redesign of how function-level configuration works in
Wasmtime's `bindgen!` macro. The main goal of this redesign is to
better support WASIp3 and component model async functions. Prior to this
redesign there was a mish mash of mechanisms to configure behavior of
imports/exports:
* The `async` configuration could turn everything async, nothing async,
only some imports async, or everything except some imports async.
* The `concurrent_{imports,exports}` keys were required to explicitly
opt-in to component model async signatures and applied to all
imports/exports.
* The `trappable_imports` configuration would indicate a list of imports
allowed to trap and it had special configuration for everything,
nothing, and only a certain list.
* The `tracing` and `verbose_tracing` keys could be applied to either
nothing or all functions.
Overall the previous state of configuration in `bindgen!` was clearly a
hodgepodge of systems that organically grew over time. In my personal
opinion it was in dire need of a refresh to take into account how
component-model-async ended up being implemented as well as
consolidating the one-off systems amongst all of these configuration
keys. A major motivation of this redesign, for example, was to inherit
behavior from WIT files by default. An `async` function in WIT should
not require `concurrent_*` keys to be configured, but rather it should
generate correct bindings by default.
In this commit, all of the above keys were removed. All keys have been
replaced with `imports` and `exports` configuration keys. Each behaves
the same way and looks like so:
bindgen!({
// ...
imports: {
// enable tracing for just this function
"my:local/interface/func": tracing,
// enable verbose tracing for just this function
"my:local/interface/other-func": tracing | verbose_tracing,
// this is blocking in WIT, but generate async bindings for
// it
"my:local/interface/[method]io.block": async,
// like above, but use "concurrent" bindings which have
// access to the store.
"my:local/interface/[method]io.block-again": async | store,
// everything else is, by default, trappable
default: trappable,
},
});
Effectively all the function-level configuration items are now bitflags.
These bitflags are by default inherited from the WIT files itself (e.g.
`async` functions are `async | store` by default). Further configuration
is then layered on top at the desires of the embedder. Supported keys are:
* `async` - this means that a Rust-level `async` function should be
generated. This is either `CallStyle::Async` or
`CallStyle::Concurrent` as it was prior, depending on ...
* `store` - this means that the generated function will have access to
the store on the host. This is only implemented right now for `async |
store` functions which map to `CallStyle::Concurrent`. In the future
I'd like to support just-`store` functions which means that you could
define a synchronous function with access to the store in addition to
an asynchronous function.
* `trappable` - this means that the function returns a
`wasmtime::Result<TheWitBindingType>`. If `trappable_errors` is
applicable then it means just a `Result<TheWitOkType,
TrappableErrorType>` is returned (like before)
* `tracing` - this enables `tracing!` integration for this function.
* `verbose_tracing` - this logs all argument values for this function
(including lists).
* `ignore_wit` - this ignores the WIT-level defaults of the function
(e.g. ignoring WIT `async`).
The way this then works is all modeled is that for any WIT function
being generated there are a set of flags associated with that function.
To calculate the flags the algorithm looks like:
1. Find the first matching rule in the `imports` or `exports` map
depending on if the function is imported or exported. If there is no
matching rule then use the `default` rule if present. This is the
initial set of flags for the function (or empty if nothing was
found).
2. If `ignore_wit` is present, return the flags from step 1. Otherwise
add in `async | store` if the function is `async` in WIT.
The resulting set of flags are then used to control how everything is
generated. For example the same split traits of today are still
generated and it's controlled based on the flags. Note though that the
previous `HostConcurrent` trait was renamed to `HostWithStore` to make
space for synchronous functions in this trait in the future too.
The end result of all these changes is that configuring imports/exports
now uses the exact same selection system as the `with` replacement map,
meaning there's only one system of selecting functions instead of 3.
WIT-level `async` is now respected by default meaning that bindings work
by default without further need to configure anything (unless more
functionality is desired).
One final minor change made here as well is that auto-generated
`instantiate` methods are now always synchronous and an
`instantiate_async` method is unconditionally generated for async mode.
This means that bindings always generate both functions and it's up to
the embedder to choose the appropriate one.
Closes bytecodealliance#11246
Closes bytecodealliance#11247
prtest:full
This helps when using the `with` mapping since that can always assume that `HostWithStore` is available in the generated bindings, avoiding the need to duplicate configuration options.
|
One other large-ish change now here, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, this is a nice improvement! In the future I'd probably like it to be tracing or tracing(verbose) instead of another keyword for verbose_tracing, but its not necessary to do that now.
This fixes a minor merge conflict between bytecodealliance#11325 and bytecodealliance#11328 which isn't currently exercised by in-repo WASI bindings but will be soon once wasip3-prototyping is finished merging.
* Redesign function configuration in `bindgen!`
This commit is a redesign of how function-level configuration works in
Wasmtime's `bindgen!` macro. The main goal of this redesign is to
better support WASIp3 and component model async functions. Prior to this
redesign there was a mish mash of mechanisms to configure behavior of
imports/exports:
* The `async` configuration could turn everything async, nothing async,
only some imports async, or everything except some imports async.
* The `concurrent_{imports,exports}` keys were required to explicitly
opt-in to component model async signatures and applied to all
imports/exports.
* The `trappable_imports` configuration would indicate a list of imports
allowed to trap and it had special configuration for everything,
nothing, and only a certain list.
* The `tracing` and `verbose_tracing` keys could be applied to either
nothing or all functions.
Overall the previous state of configuration in `bindgen!` was clearly a
hodgepodge of systems that organically grew over time. In my personal
opinion it was in dire need of a refresh to take into account how
component-model-async ended up being implemented as well as
consolidating the one-off systems amongst all of these configuration
keys. A major motivation of this redesign, for example, was to inherit
behavior from WIT files by default. An `async` function in WIT should
not require `concurrent_*` keys to be configured, but rather it should
generate correct bindings by default.
In this commit, all of the above keys were removed. All keys have been
replaced with `imports` and `exports` configuration keys. Each behaves
the same way and looks like so:
bindgen!({
// ...
imports: {
// enable tracing for just this function
"my:local/interface/func": tracing,
// enable verbose tracing for just this function
"my:local/interface/other-func": tracing | verbose_tracing,
// this is blocking in WIT, but generate async bindings for
// it
"my:local/interface/[method]io.block": async,
// like above, but use "concurrent" bindings which have
// access to the store.
"my:local/interface/[method]io.block-again": async | store,
// everything else is, by default, trappable
default: trappable,
},
});
Effectively all the function-level configuration items are now bitflags.
These bitflags are by default inherited from the WIT files itself (e.g.
`async` functions are `async | store` by default). Further configuration
is then layered on top at the desires of the embedder. Supported keys are:
* `async` - this means that a Rust-level `async` function should be
generated. This is either `CallStyle::Async` or
`CallStyle::Concurrent` as it was prior, depending on ...
* `store` - this means that the generated function will have access to
the store on the host. This is only implemented right now for `async |
store` functions which map to `CallStyle::Concurrent`. In the future
I'd like to support just-`store` functions which means that you could
define a synchronous function with access to the store in addition to
an asynchronous function.
* `trappable` - this means that the function returns a
`wasmtime::Result<TheWitBindingType>`. If `trappable_errors` is
applicable then it means just a `Result<TheWitOkType,
TrappableErrorType>` is returned (like before)
* `tracing` - this enables `tracing!` integration for this function.
* `verbose_tracing` - this logs all argument values for this function
(including lists).
* `ignore_wit` - this ignores the WIT-level defaults of the function
(e.g. ignoring WIT `async`).
The way this then works is all modeled is that for any WIT function
being generated there are a set of flags associated with that function.
To calculate the flags the algorithm looks like:
1. Find the first matching rule in the `imports` or `exports` map
depending on if the function is imported or exported. If there is no
matching rule then use the `default` rule if present. This is the
initial set of flags for the function (or empty if nothing was
found).
2. If `ignore_wit` is present, return the flags from step 1. Otherwise
add in `async | store` if the function is `async` in WIT.
The resulting set of flags are then used to control how everything is
generated. For example the same split traits of today are still
generated and it's controlled based on the flags. Note though that the
previous `HostConcurrent` trait was renamed to `HostWithStore` to make
space for synchronous functions in this trait in the future too.
The end result of all these changes is that configuring imports/exports
now uses the exact same selection system as the `with` replacement map,
meaning there's only one system of selecting functions instead of 3.
WIT-level `async` is now respected by default meaning that bindings work
by default without further need to configure anything (unless more
functionality is desired).
One final minor change made here as well is that auto-generated
`instantiate` methods are now always synchronous and an
`instantiate_async` method is unconditionally generated for async mode.
This means that bindings always generate both functions and it's up to
the embedder to choose the appropriate one.
Closes bytecodealliance#11246
Closes bytecodealliance#11247
* Update expanded test expectations
prtest:full
* Fix the min platform embedding example
* Fix doc tests
* Always generate `*WithStore` traits
This helps when using the `with` mapping since that can always assume
that `HostWithStore` is available in the generated bindings, avoiding
the need to duplicate configuration options.
* Update test expectations
* Review comments
* Account for concurrent resource destructors This fixes a minor merge conflict between bytecodealliance#11325 and bytecodealliance#11328 which isn't currently exercised by in-repo WASI bindings but will be soon once wasip3-prototyping is finished merging. * Update expanded test expectations
This commit is a redesign of how function-level configuration works in
Wasmtime's
bindgen!macro. The main goal of this redesign is tobetter support WASIp3 and component model async functions. Prior to this
redesign there was a mish mash of mechanisms to configure behavior of
imports/exports:
The
asyncconfiguration could turn everything async, nothing async,only some imports async, or everything except some imports async.
The
concurrent_{imports,exports}keys were required to explicitlyopt-in to component model async signatures and applied to all
imports/exports.
The
trappable_importsconfiguration would indicate a list of importsallowed to trap and it had special configuration for everything,
nothing, and only a certain list.
The
tracingandverbose_tracingkeys could be applied to eithernothing or all functions.
Overall the previous state of configuration in
bindgen!was clearly ahodgepodge of systems that organically grew over time. In my personal
opinion it was in dire need of a refresh to take into account how
component-model-async ended up being implemented as well as
consolidating the one-off systems amongst all of these configuration
keys. A major motivation of this redesign, for example, was to inherit
behavior from WIT files by default. An
asyncfunction in WIT shouldnot require
concurrent_*keys to be configured, but rather it shouldgenerate correct bindings by default.
In this commit, all of the above keys were removed. All keys have been
replaced with
importsandexportsconfiguration keys. Each behavesthe same way and looks like so:
Effectively all the function-level configuration items are now bitflags.
These bitflags are by default inherited from the WIT files itself (e.g.
asyncfunctions areasync | storeby default). Further configurationis then layered on top at the desires of the embedder. Supported keys are:
async- this means that a Rust-levelasyncfunction should begenerated. This is either
CallStyle::AsyncorCallStyle::Concurrentas it was prior, depending on ...store- this means that the generated function will have access tothe store on the host. This is only implemented right now for
async | storefunctions which map toCallStyle::Concurrent. In the futureI'd like to support just-
storefunctions which means that you coulddefine a synchronous function with access to the store in addition to
an asynchronous function.
trappable- this means that the function returns awasmtime::Result<TheWitBindingType>. Iftrappable_errorsisapplicable then it means just a
Result<TheWitOkType, TrappableErrorType>is returned (like before)tracing- this enablestracing!integration for this function.verbose_tracing- this logs all argument values for this function(including lists).
ignore_wit- this ignores the WIT-level defaults of the function(e.g. ignoring WIT
async).The way this then works is all modeled is that for any WIT function
being generated there are a set of flags associated with that function.
To calculate the flags the algorithm looks like:
Find the first matching rule in the
importsorexportsmapdepending on if the function is imported or exported. If there is no
matching rule then use the
defaultrule if present. This is theinitial set of flags for the function (or empty if nothing was
found).
If
ignore_witis present, return the flags from step 1. Otherwiseadd in
async | storeif the function isasyncin WIT.The resulting set of flags are then used to control how everything is
generated. For example the same split traits of today are still
generated and it's controlled based on the flags. Note though that the
previous
HostConcurrenttrait was renamed toHostWithStoreto makespace for synchronous functions in this trait in the future too.
The end result of all these changes is that configuring imports/exports
now uses the exact same selection system as the
withreplacement map,meaning there's only one system of selecting functions instead of 3.
WIT-level
asyncis now respected by default meaning that bindings workby default without further need to configure anything (unless more
functionality is desired).
One final minor change made here as well is that auto-generated
instantiatemethods are now always synchronous and aninstantiate_asyncmethod is unconditionally generated for async mode.This means that bindings always generate both functions and it's up to
the embedder to choose the appropriate one.
Closes #11246
Closes #11247