Skip to content

Conversation

@alexcrichton
Copy link
Member

This commit is a redesign of how function-level configuration works in
Wasmtime's bindgen! macro. The main goal of this redesign is to
better support WASIp3 and component model async functions. Prior to this
redesign there was a mish mash of mechanisms to configure behavior of
imports/exports:

  • The async configuration could turn everything async, nothing async,
    only some imports async, or everything except some imports async.

  • The concurrent_{imports,exports} keys were required to explicitly
    opt-in to component model async signatures and applied to all
    imports/exports.

  • The trappable_imports configuration would indicate a list of imports
    allowed to trap and it had special configuration for everything,
    nothing, and only a certain list.

  • The tracing and verbose_tracing keys could be applied to either
    nothing or all functions.

Overall the previous state of configuration in bindgen! was clearly a
hodgepodge of systems that organically grew over time. In my personal
opinion it was in dire need of a refresh to take into account how
component-model-async ended up being implemented as well as
consolidating the one-off systems amongst all of these configuration
keys. A major motivation of this redesign, for example, was to inherit
behavior from WIT files by default. An async function in WIT should
not require concurrent_* keys to be configured, but rather it should
generate correct bindings by default.

In this commit, all of the above keys were removed. All keys have been
replaced with imports and exports configuration keys. Each behaves
the same way and looks like so:

bindgen!({
    // ...
    imports: {
        // enable tracing for just this function
        "my:local/interface/func": tracing,

        // enable verbose tracing for just this function
        "my:local/interface/other-func": tracing | verbose_tracing,

        // this is blocking in WIT, but generate async bindings for
        // it
        "my:local/interface/[method]io.block": async,

        // like above, but use "concurrent" bindings which have
        // access to the store.
        "my:local/interface/[method]io.block-again": async | store,

        // everything else is, by default, trappable
        default: trappable,
    },
});

Effectively all the function-level configuration items are now bitflags.
These bitflags are by default inherited from the WIT files itself (e.g.
async functions are async | store by default). Further configuration
is then layered on top at the desires of the embedder. Supported keys are:

  • async - this means that a Rust-level async function should be
    generated. This is either CallStyle::Async or
    CallStyle::Concurrent as it was prior, depending on ...

  • store - this means that the generated function will have access to
    the store on the host. This is only implemented right now for async | store functions which map to CallStyle::Concurrent. In the future
    I'd like to support just-store functions which means that you could
    define a synchronous function with access to the store in addition to
    an asynchronous function.

  • trappable - this means that the function returns a
    wasmtime::Result<TheWitBindingType>. If trappable_errors is
    applicable then it means just a Result<TheWitOkType, TrappableErrorType> is returned (like before)

  • tracing - this enables tracing! integration for this function.

  • verbose_tracing - this logs all argument values for this function
    (including lists).

  • ignore_wit - this ignores the WIT-level defaults of the function
    (e.g. ignoring WIT async).

The way this then works is all modeled is that for any WIT function
being generated there are a set of flags associated with that function.
To calculate the flags the algorithm looks like:

  1. Find the first matching rule in the imports or exports map
    depending on if the function is imported or exported. If there is no
    matching rule then use the default rule if present. This is the
    initial set of flags for the function (or empty if nothing was
    found).

  2. If ignore_wit is present, return the flags from step 1. Otherwise
    add in async | store if the function is async in WIT.

The resulting set of flags are then used to control how everything is
generated. For example the same split traits of today are still
generated and it's controlled based on the flags. Note though that the
previous HostConcurrent trait was renamed to HostWithStore to make
space for synchronous functions in this trait in the future too.

The end result of all these changes is that configuring imports/exports
now uses the exact same selection system as the with replacement map,
meaning there's only one system of selecting functions instead of 3.
WIT-level async is now respected by default meaning that bindings work
by default without further need to configure anything (unless more
functionality is desired).

One final minor change made here as well is that auto-generated
instantiate methods are now always synchronous and an
instantiate_async method is unconditionally generated for async mode.
This means that bindings always generate both functions and it's up to
the embedder to choose the appropriate one.

Closes #11246
Closes #11247

This commit is a redesign of how function-level configuration works in
Wasmtime's `bindgen!` macro. The main goal of this redesign is to
better support WASIp3 and component model async functions. Prior to this
redesign there was a mish mash of mechanisms to configure behavior of
imports/exports:

* The `async` configuration could turn everything async, nothing async,
  only some imports async, or everything except some imports async.

* The `concurrent_{imports,exports}` keys were required to explicitly
  opt-in to component model async signatures and applied to all
  imports/exports.

* The `trappable_imports` configuration would indicate a list of imports
  allowed to trap and it had special configuration for everything,
  nothing, and only a certain list.

* The `tracing` and `verbose_tracing` keys could be applied to either
  nothing or all functions.

Overall the previous state of configuration in `bindgen!` was clearly a
hodgepodge of systems that organically grew over time. In my personal
opinion it was in dire need of a refresh to take into account how
component-model-async ended up being implemented as well as
consolidating the one-off systems amongst all of these configuration
keys. A major motivation of this redesign, for example, was to inherit
behavior from WIT files by default. An `async` function in WIT should
not require `concurrent_*` keys to be configured, but rather it should
generate correct bindings by default.

In this commit, all of the above keys were removed. All keys have been
replaced with `imports` and `exports` configuration keys. Each behaves
the same way and looks like so:

    bindgen!({
        // ...
        imports: {
            // enable tracing for just this function
            "my:local/interface/func": tracing,

            // enable verbose tracing for just this function
            "my:local/interface/other-func": tracing | verbose_tracing,

            // this is blocking in WIT, but generate async bindings for
            // it
            "my:local/interface/[method]io.block": async,

            // like above, but use "concurrent" bindings which have
            // access to the store.
            "my:local/interface/[method]io.block-again": async | store,

            // everything else is, by default, trappable
            default: trappable,
        },
    });

Effectively all the function-level configuration items are now bitflags.
These bitflags are by default inherited from the WIT files itself (e.g.
`async` functions are `async | store` by default). Further configuration
is then layered on top at the desires of the embedder. Supported keys are:

* `async` - this means that a Rust-level `async` function should be
  generated. This is either `CallStyle::Async` or
  `CallStyle::Concurrent` as it was prior, depending on ...

* `store` - this means that the generated function will have access to
  the store on the host. This is only implemented right now for `async |
  store` functions which map to `CallStyle::Concurrent`. In the future
  I'd like to support just-`store` functions which means that you could
  define a synchronous function with access to the store in addition to
  an asynchronous function.

* `trappable` - this means that the function returns a
  `wasmtime::Result<TheWitBindingType>`. If `trappable_errors` is
  applicable then it means just a `Result<TheWitOkType,
  TrappableErrorType>` is returned (like before)

* `tracing` - this enables `tracing!` integration for this function.

* `verbose_tracing` - this logs all argument values for this function
  (including lists).

* `ignore_wit` - this ignores the WIT-level defaults of the function
  (e.g. ignoring WIT `async`).

The way this then works is all modeled is that for any WIT function
being generated there are a set of flags associated with that function.
To calculate the flags the algorithm looks like:

1. Find the first matching rule in the `imports` or `exports` map
   depending on if the function is imported or exported. If there is no
   matching rule then use the `default` rule if present. This is the
   initial set of flags for the function (or empty if nothing was
   found).

2. If `ignore_wit` is present, return the flags from step 1. Otherwise
   add in `async | store` if the function is `async` in WIT.

The resulting set of flags are then used to control how everything is
generated. For example the same split traits of today are still
generated and it's controlled based on the flags. Note though that the
previous `HostConcurrent` trait was renamed to `HostWithStore` to make
space for synchronous functions in this trait in the future too.

The end result of all these changes is that configuring imports/exports
now uses the exact same selection system as the `with` replacement map,
meaning there's only one system of selecting functions instead of 3.
WIT-level `async` is now respected by default meaning that bindings work
by default without further need to configure anything (unless more
functionality is desired).

One final minor change made here as well is that auto-generated
`instantiate` methods are now always synchronous and an
`instantiate_async` method is unconditionally generated for async mode.
This means that bindings always generate both functions and it's up to
the embedder to choose the appropriate one.

Closes bytecodealliance#11246
Closes bytecodealliance#11247
@alexcrichton alexcrichton requested review from a team as code owners July 26, 2025 21:08
@alexcrichton alexcrichton requested review from pchickey and removed request for a team July 26, 2025 21:08
This helps when using the `with` mapping since that can always assume
that `HostWithStore` is available in the generated bindings, avoiding
the need to duplicate configuration options.
@alexcrichton
Copy link
Member Author

One other large-ish change now here, *WithStore is always generated no matter what. That'll make the generated docs slightly more confusing but I think it's worth the tradeoff of enabling bindings to always assume the trait is there.

Copy link
Contributor

@pchickey pchickey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is a nice improvement! In the future I'd probably like it to be tracing or tracing(verbose) instead of another keyword for verbose_tracing, but its not necessary to do that now.

@alexcrichton alexcrichton enabled auto-merge July 28, 2025 18:06
@alexcrichton alexcrichton added this pull request to the merge queue Jul 28, 2025
Merged via the queue into bytecodealliance:main with commit 1155d6d Jul 28, 2025
166 checks passed
@alexcrichton alexcrichton deleted the refactor-bindgen branch July 28, 2025 18:55
alexcrichton added a commit to alexcrichton/wasmtime that referenced this pull request Jul 30, 2025
This fixes a minor merge conflict between bytecodealliance#11325 and bytecodealliance#11328 which isn't
currently exercised by in-repo WASI bindings but will be soon once
wasip3-prototyping is finished merging.
github-merge-queue bot pushed a commit that referenced this pull request Jul 30, 2025
* Account for concurrent resource destructors

This fixes a minor merge conflict between #11325 and #11328 which isn't
currently exercised by in-repo WASI bindings but will be soon once
wasip3-prototyping is finished merging.

* Update expanded test expectations
bongjunj pushed a commit to prosyslab/wasmtime that referenced this pull request Oct 20, 2025
* Redesign function configuration in `bindgen!`

This commit is a redesign of how function-level configuration works in
Wasmtime's `bindgen!` macro. The main goal of this redesign is to
better support WASIp3 and component model async functions. Prior to this
redesign there was a mish mash of mechanisms to configure behavior of
imports/exports:

* The `async` configuration could turn everything async, nothing async,
  only some imports async, or everything except some imports async.

* The `concurrent_{imports,exports}` keys were required to explicitly
  opt-in to component model async signatures and applied to all
  imports/exports.

* The `trappable_imports` configuration would indicate a list of imports
  allowed to trap and it had special configuration for everything,
  nothing, and only a certain list.

* The `tracing` and `verbose_tracing` keys could be applied to either
  nothing or all functions.

Overall the previous state of configuration in `bindgen!` was clearly a
hodgepodge of systems that organically grew over time. In my personal
opinion it was in dire need of a refresh to take into account how
component-model-async ended up being implemented as well as
consolidating the one-off systems amongst all of these configuration
keys. A major motivation of this redesign, for example, was to inherit
behavior from WIT files by default. An `async` function in WIT should
not require `concurrent_*` keys to be configured, but rather it should
generate correct bindings by default.

In this commit, all of the above keys were removed. All keys have been
replaced with `imports` and `exports` configuration keys. Each behaves
the same way and looks like so:

    bindgen!({
        // ...
        imports: {
            // enable tracing for just this function
            "my:local/interface/func": tracing,

            // enable verbose tracing for just this function
            "my:local/interface/other-func": tracing | verbose_tracing,

            // this is blocking in WIT, but generate async bindings for
            // it
            "my:local/interface/[method]io.block": async,

            // like above, but use "concurrent" bindings which have
            // access to the store.
            "my:local/interface/[method]io.block-again": async | store,

            // everything else is, by default, trappable
            default: trappable,
        },
    });

Effectively all the function-level configuration items are now bitflags.
These bitflags are by default inherited from the WIT files itself (e.g.
`async` functions are `async | store` by default). Further configuration
is then layered on top at the desires of the embedder. Supported keys are:

* `async` - this means that a Rust-level `async` function should be
  generated. This is either `CallStyle::Async` or
  `CallStyle::Concurrent` as it was prior, depending on ...

* `store` - this means that the generated function will have access to
  the store on the host. This is only implemented right now for `async |
  store` functions which map to `CallStyle::Concurrent`. In the future
  I'd like to support just-`store` functions which means that you could
  define a synchronous function with access to the store in addition to
  an asynchronous function.

* `trappable` - this means that the function returns a
  `wasmtime::Result<TheWitBindingType>`. If `trappable_errors` is
  applicable then it means just a `Result<TheWitOkType,
  TrappableErrorType>` is returned (like before)

* `tracing` - this enables `tracing!` integration for this function.

* `verbose_tracing` - this logs all argument values for this function
  (including lists).

* `ignore_wit` - this ignores the WIT-level defaults of the function
  (e.g. ignoring WIT `async`).

The way this then works is all modeled is that for any WIT function
being generated there are a set of flags associated with that function.
To calculate the flags the algorithm looks like:

1. Find the first matching rule in the `imports` or `exports` map
   depending on if the function is imported or exported. If there is no
   matching rule then use the `default` rule if present. This is the
   initial set of flags for the function (or empty if nothing was
   found).

2. If `ignore_wit` is present, return the flags from step 1. Otherwise
   add in `async | store` if the function is `async` in WIT.

The resulting set of flags are then used to control how everything is
generated. For example the same split traits of today are still
generated and it's controlled based on the flags. Note though that the
previous `HostConcurrent` trait was renamed to `HostWithStore` to make
space for synchronous functions in this trait in the future too.

The end result of all these changes is that configuring imports/exports
now uses the exact same selection system as the `with` replacement map,
meaning there's only one system of selecting functions instead of 3.
WIT-level `async` is now respected by default meaning that bindings work
by default without further need to configure anything (unless more
functionality is desired).

One final minor change made here as well is that auto-generated
`instantiate` methods are now always synchronous and an
`instantiate_async` method is unconditionally generated for async mode.
This means that bindings always generate both functions and it's up to
the embedder to choose the appropriate one.

Closes bytecodealliance#11246
Closes bytecodealliance#11247

* Update expanded test expectations

prtest:full

* Fix the min platform embedding example

* Fix doc tests

* Always generate `*WithStore` traits

This helps when using the `with` mapping since that can always assume
that `HostWithStore` is available in the generated bindings, avoiding
the need to duplicate configuration options.

* Update test expectations

* Review comments
bongjunj pushed a commit to prosyslab/wasmtime that referenced this pull request Oct 20, 2025
* Account for concurrent resource destructors

This fixes a minor merge conflict between bytecodealliance#11325 and bytecodealliance#11328 which isn't
currently exercised by in-repo WASI bindings but will be soon once
wasip3-prototyping is finished merging.

* Update expanded test expectations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

wit-bindgen async directives are not composable Bindgen does not support WIT async directives

3 participants