add component-model-async/{fused|futures|streams}.wast tests #10106

dicej · 2025-01-24T21:09:56Z

This is another piece of #9582 which I'm splitting out to make review easier.

The fused.wast test exercises fused adapter generation for various flavors of
intercomponent async->async, async->sync, and sync->async calls.

The futures.wast and streams.wast tests exercise the various intrinsics
(e.g. stream.read, future.close_writable, etc.) involving futures and
streams.

The remaining changes fill in some TODOs to make the tests pass, plus plumbing
for a few intrinsics which aren't needed for these tests but which set the
foundation for future tests.

alexcrichton

I think this is as far as I'm gonna get today. I haven't gotten to most of trampoline.rs yet which I realize is most of the guts of this PR, but I'll do that on Monday

crates/environ/src/fact/signature.rs

crates/wasmtime/src/runtime/store.rs

crates/wasmtime/src/runtime/vm.rs

crates/cranelift/src/compiler/component.rs

crates/environ/src/fact/trampoline.rs

alexcrichton

Ok I'm trying to decipher trampoline.rs but if it's ok with you I'd prefer to ask for more documentation first before diving further into these methods. Currently there's very little documentation on how things are set up and understanding enough to be able to review this PR I think would require cross-referencing both all the details in the spec along with Wasmtime internal implementation details. A lot of this is also pretty specific to Wasmtime itself in the sense that it's all internal adapter details and we're implementing halves of the spec in some places.

Would you be ok writing up some docs for this? Ideally I'd find it most helpful to have a high-level description of how the adapters piece together and what the expected flow between them is. My hope is that such documentation would also be pretty valuable for future readers to understand this as well.

crates/environ/src/fact/trampoline.rs

alexcrichton

I'm sorry I know I sound like a broken record but I'm personally still very lost trying to understand this. On one hand one way I can fix this is to go study the component-model specification, try to piece together what an expected implementation would be, and then try to connect those dots back to this implementation. On the other hand though I also think it'd be valuable to have enough local documentation to not require that because although I can do that it would also be required for any future readers as well.

If you feel like I'm requesting too much documentation though or there's something that is well outside the purview of this module's documentation and it's more basic fundamental understanding please let me know though. I don't think we should just mirror the entire specification in comments into this repository, but at the same time I still think there are critical implementation pieces lacking comments such as the protocol between the async-start/return adapters as well as the host-provided async-enter/exit functions.

crates/environ/src/component.rs

crates/environ/src/fact/signature.rs

crates/environ/src/fact/trampoline.rs

dicej · 2025-01-29T19:28:44Z

To be clear: I absolutely agree that more docs and comments are needed -- I've only added a bit of that so far but plan to add more. Feel free to ignore this PR until that's done.

This is another piece of bytecodealliance#9582 which I'm splitting out to make review easier. The fused.wast test exercises fused adapter generation for various flavors of intercomponent async->async, async->sync, and sync->async calls. The futures.wast and streams.wast tests exercise the various intrinsics (e.g. `stream.read`, `future.close_writable`, etc.) involving `future`s and `stream`s. The remaining changes fill in some TODOs to make the tests pass, plus plumbing for a few intrinsics which aren't needed for these tests but which set the foundation for future tests. Signed-off-by: Joel Dice <[email protected]>

dicej · 2025-03-04T20:17:43Z

@alexcrichton I've rebooted this PR based on the latest code in the wasip3-prototyping repo. There are lot more comments in the fused adapter code now; let me know if more are needed or the existing ones are unclear.

This PR now includes two more tests (streams.wast and futures.wast) and the wasmtime-cranelift and wasmtime-environ intrinsic plumbing needed to make them pass. All the intrinsics follow the same pattern, and the Cranelift codegen for them share a lot of code, so I figure putting them all in a single PR might be easiest. Let me know if it's too much.

github-actions · 2025-03-04T21:44:42Z

Subscribe to Label Action

cc @fitzgen

Details

This issue or pull request has been labeled: "fuzzing", "wasmtime:api"

Thus the following users have been cc'd because of the following labels:

fitzgen: fuzzing

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

alexcrichton

Thanks for adding these comments!

crates/cranelift/src/compiler/component.rs

alexcrichton · 2025-03-04T21:50:00Z

crates/wasmtime/src/runtime/vm/component.rs

    vmctx: NonNull<VMOpaqueContext>,
    data: NonNull<u8>,
    ty: u32,
+    caller_instance: u32,


Mind documenting this parameter above? (perhaps documenting that it's currently-unused but soon-to-be-used as well)

crates/cranelift/src/compiler/component.rs

dicej · 2025-03-04T23:19:39Z

Thanks for the review, @alexcrichton! I'll address your feedback first thing tomorrow.

Signed-off-by: Joel Dice <[email protected]>

FrankReh · 2025-03-06T17:44:32Z

crates/environ/src/fact.rs

+    /// Import a host built-in function to start a subtask for a sync-lowered
+    /// import call to an async-lifted export.
+    ///
+    /// This call with block until the subtask has produced result(s) via the


"will block"

FrankReh · 2025-03-06T17:53:06Z

crates/environ/src/fact/trampoline.rs

+    }
+
+    // This closure compiles a function to be exported to the host which host to
+    // lift the parameters from the caller and lower them to the callee.


to be exported to the host which host to lift the parameters ... maybe correct but I didn't know how to parse that part of the sentence.

FrankReh · 2025-03-06T17:57:49Z

crates/environ/src/fact/trampoline.rs

+            i32::try_from(self.types[adapter.lift.ty].results.as_u32()).unwrap(),
+        ));
+        // Async-lowered imports pass params and receive results via linear
+        // memory, and those pointers are in the the first and second params to


FrankReh · 2025-03-06T18:02:01Z

crates/environ/src/fact/trampoline.rs

+        self.finish()
+    }
+
+    /// Compiles a function to be exported to the host which host to lift the


exported to the host which host to lift ... apologies if this makes sense to folks more in the know, maybe just hard for me to decipher

dicej requested a review from alexcrichton January 24, 2025 21:09

dicej requested a review from a team as a code owner January 24, 2025 21:09

dicej force-pushed the async-fused-test branch 3 times, most recently from 7e41099 to 098bd1c Compare January 24, 2025 21:40

alexcrichton reviewed Jan 24, 2025

View reviewed changes

github-actions bot added the wasmtime:api Related to the API of the `wasmtime` crate itself label Jan 24, 2025

alexcrichton reviewed Jan 27, 2025

View reviewed changes

dicej requested a review from a team as a code owner January 28, 2025 18:51

dicej requested review from abrown and removed request for a team January 28, 2025 18:51

dicej mentioned this pull request Jan 28, 2025

Support Abi::Array when compiling calls to async intrinsics #10143

Closed

alexcrichton reviewed Jan 29, 2025

View reviewed changes

dicej force-pushed the async-fused-test branch from 4dced39 to 41d7ba2 Compare March 4, 2025 18:46

dicej requested a review from a team as a code owner March 4, 2025 18:46

dicej requested review from alexcrichton and removed request for a team March 4, 2025 18:46

dicej changed the title ~~add component-model-async/fused.wast test~~ add component-model-async/{fused|futures|streams}.wast tests Mar 4, 2025

github-actions bot added the fuzzing Issues related to our fuzzing infrastructure label Mar 4, 2025

alexcrichton approved these changes Mar 4, 2025

View reviewed changes

alexcrichton mentioned this pull request Mar 5, 2025

Update wasm-tools to the latest revision #10314

Merged

address review feedback

32f9b35

Signed-off-by: Joel Dice <[email protected]>

dicej enabled auto-merge March 5, 2025 16:49

dicej added this pull request to the merge queue Mar 5, 2025

Merged via the queue into bytecodealliance:main with commit 812dd1e Mar 5, 2025
39 checks passed

dicej deleted the async-fused-test branch March 5, 2025 17:23

FrankReh reviewed Mar 6, 2025

View reviewed changes

add component-model-async/{fused|futures|streams}.wast tests #10106

add component-model-async/{fused|futures|streams}.wast tests #10106

Uh oh!

Conversation

dicej commented Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexcrichton left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexcrichton left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexcrichton left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dicej commented Jan 29, 2025

Uh oh!

dicej commented Mar 4, 2025

Uh oh!

github-actions bot commented Mar 4, 2025

Subscribe to Label Action

Uh oh!

alexcrichton left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexcrichton Mar 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dicej commented Mar 4, 2025

Uh oh!

Uh oh!

FrankReh Mar 6, 2025

Choose a reason for hiding this comment

Uh oh!

FrankReh Mar 6, 2025

Choose a reason for hiding this comment

Uh oh!

FrankReh Mar 6, 2025

Choose a reason for hiding this comment

Uh oh!

FrankReh Mar 6, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

dicej commented Jan 24, 2025 •

edited

Loading