Support custom Rust types for `list` #1442

azdle · 2025-11-26T20:34:28Z

Any type that implements From<Vec<T>>, Into<Vec<T>> ~~and Deref<Target = Vec<T>>~~ can be used for a WIT type list<T> via the with option.

This was simpler than I originally expected, by omitting the type names, I can just let the compiler fall back to calling <Vec<T> as Into<Vec<T>>::into when no custom type is used, allowing the codegen to not need to track the custom type.

Resolves #1441

crates/rust/src/lib.rs

azdle · 2025-11-26T21:54:05Z

Status (mostly as a reminder to myself):

I ended up removing the impl for Deref<Target = Vec<T>> and the test passes, but I do actually need that impl in my real application. The test probably needs to test more, I'm just not sure exactly what yet.
When I don't have the deref impl the errors the compiler spits out are not the most helpful, I want to see if I can find some way to require that trait explicitly to get better errors.

alexcrichton

Would this work for your use case with Bytes? Receiving a Bytes from the outside world would be Bytes: From<Vec<u8>> which is the best we can do, but sending a Bytes to the outside world would unnecessarily require a copy since the shared-reference of Bytes would be copied into a uniquely owned Vec<u8>

azdle · 2025-12-01T21:42:09Z

but sending a Bytes to the outside world would unnecessarily require a copy since the shared-reference of Bytes would be copied into a uniquely owned Vec

I'm not sure what shared reference you're talking about here.

Just to make sure we're on the same page, the Bytes type was talking about in the issue was struct Bytes(Vec<u8>); in the first code block, not a Bytes from the bytes crate. ~~I didn't use the bytes crate because exactly for the reason that it would cause a fully copy of the data.~~ [edit: Actually, this may not be true.]

azdle · 2025-12-01T22:25:01Z

I didn't use the bytes crate because exactly for the reason that it would cause a fully copy of the data.

I was assuming this, but looking at the bytes crate. I'm not sure this is actually true. It sounds like it's capable of converting a Bytes into a BytesMut (https://docs.rs/bytes/latest/bytes/struct.BytesMut.html#method.from) and BytesMut into a Vec<u8> (https://docs.rs/bytes/latest/src/bytes/bytes_mut.rs.html#1727-1760) without copying, as long as it's all unique.

I tried swapping out my Bytes type for the bytes crate Bytes and it all just worked. Though, I can't be sure if it's doing a full copy or not, but by my reading it shouldn't be.

alexcrichton · 2025-12-04T15:49:45Z

I can ask perhaps a different question, are you looking for zero-copy when you pass a Bytes to an imported function? This PR, for example, does not enable that. Conversion from Bytes to Vec<u8> I believe always performs a copy. I'm less sure about BytesMut, but reading the code it looks like the answer is "sometimes".

azdle · 2025-12-04T16:13:55Z

Sorry if I added to the confusion by looking into the bytes crate types. But my original intent wasn't specifically about the Bytes and/or BytesMut types from the bytes crate. It was only about supporting newtype wrappers. The Bytes type in #1441 was just a newtype wrapper around a Vec<u8>.

When I started implementing this I just figured there was no reason to limit this to straight newtype wrappers and support anything that impl From<Vec<u8>> and Into<Vec<u8>>.

azdle · 2025-12-04T23:15:54Z

At the risk of adding to the confusion, I decided to dig into what the bytes crate does in this case.

As long as you're not doing anything 'interesting' with the Bytes you can roundtrip through it without causing a full copy of the data.

When converting from Vec<u8> to Bytes, it will:

Then, as long as the Bytes hasn't been cloned (or otherwise turned into a KIND_ARC), it will:

Though there is a call to ptr::copy, it's called with ptr and buf equal to each other since our Bytes is pointing to the start of the original Vec, which I have to imagine doesn't cause any actual copying to happen.

[Edit: Because I was curious, my assumption is essentially correct, at least on my system / version of glibc. As long as there's more than 512 bytes of data it'll do a check on src and dst and realize it doesn't need to do anything. At 512 or below it just copies into registers and back. Which could be not ideal in some cases, but probably isn't that big of a deal, though that's getting beyond my experience to know for sure.]

But again, this PR isn't specifically about the bytes crate's Bytes, I just happened to name my newtype struct Bytes in the original issue.

I also pushed more test cases that I wrote to prove to myself that you weren't implying something else, so that's unrelated and I plan to remove the changes around non-canonical lists because that definitely does have a copying problem that I'm not sure how I'd solve.

alexcrichton

Ah ok I see now makes sense, thanks for clarifying!

While this wasn't really the intended purpose of the with key I think it's ok to go ahead and support this. I don't see any problems with it for example per se, and it's a convenient feature to have too.

Thanks for the PR again! With green CI I'd be happy to merge

crates/rust/Cargo.toml

crates/rust/src/bindgen.rs

crates/rust/src/lib.rs

Any type that implements `From<Vec<T>>`, `Into<Vec<T>>`, `Deref<Target=Vec<T>>`, and `Clone` can be used for a WIT type `list<T>` via the `with` option. The `Deref` and `Clone` impls are only required when the type is used as part of a non-canonical type.

azdle · 2025-12-10T17:12:15Z

crates/rust/tests/codegen.rs

+    #[derive(Debug, Clone)]
+    pub struct NewtypedListOfByte(Vec<u8>);
+
+    impl From<Vec<u8>> for NewtypedListOfByte {
+        fn from(value: Vec<u8>) -> Self {
+            NewtypedListOfByte(value)
+        }
+    }
+
+    impl From<NewtypedListOfByte> for Vec<u8> {
+        fn from(value: NewtypedListOfByte) -> Self {
+            value.0
+        }
+    }
+
+    impl Deref for NewtypedListOfByte {
+        type Target = Vec<u8>;
+
+        fn deref(&self) -> &Self::Target {
+            &self.0
+        }
+    }


Just to note Debug, Clone, and Deref are needed because NewtypedListOfBytes appears in a record and the generated code for records requires them. As far as I can tell, Clone isn't strictly necessary, the generated code just derives Clone on the struct. Commenting out the clone derive here doesn't break anything for me:

wit-bindgen/crates/rust/src/interface.rs

Line 2080 in f7634e9

derives.insert("Clone".to_string());

Ideally that clone would be derived only if all the children implement it, but as far as I can tell the only way to do that requires a bit of a hack using some syntax that seems to be in stable by accident (as well as generating the clone impl without derive). Though this is getting a bit off topic...

azdle · 2025-12-10T17:14:41Z

I removed the changes around non-canonical lists and polished up the tests covering more generated code, making the naming more descriptive, and deduplicating a lot of things.

I think this should be ready to go now.

azdle marked this pull request as draft November 26, 2025 20:41

azdle force-pushed the support-list-types branch 2 times, most recently from b3b75d8 to f41ad1f Compare November 26, 2025 21:03

azdle commented Nov 26, 2025

View reviewed changes

crates/rust/src/lib.rs Show resolved Hide resolved

alexcrichton reviewed Dec 1, 2025

View reviewed changes

alexcrichton reviewed Dec 5, 2025

View reviewed changes

crates/rust/Cargo.toml Outdated Show resolved Hide resolved

crates/rust/src/bindgen.rs Outdated Show resolved Hide resolved

crates/rust/src/lib.rs Show resolved Hide resolved

azdle force-pushed the support-list-types branch from 8368d74 to f7634e9 Compare December 10, 2025 16:38

azdle commented Dec 10, 2025

View reviewed changes

azdle marked this pull request as ready for review December 10, 2025 17:14

alexcrichton approved these changes Dec 10, 2025

View reviewed changes

alexcrichton added this pull request to the merge queue Dec 10, 2025

Merged via the queue into bytecodealliance:main with commit a493beb Dec 10, 2025
25 checks passed

azdle deleted the support-list-types branch December 10, 2025 21:21

Support custom Rust types for list #1442

Support custom Rust types for list #1442

Uh oh!

Conversation

azdle commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

azdle commented Nov 26, 2025

Uh oh!

alexcrichton left a comment

Choose a reason for hiding this comment

Uh oh!

azdle commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

azdle commented Dec 1, 2025

Uh oh!

alexcrichton commented Dec 4, 2025

Uh oh!

azdle commented Dec 4, 2025

Uh oh!

azdle commented Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexcrichton left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

azdle Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

azdle commented Dec 10, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Support custom Rust types for `list` #1442

Support custom Rust types for `list` #1442

azdle commented Nov 26, 2025 •

edited

Loading

azdle commented Dec 1, 2025 •

edited

Loading

azdle commented Dec 4, 2025 •

edited

Loading