Improve Custom Transaction Experience in Rust Client and Web Client

[dagarcia7]

Preface

I was tempted to tack on a comment to the discussion already open here, but figured this may be different enough to merit its own discussion.

Currently, the API of the WebClient mirrors that of the rust client 1:1 (or at least is very close to 1:1). For the most part, the functions exposed via the WebClient and rust client are pretty user friendly -- i.e. think getAccount(), getTransactions(), newMintTransactionRequest(), etc.

However, the simplicity of the API breaks down for custom transactions. This rust test is a good example of the complexity involved in executing a transaction with a custom script. This is the web client equivalent.

Already, around 90% or so of the web client classes listed here are a result of supporting custom transactions in web. Additionally, as more requests like this pop up to augment the capabilities of the web client to provide feature parity with that of the rust client, the more down the rabbit hole you have to go to expose the necessary classes (and corresponding APIs) in order to expose this functionality in web.

Discussion Topic

So in light of this, I was curious for opinions on whether I should keep going down this path of attempting to expose every function and every class/struct in web that the rust client relies on for its core functionality, or should we be looking to simplify the APIs on both ends to make them user friendly and abstract some of these details away from users in general?

My thinking directly goes to this issue already mentioned where we could achieve this in web by exposing the TransactionScript struct and API, Assembler struct and API, AccountComponent struct and API, etc., but even if we allow web users to use all of these objects, it still seems like a confusing and difficult experience for them to achieve their goal.

Like I said, I think the majority of the rust client and subsequently web client is intuitive enough for users to get their hands dirty quickly. However, beyond most of the pre-packaged transaction types like mint, send, swap, etc., I think taking another pass at the custom transactions flow in general would be super beneficial as this will be what most users need to develop cool things on Miden.

[partylikeits1983]

So in light of this, I was curious for opinions on whether I should keep going down this path of attempting to expose every function and every class/struct in web that the rust client relies on for its core functionality, or should we be looking to simplify the APIs on both ends to make them user friendly and abstract some of these details away from users in general?

I think the WebClient should expose all the same methods that the rust client has. This does mean however that there will be a lot of complexity that frontend users will encounter initially. But I think we can then hide the complexity by building useful WebClient specific abstractions. Right now there is a bit of a roadblock because the WebClient doesn't have all the features of the Rust client making it difficult (or impossible) to do the things that can be done in the Rust client.

I think a good example is the ClientBuilder. It hides certain abstractions, but it didn't limit the developer in any way, it is still possible to initialize the client manually in the more verbose way. I think this concept should apply to the WebClient as well, there should be a 1:1 export of all rust client features to the WebClient, but the WebClient can and should have its own abstractions that build on the primitives exported from the Rust client.

[igamigo]

I too think that the webclient should ideally be as close to the Rust client as possible. I think of the client as a "low-level" tool to interact with Miden, so I think the complexity is fine as long as you can also provide good abstractions as you are suggesting.

However, I think there's probably room for more opinionated and high-level wrappers that could have their own direction. So I wouldn't mind if this route is preferred for the webclient. Though I'd probably suggest separating the project into its own repo since having different directions would likely make it hard to make general changes here.

[igamigo]

I agree, the transaction flow can be a bit more complicated than it would ideally have to be. I'm not sure that test is the best example because a bunch of the code is testing boilerplate and it's sort of more complicated than it has to be (ie, it may not reflect a "realistic" usecase) because it also tests that note args are passed correctly, etc. It's also fairly old and so the setup itself could use some improvements that came after it was initially written. However, on the other hand, that test also does not make use of some of the other tools which are also a pain point IMO. Specifically, setting up output notes data is not the best. The main thing here is that, right now, you need to understand in Rust terms what the output of a MASM script will be. And then you need to create a recipient and mark it as the recipient of an expected output note. The outcome here can be a bit unclear: If you set it up incorrectly on the TransactionRequest, you may get an error because an expected output note did not have the recipient hash you expected. If you don't set it up at all, the transaction will not error but you'll essentially get a partial note out of it, instead of a full note object. If you set it correctly, then you get a full note object out of it. For non-trivial usecases, it can also be a bit more difficult to line up the expected output notes correctly and it may get confusing (example).

I think some of this can improve with docs, some other with tooling and some other with deeper changes (maybe the VM itself can output full note objects if you can create the recipient within the transaction?). In any case, I think user feedback here is crucial as well as I have not really used the client to develop real-world projects yet.

[Dominik1999]

This is a super helpful discussion.

We don't know enough (yet)

As already pointed out one problem is that we don't know what people need in order to build real-world applications using the WebClient or the Rust client. We need to work even more with builders to get to know how they want to build applications. Probably both, the Rust client and WebClient will change quite a bit once we see traction and get feedback.

So my current thinking is assumption based.

(Assumption 1): To get to equality of the Rust client and the WebClient, it is a lot of work. We would need to expose a lot of structures from miden-objects, miden-transaction, and even miden-vm in order to make this work.

(Assumption 2): The WebClient and the Rust client serve different purposes. The WebClient will be used for "smaller" applications that want to interaction with a smart contract. Users will probably just connect their wallet on a website and the website will create a "custom" TransactionRequest for the user to sign. The Rust client will be used for server-side implementations that runs more complex on-chain and off-chain code and synchronisation.

A suggestion for an agile approach

I suggest, we add more and more useful and demanded features to the WebClient, but we don't go for feature parity for the sake of feature parity.

First, we should ensure that it is possible to build all the Rust client tutorials that we have in our docs also in the WebClient. For example, the counter contract including its transaction requests should be easy to build on a WebApp. Here we should ensure that it is possible in a non-hacky way. Especially, the custom TransactionRequest will be part of almost any WebApp (if the app does more than a simple payment). Here, we might need to find better abstraction to have a simpler API for a TransactionRequest than what we have in Rust.

Second, we should have better docs for the WebClient. Currently we have some basic docs as README of the npm package. Ideally, we would have something similar to the Rust docs. Especially, creating a TransactionRequest should be clear from the docs.

Third, @partylikeits1983 should continue to provide feedback on missing features and ways to simplify the WebClient (maybe even the Rust client) by adding tutorials and how tos. First, get all tutorials that are currently in Rust also into the WebClient.

That approach could save some resources by not chasing full feature parity. At the same time, the Rust client could profit from abstractions found on the WebClient and vice versa. A potential downside is that people who are familiar with the Rust client might find it hard to use the WebClient. So for people who use both, this approach might be even more confusing.

Another open question is, how do we integrate the compiler into this. Currently, in both examples of creating a custom TransactionRequest we compile the custom MASM and use that in the request. It is not clear to me yet, how we integrate the Rust client into the TypeScript transaction creation flow.