Reimplement Wallet, ElectrumExt and Esplora{Async}Ext with redesigned structures.#976
Conversation
evanlinjin
force-pushed
the
wallet_redesign
branch
from
f7b24f9 to
e5de79f
Compare
evanlinjin
changed the title
Wallet with redesinged structures.Wallet with redesigned structures.
evanlinjin
requested review from
LLFourn,
afilini,
danielabrozzoni,
notmandatory and
rajarshimaitra
LLFourn
left a comment
LLFourn
left a comment
There was a problem hiding this comment.
Design looks good. I think we should try and avoid adding another argument to insert_tx on Wallet.
evanlinjin
force-pushed
the
wallet_redesign
branch
from
03817de to
a11b6ba
Compare
evanlinjin
changed the title
Wallet with redesigned structures.Wallet and ElectrumExt with redesigned structures.
evanlinjin
force-pushed
the
wallet_redesign
branch
from
74c16f1 to
e8b9118
Compare
LLFourn
left a comment
LLFourn
left a comment
There was a problem hiding this comment.
The examples are nice.
Instead of commenting out all these files can you just remove them from workspace for a while?
evanlinjin
force-pushed
the
wallet_redesign
branch
from
61d2e9e to
a66dedd
Compare
|
Oh yeah I still need to remove the commented-out examples. |
evanlinjin
changed the title
Wallet and ElectrumExt with redesigned structures.Wallet, ElectrumExt and Esplora{Async}Ext with redesigned structures.
evanlinjin
force-pushed
the
wallet_redesign
branch
3 times, most recently
from
2f1347b to
2e56f1b
Compare
evanlinjin
force-pushed
the
wallet_redesign
branch
from
cd096df to
2e56f1b
Compare
evanlinjin
force-pushed
the
wallet_redesign
branch
3 times, most recently
from
b0c5552 to
d157061
Compare
This allows us to skip adding an extra input to `Wallet::insert_tx`. Also remove redundant logic.
There are a number of improvements that can be done, but it is in a decent state to be usable. Possible improvements: * Remove requirement to retry obtaining ALL data after reorg is detected. Transactions can be anchored to a lower block (not block tip), and an `assume_final_depth` value can be used. * The logic to finalize an update with confirmation time can be improved during reorgs to not require returning an error.
This is the equivalent of `keychain_tracker_example_cli` that works with the redesigned structures.
This is a version of `keychain_tracker_electrum` that uses the redesigned structures instead.
This corresponds to `keychain::KeychainChangeSet` but for the redesigned structures with `LocalChain`. This structure is now used in `Wallet` as well as the examples.
As per reviews by @danielabrozzoni and @LLFourn
* `ElectrumUpdate::missing_full_txs` now returns a `Vec<Txid>` so we don't keep a reference to the passed-in `graph`. * `ElectrumUpdate::finalize*` methods now takes in `missing` txids instead of `full_txs`. `Client::batch_transaction_get` is called within the methods. Other changes: * `wallet::ChangeSet` is now made public externally. This is required as a wallet db should implement `PersistBackend<wallet::ChangeSet>`.
All associated examples are also updated.
Other changes: * The `async-https` feature of `bdk_esplora` is no longer default. * Rename `ObservedAs` to `ChainPosition`. * Set temporary MSRV to 1.60.0 to compile all workspace members will all features.
* Changed `tx` to `txs` * Changed `txout` to `txouts`
Also updated the documentation.
* Add `warn(missing_docs)` for `bdk_wallet` and `bdk_chain`. * Add missing documentation. * Remove `LocalChain::heights` method. * Remove old TODOs.
evanlinjin
force-pushed
the
wallet_redesign
branch
from
250458a to
75f8b81
Compare
danielabrozzoni
left a comment
danielabrozzoni
left a comment
There was a problem hiding this comment.
I finished reviewing electrum as well, I haven't really looked into electrum_ext as I had troubles figuring out what was happening, I left a few comments asking for more comments (lol)
It's probably obvious at this point but I really think it would have helped w/ reviewing and updating if this whole PR was split in at least 3 or 4 smaller ones (wallet, esplora, electrum, example cli). Something to keep in mind for next time!
| pub command: Commands<S>, | ||
| } | ||
|
|
||
| #[allow(clippy::almost_swapped)] |
There was a problem hiding this comment.
Weird, I don't have warnings for clippy on cargo 1.69 nor 1.57 🤷🏻♀️
There was a problem hiding this comment.
I think it was caused by this regression (rust-lang/rust-clippy#10421) which is fixed in the latest version for cargo clippy. Will remove.
| @@ -0,0 +1,736 @@ | |||
| pub use anyhow; | |||
There was a problem hiding this comment.
This whole file would be way easier to review if there was one move-only commit, followed by commits changing stuff, if needed. In this way, I could just review the commits changing the logic. Just something to keep in mind for next time!
There was a problem hiding this comment.
Apologies. Will keep in mind for next time.
| .blocks() | ||
| .iter() | ||
| .rev() | ||
| .take(ASSUME_FINAL_DEPTH) |
There was a problem hiding this comment.
I'm a bit lost. It seems that you're taking the latest ASSUME_FINAL_DEPTH blocks. Why? Can you add a comment in the code explaining what's happening?
There was a problem hiding this comment.
Will add a comment. Basically anything as deep or deeper than ASSUME_FINAL_DEPTH will not be reorged out (based on our assumption). This is essentially the minimum checkpoints from the tip to include (from the original chain) to guarantee that we will find a point of agreement.
There was a problem hiding this comment.
Note that #1002 is removing this ASSUME_FINAL_DEPTH thing.
| .collect() | ||
| } | ||
|
|
||
| pub fn finalize( |
There was a problem hiding this comment.
A comment explaining what it means to finalize an update would be helpful
|
|
||
| impl ElectrumExt<ConfirmationHeightAnchor> for Client { | ||
| fn get_tip(&self) -> Result<(u32, BlockHash), Error> { | ||
| // TODO: unsubscribe when added to the client, or is there a better call to use here? |
There was a problem hiding this comment.
What does it mean to "unsubscribe when added to the client"?
There was a problem hiding this comment.
The electrum API has a subscription model. We are subscribing to new tips coming in (we get the first tip), but we don't process later tips, so we should unsubscribe from receiving more tips.
There was a problem hiding this comment.
The electrum API has a subscription model. We are subscribing to new tips coming in (we get the first tip), but we don't process later tips, so we should unsubscribe from receiving more tips.
Yep, but why "when added to the client" - unsubscribe when what is added to the client, the first tip?
| } | ||
| } | ||
|
|
||
| fn populate_with_outpoints<K>( |
There was a problem hiding this comment.
Pls put a comment explaining what this method does. What is being populate with outpoints? The update?
There was a problem hiding this comment.
Yes, the update. We already have a comment on scan explaining the outpoints input though?
| None => continue, | ||
| }; | ||
| // attempt to find the following transactions (alongside their chain positions), and | ||
| // add to our sparsechain `update`: |
There was a problem hiding this comment.
not a sparsechain anymore :)
| } | ||
| } | ||
|
|
||
| // Insert the new tip so new transactions will be accepted into the sparsechain. |
There was a problem hiding this comment.
Not a sparsechain anymore
| .map(|data| (data.height as u32, data.header.block_hash())) | ||
| } | ||
|
|
||
| fn scan<K: Ord + Clone>( |
There was a problem hiding this comment.
I think this whole method needs comments explaining what's going on
| .rev() | ||
| .take(ASSUME_FINAL_DEPTH) | ||
| .map(|(k, v)| (*k, *v)) | ||
| .collect::<BTreeMap<u32, BlockHash>>(); |
| .blocks() | ||
| .iter() | ||
| .rev() | ||
| .take(ASSUME_FINAL_DEPTH) |
There was a problem hiding this comment.
Note that #1002 is removing this ASSUME_FINAL_DEPTH thing.
danielabrozzoni
left a comment
danielabrozzoni
left a comment
There was a problem hiding this comment.
Partial ACK 75f8b81 - the Wallet code looks good to me, I don't have a good enough understanding of the esplora/electrum code to confidently ACK it.
7 participants
Description
Closes #938
Walletto use redesigned structures.bdk_electrum::ElectrumExtto produce updates for redesigned structures.bdk_esplora::EsploraExtandbdk_esplora::EsploraAsyncExtto produce updates for redesigned structures.example-crates/example_clilibrary for implementing examples with redesigned structures.example-crate/example_electrumwhich is an electrum CLI wallet using the redesigned structures.example-crate/{wallet_electrum|wallet_esplora|wallet_esplora_async}examples to use redesigned structures.Notes to the reviewers
These changes bump ourNo longer needed due to #993all-featuresMSRV to1.60.0because of the introduction ofbdk_esplora. As long as thebdk_chainandbdk_walletcrates hit a MSRV of1.48.0, it will be fine (this work is done in #987).I had to comment out the examples that useWalletwith our chain sources. Once we update the helper-packages for those chain sources, we can also update the examples.Possible future improvements for
ElectrumExt:Remove requirement to retry obtaining ALL data after reorg is detected. Transactions can be anchored to a lower block (not block tip), and an
assume_final_depthvalue can be used.The logic to finalize an update with confirmation time can be improved during reorgs to not require returning an error.
Use the subscription model of electrum, as intended by the API.
Changelog notice
Checklists
All Submissions:
cargo fmtandcargo clippybefore committingNew Features: