Refactor wallet and persist mod, remove bdk_persist crate by notmandatory · Pull Request #1454 · bitcoindevkit/bdk

notmandatory · 2024-05-31T23:39:55Z

Description

Sorry to submit another refactor PR for the persist related stuff, but I think it's worth revisiting. My primary motivations are:

remove db from Wallet so users have the ability to use async storage crates, for example using sqlx. I updated docs and examples to let users know they are responsible for persisting changes.
remove the anyhow dependency everywhere (except as a dev test dependency). It really doesn't belong in a lib and by removing persistence from Wallet it isn't needed.
remove the bdk_persist crate and revert back to the original design with generic error types. I kept the Debug and Display constrains on persist errors so they could still be used with the anyhow! macro.

Notes to the reviewers

I also replaced/renamed old Persist with StagedPersist struct inspired by #1453, it is only used in examples. The Wallet handles it's own staging.

Changelog notice

Changed

Removed db from Wallet, users are now responsible for persisting changes, see docs and examples.
Removed the bdk_persist crate and moved logic back to bdk_chain::persist module.
Added PersistBackendAsync, StagedExt, and StageExtAsync traits.
Added Wallet functions commit_to and commit_to_async.

Checklists

All Submissions:

I've signed all my commits
I followed the contribution guidelines
I ran cargo fmt and cargo clippy before committing

crates/chain/src/persist.rs

notmandatory · 2024-06-02T05:05:43Z

@matthiasdebernardini I added an async version of the persist trait called PersistAsync for you to test out with your sqlx crate, see 4ab0648.

evanlinjin · 2024-06-03T15:47:15Z

This is looking good. Removing the persistence backend from wallet is a welcome change. The main concern with doing so is newly revealed addresses not being persisted, but this is addressed with good documentation. ~~Will have a proper review tomorrow.~~ (give me a couple more days)

crates/wallet/src/wallet/mod.rs

crates/chain/src/persist.rs

crates/sqlite/src/store.rs

ValuedMammal · 2024-06-10T23:06:26Z

Maybe worth keeping the Wallet::staged method in order to preview what's currently staged? or else consider returning a cloned changeset from apply_update.

crates/wallet/src/wallet/mod.rs

crates/chain/Cargo.toml

crates/chain/src/persist.rs

evanlinjin

ACK c9d1b41

notmandatory

ACK 6323d55

I rebased to add one more small fix to the bdk_wallet README.md.

Also add refactored StagedPersist to chain crate with tests.

…::persist Also update examples and remove bdk_persist crate.

Still enable the `persist` submodule without `miniscript` feature flag. Only disable `CombinedChangeSet`. Also stop `cargo clippy` from complaining about unused imports when `miniscript` is disabled.

A staging area is helpful because we can contain logic to ignore empty changesets and not clear staging area if the persistence backend fails.

notmandatory · 2024-06-13T18:11:13Z

Had to do one final force push after rebasing on master.

ValuedMammal · 2024-06-13T20:03:15Z

ACK ec36c7e

notmandatory added module-blockchain module-database api A breaking API change labels May 31, 2024

notmandatory added this to the 1.0.0-alpha milestone May 31, 2024

notmandatory self-assigned this May 31, 2024

notmandatory force-pushed the refactor/persist_mod branch from e5a10f7 to e5c73ec Compare June 1, 2024 00:40

notmandatory added the module-wallet label Jun 1, 2024

notmandatory force-pushed the refactor/persist_mod branch 6 times, most recently from 2fc8540 to 6b047ea Compare June 1, 2024 05:08

notmandatory marked this pull request as ready for review June 1, 2024 18:57

notmandatory force-pushed the refactor/persist_mod branch from 6b047ea to 39d4c86 Compare June 1, 2024 19:19

rustaceanrob reviewed Jun 1, 2024

View reviewed changes

crates/chain/src/persist.rs Show resolved Hide resolved

notmandatory modified the milestones: 1.0.0-alpha, 1.0.0-beta Jun 1, 2024

notmandatory force-pushed the refactor/persist_mod branch 3 times, most recently from 96c758c to 4ab0648 Compare June 2, 2024 05:00

ValuedMammal reviewed Jun 10, 2024

View reviewed changes

crates/wallet/src/wallet/mod.rs Outdated Show resolved Hide resolved

ValuedMammal reviewed Jun 10, 2024

View reviewed changes

crates/wallet/src/wallet/mod.rs Show resolved Hide resolved

ValuedMammal reviewed Jun 10, 2024

View reviewed changes

crates/chain/src/persist.rs Outdated Show resolved Hide resolved

crates/sqlite/src/store.rs Outdated Show resolved Hide resolved

ValuedMammal reviewed Jun 10, 2024

View reviewed changes

crates/wallet/src/wallet/mod.rs Show resolved Hide resolved

notmandatory force-pushed the refactor/persist_mod branch 2 times, most recently from c2ac570 to c23d2bc Compare June 12, 2024 23:46

evanlinjin reviewed Jun 13, 2024

View reviewed changes

crates/chain/Cargo.toml Outdated Show resolved Hide resolved

evanlinjin reviewed Jun 13, 2024

View reviewed changes

crates/chain/src/persist.rs Outdated Show resolved Hide resolved

notmandatory force-pushed the refactor/persist_mod branch 2 times, most recently from 395ffa3 to 4b5166b Compare June 13, 2024 05:21

evanlinjin force-pushed the refactor/persist_mod branch 3 times, most recently from c1b3e0c to ded0bfe Compare June 13, 2024 10:42

evanlinjin approved these changes Jun 13, 2024

View reviewed changes

notmandatory force-pushed the refactor/persist_mod branch from c9d1b41 to dbbc8ce Compare June 13, 2024 17:31

notmandatory commented Jun 13, 2024

View reviewed changes

notmandatory force-pushed the refactor/persist_mod branch from dbbc8ce to 6323d55 Compare June 13, 2024 17:35

notmandatory and others added 7 commits June 13, 2024 12:40

refactor(persist): rename PersistBackend to Persist, move to chain crate

aa640ab

Also add refactored StagedPersist to chain crate with tests.

feat(persist): add PersistAsync trait and StagedPersistAsync struct

54b0c11

refactor(persist): update file_store, sqlite, wallet to use bdk_chain…

9e97ac0

…::persist Also update examples and remove bdk_persist crate.

chore(chain): relax miniscript feature flag scope

36e82ec

Still enable the `persist` submodule without `miniscript` feature flag. Only disable `CombinedChangeSet`. Also stop `cargo clippy` from complaining about unused imports when `miniscript` is disabled.

feat(chain): reintroduce a way to stage changesets before persisting

2e40b01

A staging area is helpful because we can contain logic to ignore empty changesets and not clear staging area if the persistence backend fails.

feat(wallet)!: change persist API to use StageExt and StageExtAsync

19328d4

feat(example): use changeset staging with rpc polling example

ec36c7e

notmandatory force-pushed the refactor/persist_mod branch from 6323d55 to ec36c7e Compare June 13, 2024 18:10

ValuedMammal approved these changes Jun 13, 2024

View reviewed changes

notmandatory merged commit 782eb56 into bitcoindevkit:master Jun 13, 2024

This was referenced Jun 14, 2024

Release 1.0.0-alpha.13 #1471

Closed

YANK bdk_persist releases from crates.io #1474

Closed

evanlinjin mentioned this pull request Jun 24, 2024

Support for an async Database in Wallet<D> #795

Closed

rebustron mentioned this pull request Feb 25, 2025

docs: fix dead links in README.md #1856

Closed

8 tasks

notmandatory deleted the refactor/persist_mod branch May 26, 2025 21:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor wallet and persist mod, remove bdk_persist crate#1454

Refactor wallet and persist mod, remove bdk_persist crate#1454
notmandatory merged 7 commits intobitcoindevkit:masterfrom
notmandatory:refactor/persist_mod

notmandatory commented May 31, 2024 •

edited

Loading

Uh oh!

Uh oh!

notmandatory commented Jun 2, 2024

Uh oh!

evanlinjin commented Jun 3, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ValuedMammal commented Jun 10, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

evanlinjin left a comment

Uh oh!

notmandatory left a comment •

edited

Loading

Uh oh!

notmandatory commented Jun 13, 2024 •

edited

Loading

Uh oh!

ValuedMammal commented Jun 13, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

notmandatory commented May 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Notes to the reviewers

Changelog notice

Checklists

All Submissions:

Uh oh!

Uh oh!

notmandatory commented Jun 2, 2024

Uh oh!

evanlinjin commented Jun 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ValuedMammal commented Jun 10, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

evanlinjin left a comment

Choose a reason for hiding this comment

Uh oh!

notmandatory left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

notmandatory commented Jun 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ValuedMammal commented Jun 13, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

notmandatory commented May 31, 2024 •

edited

Loading

evanlinjin commented Jun 3, 2024 •

edited

Loading

notmandatory left a comment •

edited

Loading

notmandatory commented Jun 13, 2024 •

edited

Loading