Persistance for redesigned structures. by evanlinjin · Pull Request #943 · bitcoindevkit/bdk

evanlinjin · 2023-04-21T07:12:30Z

This is part of #895, extends the work done in #926, and replaces the work done in #937.

Description

Within the bdk_chain module, the PR introduces a more generic Persist structure and PersistBackend trait. These allow for any tracker and changeset.

Introduce Store structure for the bdk_chain_store module. Similar to KeychainStore, this is intended to be a PersistBackend. The difference is that Store is more generic, allowing support for different changeset and tracker types.

Notes to the reviewers

I think the Loadable trait that I introduced is a bad idea. Bear with me while I figure out how to remove it!

Changelog notice

Checklists

All Submissions:

I've signed all my commits
I followed the contribution guidelines
I ran cargo fmt and cargo clippy before committing

New Features:

I've added tests for the new feature
I've added docs for the new feature

rajarshimaitra · 2023-04-28T11:33:56Z

Concept ACK.. The new Persistence structure looks simple and good.. I will try to run some manual testing over the Electrum example.,

rajarshimaitra · 2023-04-28T11:48:17Z

example-crates/tracker_example_cli/src/tracker.rs

+pub type RemoteTracker<K, A, O> = Tracker<K, A, RemoteChain<O>>;
+pub type RemoteTrackerStore<K, A, O> = TrackerStore<K, A, RemoteChain<O>, RemoteChainChangeSet>;
+pub type RemoteTrackerChangeSet<K, A> = ChangeSet<K, A, RemoteChainChangeSet>;


Do we plan to have an example with RemoteChain? It could be interesting to measure performance with and without caching chaindata. Also to show how to impl ChainOracle on remote block sources directly.

This is similar to `keychain::Persist`, however, we allow for a generic tracker and changeset.

Similar to `KeychainStore`, this is intended to be a `PersistBackend`. The difference is that `Store` is more generic, allowing support for different changeset and tracker types. `Store` does not implement `PersistBackend` directly (it cannot), but can the logic can be easily reused to implement it.

`PersistBackend` is now two traits: * `PersistBackend` has method `write_changes()`. * `LoadablePersistBackend` has method `load_into_tracker()`. This is because `LoadablePersistBackend` requires a second generic `T` (for the tracker) which `Store` cannot implement for without introducing a new tracker trait, or having a concrete type for `T`. By having two traits, `Store` can still implement `PersistBackend` and the `LoadablePersistBackend` can be implemented individually with each concrete tracker type.

This is an `Anchor` implementation that also informs of the exact confirmation height of the transaction.

Essentially a copy of `keychain_tracker_example_cli` but uses the new data structures. This also requires us to create an example `Tracker`.

Instead of splitting `PersistBackend` into two traits, we introduce `Loadable` which is the mimimum functionality for a tracker to be able to be "loaded into" by the `PersistBackend`. `Loadable` also makes it easier to make trakers "composable". Typically, trackers would wrap other tracker implementations internally, and so the associated changeset would also be wrapped. Having a `Loadable` trait would allow the new tracker implementation to also have a corresponding `PersistBackend`.

Introduce field `Tracker::chain` which represents the best-chain history. It can either be `LocalChain` or `RemoteChain<O: ChainOracle>`. `Tracker::chain` implements `Loadable`, thus allowing persistance no matter the implementation. For `RemoteChain`s, we would want to persist the last-seen height so we know which height to starting syncing from for the next scan.

This creates a duplicate module of `electrum` as `electrum::v2`.

This is `keychain_tracker_electrum_example` using the redesigned structures.

It is not needed. Also moved `tracker_example_cli::RemoteChain` into it's own file.

evanlinjin · 2023-05-01T04:27:36Z

crates/chain/src/chain_data.rs

+    pub anchor_block: BlockId,
+
+    /// The exact confirmation height of the transaction (if any).
+    pub confirmation_height: Option<u32>,


This shouldn't be an option.

The internals are replaced with redesigned structures. It turns out that we need an `Tracker` struct. Fixes: * Fix `FinalExectrumUpdate::into_confirmation_time_update()` * The behaviour of `TxGraph::try_get_chain_position` is also changed so that unconfirmed txs with `last_seen` value of 0 can be part of the chain. Additional changes: * Also introduce `TxGraph::all_anchors` method.

evanlinjin · 2023-05-03T09:56:56Z

This PR is a mess. I will split this up into multiple PRs.

evanlinjin force-pushed the chain_redesign_persist branch from 8117189 to 4e57a9d Compare April 22, 2023 16:28

evanlinjin mentioned this pull request Apr 22, 2023

Fix examples for chain_redesign_tx_graph_anchor #941

Closed

8 tasks

evanlinjin force-pushed the chain_redesign_persist branch from a49123d to 45d376a Compare April 22, 2023 20:21

evanlinjin added the BDK 1.0.0 label Apr 22, 2023

evanlinjin self-assigned this Apr 22, 2023

evanlinjin force-pushed the chain_redesign_persist branch 2 times, most recently from d2ba50e to d0e6e9c Compare April 24, 2023 07:30

evanlinjin mentioned this pull request Apr 25, 2023

W17 BDK Lib Call bitcoindevkit/.github#55

Closed

20 tasks

evanlinjin force-pushed the chain_redesign_persist branch 4 times, most recently from bc7beab to 5ebedaf Compare April 25, 2023 17:52

notmandatory added this to the 1.0.0-alpha.1 milestone Apr 25, 2023

evanlinjin mentioned this pull request Apr 26, 2023

[Feature request] Add timestamp when transaction has been boardcasted/seen first. #587

Closed

rajarshimaitra mentioned this pull request Apr 28, 2023

Update persistance With new IndexedTxGraph #937

Closed

3 tasks

rajarshimaitra reviewed Apr 28, 2023

View reviewed changes

evanlinjin added 13 commits May 1, 2023 11:22

[persist_redesign] Introduce Persist and PersistBackend

841cb92

This is similar to `keychain::Persist`, however, we allow for a generic tracker and changeset.

[persist_redesign] Move EntryIter into it's own file

382f365

[persist_redesign] Implement ConfirmationHeightAnchor

7b022bb

This is an `Anchor` implementation that also informs of the exact confirmation height of the transaction.

[examples_redesign] Introduce tracker_example_cli example crate

b91116c

Essentially a copy of `keychain_tracker_example_cli` but uses the new data structures. This also requires us to create an example `Tracker`.

[bdk_chain_redesign] Add LocalChain::insert_block

5d5735e

[redesign] Reimplement electrum crate to support redesigned structures

0f11556

This creates a duplicate module of `electrum` as `electrum::v2`.

[examples_redesign] Implement tracker_electrum_example

c0a006f

This is `keychain_tracker_electrum_example` using the redesigned structures.

[examples_redesign] Change init to have custom default db path

b44e4c6

[persist_redesign] Remove Loadable trait

e4347b3

It is not needed. Also moved `tracker_example_cli::RemoteChain` into it's own file.

evanlinjin force-pushed the chain_redesign_persist branch from 4e5424e to e4c1661 Compare May 1, 2023 04:25

evanlinjin commented May 1, 2023

View reviewed changes

evanlinjin force-pushed the chain_redesign_persist branch 4 times, most recently from d055f1a to b6280a9 Compare May 1, 2023 06:18

evanlinjin force-pushed the chain_redesign_persist branch from b6280a9 to 7fe84f0 Compare May 1, 2023 06:36

evanlinjin linked an issue May 1, 2023 that may be closed by this pull request

Update Wallet to work with IndexedTxGraph #938

Closed

[wallet_redesign] Have a single Tracker implementation in bdk_chain

26dc69a

evanlinjin closed this May 3, 2023

notmandatory removed the BDK 1.0.0 label May 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Persistance for redesigned structures.#943

Persistance for redesigned structures.#943
evanlinjin wants to merge 15 commits intobitcoindevkit:masterfrom
evanlinjin:chain_redesign_persist

evanlinjin commented Apr 21, 2023 •

edited

Loading

Uh oh!

rajarshimaitra commented Apr 28, 2023

Uh oh!

rajarshimaitra Apr 28, 2023 •

edited

Loading

Uh oh!

evanlinjin May 1, 2023

Uh oh!

evanlinjin commented May 3, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

evanlinjin commented Apr 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Notes to the reviewers

Changelog notice

Checklists

All Submissions:

New Features:

Uh oh!

rajarshimaitra commented Apr 28, 2023

Uh oh!

rajarshimaitra Apr 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evanlinjin May 1, 2023

Choose a reason for hiding this comment

Uh oh!

evanlinjin commented May 3, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

evanlinjin commented Apr 21, 2023 •

edited

Loading

rajarshimaitra Apr 28, 2023 •

edited

Loading