Skip to content

Conversation

@meyer9
Copy link
Collaborator

@meyer9 meyer9 commented Oct 8, 2025

Based on #203 , #204

This PR implements StateProvider given a OpProofsStorage instance. It reads most data from the external database, but falls back on the latest provider for block hashes and code by hash similar to the existing historical provider in Reth.

This is an important part to implementing live syncing since we're running the sync process on the DB being created.

In the Reth implementation, proof.rs is contained in trie/db, so I think it makes sense to go in our DB crate. The provider.rs is in the reth-provider crate, but I don't think a separate provider crate helps us here, so I think we should also include that in the trie crate as well.

@wiz-b4c72f16a4
Copy link

wiz-b4c72f16a4 bot commented Oct 8, 2025

Wiz Scan Summary

Scanner Findings
Vulnerability Finding Vulnerabilities -
Data Finding Sensitive Data -
Total -

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try using Wiz Code VS Code Extension.

@meyer9 meyer9 force-pushed the meyer9/169-1-state-provider branch from 0383453 to 1d82611 Compare October 9, 2025 16:22
@meyer9 meyer9 changed the base branch from meyer9/166-backfill-script to meyer9/rename-external October 9, 2025 16:22
@meyer9 meyer9 force-pushed the meyer9/169-1-state-provider branch from 5f2e461 to 74d1ac9 Compare October 9, 2025 16:32
@meyer9 meyer9 marked this pull request as ready for review October 9, 2025 16:34
@emhane emhane added K-feature Kind: feature A-db Area: database W-historical-proofs Workstream: historical-proofs labels Oct 10, 2025
@meyer9 meyer9 marked this pull request as draft October 10, 2025 16:15
@meyer9 meyer9 marked this pull request as ready for review October 13, 2025 17:03
Base automatically changed from meyer9/rename-external to unstable October 14, 2025 06:45
@emhane emhane changed the base branch from unstable to emhane/op-exex October 14, 2025 13:58
Base automatically changed from emhane/op-exex to unstable October 14, 2025 18:08
@meyer9 meyer9 force-pushed the meyer9/169-1-state-provider branch from 067c64e to c365ff3 Compare October 14, 2025 19:23
Copilot AI review requested due to automatic review settings October 14, 2025 19:23
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Implements proof, root, and witness generation abstractions over OpProofsStorage, enabling trie-based state/account/storage proofs, roots, and witnesses sourced from an external (live-sync) storage backend while reusing existing Reth trie logic.

  • Adds OpProofs-backed cursor adapters and factories for trie and hashed views
  • Adds overlay_* helper trait extensions for Proof, StorageProof, StateRoot, StorageRoot, and TrieWitness to combine external DB data with in-memory post state / cached nodes
  • Exposes the new functionality via a new proof module

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
crates/optimism/trie/src/proof.rs Introduces cursor wrappers, factories, and overlay trait implementations for proofs, roots, and witnesses over external storage.
crates/optimism/trie/src/lib.rs Exposes the new proof module publicly.

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines +248 to +255
Self::from_tx(storage.clone(), block_number)
.with_trie_cursor_factory(InMemoryTrieCursorFactory::new(
OpProofsTrieCursorFactory::new(storage.clone(), block_number),
&nodes_sorted,
))
.with_hashed_cursor_factory(HashedPostStateCursorFactory::new(
OpProofsHashedAccountCursorFactory::new(storage, block_number),
&state_sorted,
Copy link

Copilot AI Oct 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

storage is cloned twice (lines 248 and 250) before the original is finally moved (line 254); you can reduce redundant cloning by introducing a single let storage_clone = storage.clone(); and reusing it (same pattern appears in overlay_multiproof at lines 270–273 and overlay_witness at lines 560–563).

Copilot uses AI. Check for mistakes.
),
address,
prefix_set,
// #[cfg(feature = "metrics")]
Copy link

Copilot AI Oct 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leftover commented attribute (#[cfg(feature = "metrics")]) is dead code; remove it or reintroduce proper conditional compilation to avoid confusion.

Suggested change
// #[cfg(feature = "metrics")]

Copilot uses AI. Check for mistakes.
@emhane
Copy link
Member

emhane commented Oct 15, 2025

first pass. which issue does this pr reference/close?

I linked these in the sidebar since the Closes #... wasn't working when I opened this. The sidebar always has the most up to date info though.

ok, thanks, my bad didn't pay attention to that feature

@meyer9 meyer9 requested a review from emhane October 15, 2025 15:26
@emhane emhane added this pull request to the merge queue Oct 15, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 15, 2025
@meyer9 meyer9 added this pull request to the merge queue Oct 15, 2025
@meyer9 meyer9 removed this pull request from the merge queue due to a manual request Oct 15, 2025
@meyer9 meyer9 added this pull request to the merge queue Oct 15, 2025
Merged via the queue into unstable with commit 16041de Oct 15, 2025
40 checks passed
@meyer9 meyer9 deleted the meyer9/169-1-state-provider branch October 15, 2025 21:50
github-merge-queue bot pushed a commit that referenced this pull request Oct 16, 2025
Based on #197

This PR implements the live state collector on top of the
`OpProofsStateProvider` created in the previous PR. This sync process
tries to re-execute all blocks from the current block of the external
database to tip.

---------

Co-authored-by: Emilia Hane <[email protected]>
emhane added a commit that referenced this pull request Oct 20, 2025
Based on #203 ,
#204

This PR implements `StateProvider` given a `OpProofsStorage` instance.
It reads most data from the external database, but falls back on the
latest provider for block hashes and code by hash similar to the
existing historical provider in Reth.

This is an important part to implementing live syncing since we're
running the sync process on the DB being created.

In the Reth implementation, `proof.rs` is contained in `trie/db`, so I
think it makes sense to go in our DB crate. The `provider.rs` is in the
`reth-provider` crate, but I don't think a separate provider crate helps
us here, so I think we should also include that in the trie crate as
well.

---------

Co-authored-by: Emilia Hane <[email protected]>
emhane added a commit that referenced this pull request Oct 20, 2025
Based on #197

This PR implements the live state collector on top of the
`OpProofsStateProvider` created in the previous PR. This sync process
tries to re-execute all blocks from the current block of the external
database to tip.

---------

Co-authored-by: Emilia Hane <[email protected]>
meyer9 added a commit that referenced this pull request Oct 31, 2025
Based on #203 ,
#204

This PR implements `StateProvider` given a `OpProofsStorage` instance.
It reads most data from the external database, but falls back on the
latest provider for block hashes and code by hash similar to the
existing historical provider in Reth.

This is an important part to implementing live syncing since we're
running the sync process on the DB being created.

In the Reth implementation, `proof.rs` is contained in `trie/db`, so I
think it makes sense to go in our DB crate. The `provider.rs` is in the
`reth-provider` crate, but I don't think a separate provider crate helps
us here, so I think we should also include that in the trie crate as
well.

---------

Co-authored-by: Emilia Hane <[email protected]>
meyer9 added a commit that referenced this pull request Oct 31, 2025
Based on #197

This PR implements the live state collector on top of the
`OpProofsStateProvider` created in the previous PR. This sync process
tries to re-execute all blocks from the current block of the external
database to tip.

---------

Co-authored-by: Emilia Hane <[email protected]>
emhane added a commit that referenced this pull request Nov 11, 2025
Based on #203 ,
#204

This PR implements `StateProvider` given a `OpProofsStorage` instance.
It reads most data from the external database, but falls back on the
latest provider for block hashes and code by hash similar to the
existing historical provider in Reth.

This is an important part to implementing live syncing since we're
running the sync process on the DB being created.

In the Reth implementation, `proof.rs` is contained in `trie/db`, so I
think it makes sense to go in our DB crate. The `provider.rs` is in the
`reth-provider` crate, but I don't think a separate provider crate helps
us here, so I think we should also include that in the trie crate as
well.

---------

Co-authored-by: Emilia Hane <[email protected]>
emhane added a commit that referenced this pull request Nov 11, 2025
Based on #197

This PR implements the live state collector on top of the
`OpProofsStateProvider` created in the previous PR. This sync process
tries to re-execute all blocks from the current block of the external
database to tip.

---------

Co-authored-by: Emilia Hane <[email protected]>
meyer9 added a commit that referenced this pull request Nov 13, 2025
Based on #203 ,
#204

This PR implements `StateProvider` given a `OpProofsStorage` instance.
It reads most data from the external database, but falls back on the
latest provider for block hashes and code by hash similar to the
existing historical provider in Reth.

This is an important part to implementing live syncing since we're
running the sync process on the DB being created.

In the Reth implementation, `proof.rs` is contained in `trie/db`, so I
think it makes sense to go in our DB crate. The `provider.rs` is in the
`reth-provider` crate, but I don't think a separate provider crate helps
us here, so I think we should also include that in the trie crate as
well.

---------

Co-authored-by: Emilia Hane <[email protected]>
meyer9 added a commit that referenced this pull request Nov 13, 2025
Based on #197

This PR implements the live state collector on top of the
`OpProofsStateProvider` created in the previous PR. This sync process
tries to re-execute all blocks from the current block of the external
database to tip.

---------

Co-authored-by: Emilia Hane <[email protected]>
emhane added a commit that referenced this pull request Nov 18, 2025
Based on #203 ,
#204

This PR implements `StateProvider` given a `OpProofsStorage` instance.
It reads most data from the external database, but falls back on the
latest provider for block hashes and code by hash similar to the
existing historical provider in Reth.

This is an important part to implementing live syncing since we're
running the sync process on the DB being created.

In the Reth implementation, `proof.rs` is contained in `trie/db`, so I
think it makes sense to go in our DB crate. The `provider.rs` is in the
`reth-provider` crate, but I don't think a separate provider crate helps
us here, so I think we should also include that in the trie crate as
well.

---------

Co-authored-by: Emilia Hane <[email protected]>
emhane added a commit that referenced this pull request Nov 18, 2025
Based on #197

This PR implements the live state collector on top of the
`OpProofsStateProvider` created in the previous PR. This sync process
tries to re-execute all blocks from the current block of the external
database to tip.

---------

Co-authored-by: Emilia Hane <[email protected]>
emhane added a commit that referenced this pull request Nov 25, 2025
Based on #203 ,
#204

This PR implements `StateProvider` given a `OpProofsStorage` instance.
It reads most data from the external database, but falls back on the
latest provider for block hashes and code by hash similar to the
existing historical provider in Reth.

This is an important part to implementing live syncing since we're
running the sync process on the DB being created.

In the Reth implementation, `proof.rs` is contained in `trie/db`, so I
think it makes sense to go in our DB crate. The `provider.rs` is in the
`reth-provider` crate, but I don't think a separate provider crate helps
us here, so I think we should also include that in the trie crate as
well.

---------

Co-authored-by: Emilia Hane <[email protected]>
emhane added a commit that referenced this pull request Nov 25, 2025
Based on #197

This PR implements the live state collector on top of the
`OpProofsStateProvider` created in the previous PR. This sync process
tries to re-execute all blocks from the current block of the external
database to tip.

---------

Co-authored-by: Emilia Hane <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-db Area: database K-feature Kind: feature W-historical-proofs Workstream: historical-proofs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement ExternalStateProvider Implement ExternalTrieCursor and ExternalHashedCursor

4 participants