-
Notifications
You must be signed in to change notification settings - Fork 12.4k
Add TrieProof library #5826
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+1,096
−5
Merged
Add TrieProof library #5826
Changes from all commits
Commits
Show all changes
146 commits
Select commit
Hold shift + click to select a range
9eb5f1c
Add memory utils
ernestognw 2d397f4
Fix tests upgradeable
ernestognw 2a0fb7e
Add docs
ernestognw a7e61c3
Make use of the library
ernestognw 1aae8bb
Update docs/modules/ROOT/pages/utilities.adoc
ernestognw 1b2679a
Merge branch 'master' into utils/memory
Amxx d514606
fix tests
Amxx 14fa04e
Update contracts/utils/Memory.sol
ernestognw d0d55fc
Update contracts/utils/Memory.sol
arr00 7b3cb66
Add RLP library
ernestognw 95149f8
Add TrieProof library
ernestognw ad5d4ac
up
ernestognw 18540ef
Add docs
ernestognw 163f27c
Workaround stack too deep
ernestognw c484289
Add Changesets
ernestognw e0d4790
Add more changesets
ernestognw a6f9053
Add FV and fuzz tests
ernestognw e2b5e4c
Merge branch 'master' into feature/rlp
ernestognw 203d1a2
up
ernestognw 48eabc1
docs
ernestognw 63ced95
up pragma
ernestognw f342756
Add missing Bytes test
ernestognw 23dba37
Add unit tests
ernestognw 0cacca2
up pragma
ernestognw 831e8ab
Move TrieProof
ernestognw 5da111f
Fix countLeadingZeroes
ernestognw ba2293e
nits
ernestognw 9409bc6
Improve
ernestognw e740dac
Fix
ernestognw 0332ffe
Add Memory.sol library
ernestognw 608e3cd
Merge branch 'master' into utils/memory
ernestognw ac92bb4
up
ernestognw 6094bb7
Merge branch 'master' into utils/memory
ernestognw 6bb96d5
WIP: Add more Memory functions
ernestognw 860e5a8
up
ernestognw ecdb768
revert
ernestognw 95907aa
Update docs
ernestognw 124ccee
Nit
ernestognw c3237df
Finish fuzz tests and FV
ernestognw 27f0a9b
up
ernestognw 282ce39
up
ernestognw bdd2cf1
Add operations to Math.sol
ernestognw 42c79f1
Add new equal, nibbles and countLeadingZeroes functions
ernestognw 5754ab8
Rename countLeadingZeroes to clz
ernestognw 44f0e14
up
ernestognw 05c73bd
Pragma changes
ernestognw 3a6fbf6
up
ernestognw e67e8b4
up
ernestognw 3385718
Rename to in Math library and update corresponding tests for consis…
ernestognw 40d7922
Update return types of reverseBits functions to match their respectiv…
ernestognw 89860bc
Refactor reverseBits functions in to use fixed-size byte types
ernestognw 9b58730
Test nits
ernestognw 77ffa8c
Simplify
ernestognw ce91c80
up
ernestognw b3e3add
Move reverse functions to Bytes.sol
ernestognw 2f3107c
Move Bytes.t.sol
ernestognw 4383e01
Merge branch 'master' into feat/bytes-rlp
ernestognw 5a44b11
up
ernestognw d6db2d7
Document
ernestognw 3847050
Remove extra functions
ernestognw 4fd1947
Update docs
ernestognw c4e0375
up
ernestognw acb14cb
Merge branch 'utils/memory' into feature/rlp
ernestognw 2208006
Merge branch 'feat/math-reverse-bits' into feature/rlp
ernestognw 13f4d8f
Merge branch 'feat/bytes-rlp' into feature/rlp
ernestognw 502a520
Merge branch 'master' into feature/rlp
ernestognw aab9274
Merge branch 'master' into feature/rlp
Amxx aa26e48
up
Amxx 948f0a1
Merge branch 'master' into feature/rlp
ernestognw d4bfb8b
Fix compilation
ernestognw 138de7f
Remove dangling clz
ernestognw 5efeb37
Make nibbles function private
ernestognw 00ff228
Remove nibbles test
ernestognw fd7d2b5
up
ernestognw 940ede8
Merge remote-tracking branch 'upstream/master' into feat/trie-proof
james-toussaint 5b93acb
Verify proof
james-toussaint 48c7dcb
Mirror storage on both networks
james-toussaint eaf3527
Gracefully kill anvil
james-toussaint f587b88
Verify proof with hashed key
james-toussaint 03f42b9
Traverse after extension node
james-toussaint 67a74b0
Delete .changeset/lovely-cooks-add.md
Amxx 85d638e
tests update
Amxx 62f255d
remove unecessary string casting
Amxx 4d4edb8
refactor TrieProof library
Amxx 1286c96
add to stateless
Amxx 9853d3d
fuzz tests for Memory.getHash and RLP.readBytesHash
Amxx 2ef93d2
remove radix reference
Amxx 6211aed
update
Amxx 466ecdf
Return error if leaf value is too large
james-toussaint 0b6d207
fix issue identified though optimism-bedrock unit tests
Amxx d47d52e
update
Amxx fc69547
Test invalid internal short node
james-toussaint 702bf9c
update
Amxx 7020ab4
Merge branch 'feat/trie-proof' of https://github.com/ernestognw/openz…
Amxx a37e305
fix issue with small nodes and add unit test from optimism
Amxx 4366235
more unit tests
Amxx 2a9a3ec
Make unit tests validProof 8, 9 and 10 pass
Amxx 60f5d7d
more optimism unit testing
Amxx c3cacbd
testing
Amxx 1ed7f87
typo
Amxx b3c7f16
return bytes memory
Amxx 658c9ca
check node length
Amxx 8f60bec
refactor tests
Amxx 278ce24
expand testing to proving accounts against the block state root
Amxx ea53d08
typo
Amxx 8fbcb34
Apply suggestions from code review
Amxx ff31d24
revert unused new features in RLP
Amxx 0706595
Merge branch 'feat/trie-proof' of https://github.com/ernestognw/openz…
Amxx 0a3e229
Merge branch 'master' into feat/trie-proof
Amxx 6e38f3e
Remove _commonPrefixLength function
Amxx b5bc785
Update test/utils/cryptography/TrieProof.test.js
Amxx 1725d92
use some assembly to optimize memory operations
Amxx 05d05c1
fix test flackiness
Amxx 926d704
move toNibbles to Bytes.sol and update doc
Amxx 47a7d2f
cache pathRemainder.length()
Amxx 3e6b05f
Add toNibbles tests
james-toussaint 77192fe
Update Bytes.t.sol
Amxx 18e1c8e
reorder arguments
Amxx 70d76d9
update
Amxx 13619d5
Update contracts/utils/cryptography/TrieProof.sol
Amxx 8d80329
Handle empty leaf/ext path and empty ext path remainder
james-toussaint 66ddaa1
Rename and reorder errors
james-toussaint 217fdf1
Return extra proof error without reading value
james-toussaint 7a7bb97
Merge branch 'master' into feat/trie-proof
james-toussaint 536dc09
Update TrieProof.sol
Amxx 9603aee
lint
Amxx 39e56bb
add travese / tryTraverse options
Amxx 8f7ed6a
remove superflous check that can be proven as a loop invariant
Amxx 4004388
remove the Node structure
Amxx ad5476b
Update contracts/utils/cryptography/TrieProof.sol
Amxx 19336bf
Remove skipped tests related to removed scenario
james-toussaint 14fe8e4
Update doc with traverse naming
james-toussaint 8707438
fix test flakyness
Amxx 486a562
sanity check the proof value and key
Amxx bdff2ad
spelling
Amxx 80149e4
Merge branch 'master' into feat/trie-proof
ernestognw 0600dd4
Update contracts/utils/cryptography/TrieProof.sol
Amxx 8a04d05
Add comment on path expansion
ernestognw b683357
Add note about 32 length in _getNodeId
ernestognw 86ed474
Remove `Memory.getHash` in favor of `Memory.equal`
ernestognw 6a6a4b2
Test verification of transaction inclusion
Amxx 30cefc6
up
Amxx 784ff8a
Update contracts/utils/cryptography/TrieProof.sol
Amxx d19f5f9
Update contracts/utils/cryptography/TrieProof.sol
ernestognw 2ca2a3d
Lint
ernestognw bb02a8e
verify event inclusion
Amxx File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| 'openzeppelin-solidity': minor | ||
| --- | ||
|
|
||
| `TrieProof`: Add library for verifying Ethereum Merkle-Patricia trie inclusion proofs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| 'openzeppelin-solidity': minor | ||
| --- | ||
|
|
||
| `Bytes`: Add the `toNibbles` function that expands the nibbles (4 bits chunk) of a `bytes` buffer. Used for manipulating Patricia Merkle Trees keys and paths. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,231 @@ | ||
| // SPDX-License-Identifier: MIT | ||
| pragma solidity ^0.8.27; | ||
|
|
||
| import {Math} from "../math/Math.sol"; | ||
| import {Bytes} from "../Bytes.sol"; | ||
| import {Memory} from "../Memory.sol"; | ||
| import {RLP} from "../RLP.sol"; | ||
|
|
||
| /** | ||
| * @dev Library for verifying Ethereum Merkle-Patricia trie inclusion proofs. | ||
| * | ||
| * The {traverse} and {verify} functions can be used to prove the following value: | ||
| * | ||
| * * Transaction against the transactionsRoot of a block. | ||
| * * Event against receiptsRoot of a block. | ||
| * * Account details (RLP encoding of [nonce, balance, storageRoot, codeHash]) against the stateRoot of a block. | ||
| * * Storage slot (RLP encoding of the value) against the storageRoot of a account. | ||
| * | ||
| * Proving a storage slot is usually done in 3 steps: | ||
| * | ||
| * * From the stateRoot of a block, process the account proof (see `eth_getProof`) to get the account details. | ||
| * * RLP decode the account details to extract the storageRoot. | ||
| * * Use storageRoot of that account to process the storageProof (again, see `eth_getProof`). | ||
| * | ||
| * See https://ethereum.org/en/developers/docs/data-structures-and-encoding/patricia-merkle-trie[Merkle-Patricia trie] | ||
| * | ||
| * Based on https://github.com/ethereum-optimism/optimism/blob/ef970556e668b271a152124023a8d6bb5159bacf/packages/contracts-bedrock/src/libraries/trie/MerkleTrie.sol[this implementation from optimism]. | ||
| */ | ||
| library TrieProof { | ||
| using Bytes for *; | ||
| using RLP for *; | ||
| using Memory for *; | ||
|
|
||
| enum Prefix { | ||
| EXTENSION_EVEN, // 0 - Extension node with even length path | ||
| EXTENSION_ODD, // 1 - Extension node with odd length path | ||
| LEAF_EVEN, // 2 - Leaf node with even length path | ||
| LEAF_ODD // 3 - Leaf node with odd length path | ||
| } | ||
|
|
||
| enum ProofError { | ||
| NO_ERROR, // No error occurred during proof traversal | ||
| EMPTY_KEY, // The provided key is empty | ||
| INVALID_ROOT, // The validation of the root node failed | ||
| INVALID_LARGE_NODE, // The validation of a large node failed | ||
| INVALID_SHORT_NODE, // The validation of a short node failed | ||
| EMPTY_PATH, // The path in a leaf or extension node is empty | ||
| INVALID_PATH_REMAINDER, // The path remainder in a leaf or extension node is invalid | ||
| EMPTY_EXTENSION_PATH_REMAINDER, // The path remainder in an extension node is empty | ||
| INVALID_EXTRA_PROOF_ELEMENT, // A leaf value should be the last proof element | ||
| EMPTY_VALUE, // The leaf value is empty | ||
| MISMATCH_LEAF_PATH_KEY_REMAINDER, // The path remainder in a leaf node doesn't match the key remainder | ||
| UNKNOWN_NODE_PREFIX, // The node prefix is unknown | ||
| UNPARSEABLE_NODE, // The node cannot be parsed from RLP encoding | ||
| INVALID_PROOF // General failure during proof traversal | ||
| } | ||
|
|
||
| error TrieProofTraversalError(ProofError err); | ||
|
|
||
| /// @dev The radix of the Ethereum trie | ||
| uint256 internal constant EVM_TREE_RADIX = 16; | ||
|
|
||
| /// @dev Number of items in a branch node (16 children + 1 value) | ||
| uint256 internal constant BRANCH_NODE_LENGTH = EVM_TREE_RADIX + 1; | ||
|
|
||
| /// @dev Number of items in leaf or extension nodes (always 2) | ||
| uint256 internal constant LEAF_OR_EXTENSION_NODE_LENGTH = 2; | ||
|
|
||
| /// @dev Verifies a `proof` against a given `key`, `value`, `and root` hash. | ||
| function verify( | ||
| bytes memory value, | ||
| bytes32 root, | ||
| bytes memory key, | ||
| bytes[] memory proof | ||
| ) internal pure returns (bool) { | ||
| (bytes memory processedValue, ProofError err) = tryTraverse(root, key, proof); | ||
| return processedValue.equal(value) && err == ProofError.NO_ERROR; | ||
| } | ||
|
|
||
| /** | ||
| * @dev Traverses a proof with a given key and returns the value. | ||
| * | ||
| * Reverts with {TrieProofTraversalError} if proof is invalid. | ||
| */ | ||
| function traverse(bytes32 root, bytes memory key, bytes[] memory proof) internal pure returns (bytes memory) { | ||
| (bytes memory value, ProofError err) = tryTraverse(root, key, proof); | ||
| require(err == ProofError.NO_ERROR, TrieProofTraversalError(err)); | ||
| return value; | ||
| } | ||
|
|
||
| /** | ||
| * @dev Traverses a proof with a given key and returns the value and an error flag | ||
| * instead of reverting if the proof is invalid. This function may still revert if | ||
| * malformed input leads to RLP decoding errors. | ||
| */ | ||
| function tryTraverse( | ||
| bytes32 root, | ||
| bytes memory key, | ||
| bytes[] memory proof | ||
| ) internal pure returns (bytes memory value, ProofError err) { | ||
| if (key.length == 0) return (_emptyBytesMemory(), ProofError.EMPTY_KEY); | ||
|
|
||
| // Expand the key | ||
| bytes memory keyExpanded = key.toNibbles(); | ||
|
|
||
| bytes32 currentNodeId; | ||
| uint256 currentNodeIdLength; | ||
|
|
||
| // Free memory pointer cache | ||
| Memory.Pointer fmp = Memory.getFreeMemoryPointer(); | ||
|
|
||
| // Traverse proof | ||
| uint256 keyIndex = 0; | ||
| uint256 proofLength = proof.length; | ||
| for (uint256 i = 0; i < proofLength; ++i) { | ||
| // validates the encoded node matches the expected node id | ||
| bytes memory encoded = proof[i]; | ||
| if (keyIndex == 0) { | ||
| // Root node must match root hash | ||
| if (keccak256(encoded) != root) return (_emptyBytesMemory(), ProofError.INVALID_ROOT); | ||
| } else if (encoded.length >= 32) { | ||
| // Large nodes are stored as hashes | ||
| if (currentNodeIdLength != 32 || keccak256(encoded) != currentNodeId) | ||
| return (_emptyBytesMemory(), ProofError.INVALID_LARGE_NODE); | ||
| } else { | ||
| // Small nodes must match directly | ||
| if (currentNodeIdLength != encoded.length || bytes32(encoded) != currentNodeId) | ||
| return (_emptyBytesMemory(), ProofError.INVALID_SHORT_NODE); | ||
| } | ||
|
|
||
| // decode the current node as an RLP list, and process it | ||
| Memory.Slice[] memory decoded = encoded.decodeList(); | ||
| if (decoded.length == BRANCH_NODE_LENGTH) { | ||
| // If we've consumed the entire key, the value must be in the last slot | ||
| // Otherwise, continue down the branch specified by the next nibble in the key | ||
| if (keyIndex == keyExpanded.length) { | ||
| return _validateLastItem(decoded[EVM_TREE_RADIX], proofLength, i); | ||
| } else { | ||
| bytes1 branchKey = keyExpanded[keyIndex]; | ||
| (currentNodeId, currentNodeIdLength) = _getNodeId(decoded[uint8(branchKey)]); | ||
| keyIndex += 1; | ||
| } | ||
| } else if (decoded.length == LEAF_OR_EXTENSION_NODE_LENGTH) { | ||
| bytes memory path = decoded[0].readBytes().toNibbles(); // expanded path | ||
| // The following is equivalent to path.length < 2 because toNibbles can't return odd-length buffers | ||
| if (path.length == 0) { | ||
| return (_emptyBytesMemory(), ProofError.EMPTY_PATH); | ||
| } | ||
| uint8 prefix = uint8(path[0]); | ||
| Memory.Slice keyRemainder = keyExpanded.asSlice().slice(keyIndex); // Remaining key to match | ||
| Memory.Slice pathRemainder = path.asSlice().slice(2 - (prefix % 2)); // Path after the prefix | ||
ernestognw marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| uint256 pathRemainderLength = pathRemainder.length(); | ||
|
|
||
Amxx marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| // pathRemainder must not be longer than keyRemainder, and it must be a prefix of it | ||
| if ( | ||
| pathRemainderLength > keyRemainder.length() || | ||
| !pathRemainder.equal(keyRemainder.slice(0, pathRemainderLength)) | ||
| ) { | ||
| return (_emptyBytesMemory(), ProofError.INVALID_PATH_REMAINDER); | ||
| } | ||
|
|
||
| if (prefix <= uint8(Prefix.EXTENSION_ODD)) { | ||
| // Eq to: prefix == EXTENSION_EVEN || prefix == EXTENSION_ODD | ||
| if (pathRemainderLength == 0) { | ||
| return (_emptyBytesMemory(), ProofError.EMPTY_EXTENSION_PATH_REMAINDER); | ||
| } | ||
| // Increment keyIndex by the number of nibbles consumed and continue traversal | ||
| (currentNodeId, currentNodeIdLength) = _getNodeId(decoded[1]); | ||
| keyIndex += pathRemainderLength; | ||
| } else if (prefix <= uint8(Prefix.LEAF_ODD)) { | ||
| // Eq to: prefix == LEAF_EVEN || prefix == LEAF_ODD | ||
| // | ||
| // Leaf node (terminal) - return its value if key matches completely | ||
| // we already know that pathRemainder is a prefix of keyRemainder, so checking the length sufficient | ||
| return | ||
| pathRemainderLength == keyRemainder.length() | ||
| ? _validateLastItem(decoded[1], proofLength, i) | ||
| : (_emptyBytesMemory(), ProofError.MISMATCH_LEAF_PATH_KEY_REMAINDER); | ||
| } else { | ||
| return (_emptyBytesMemory(), ProofError.UNKNOWN_NODE_PREFIX); | ||
| } | ||
| } else { | ||
| return (_emptyBytesMemory(), ProofError.UNPARSEABLE_NODE); | ||
| } | ||
|
|
||
| // Reset memory before next iteration. Deallocates `decoded` and `path`. | ||
ernestognw marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| Memory.setFreeMemoryPointer(fmp); | ||
| } | ||
|
|
||
| // If we've gone through all proof elements without finding a value, the proof is invalid | ||
| return (_emptyBytesMemory(), ProofError.INVALID_PROOF); | ||
| } | ||
|
|
||
| /** | ||
| * @dev Validates that we've reached a valid leaf value and this is the last proof element. | ||
| * Ensures the value is not empty and no extra proof elements exist. | ||
| */ | ||
| function _validateLastItem( | ||
| Memory.Slice item, | ||
| uint256 trieProofLength, | ||
| uint256 i | ||
| ) private pure returns (bytes memory, ProofError) { | ||
| if (i != trieProofLength - 1) { | ||
| return (_emptyBytesMemory(), ProofError.INVALID_EXTRA_PROOF_ELEMENT); | ||
| } | ||
| bytes memory value = item.readBytes(); | ||
| return (value, value.length == 0 ? ProofError.EMPTY_VALUE : ProofError.NO_ERROR); | ||
| } | ||
|
|
||
| /** | ||
| * @dev Extracts the node ID (hash or raw data based on size) | ||
| * | ||
| * For small nodes (encoded length < 32 bytes) the node ID is the node content itself, | ||
| * For larger nodes, the node ID is the hash of the encoded node data. | ||
| * | ||
| * NOTE: Under normal operation, the input should never be exactly 32-byte inputs. If such an input is provided, | ||
| * it will be used directly, similarly to how small nodes are processed. The following traversal check whether | ||
| * the next node is a large one, and whether its hash matches the raw 32 bytes we have here. If that is the case, | ||
| * the value will be accepted. Otherwise, the next step will return an {INVALID_LARGE_NODE} error. | ||
| */ | ||
| function _getNodeId(Memory.Slice node) private pure returns (bytes32 nodeId, uint256 nodeIdLength) { | ||
| uint256 nodeLength = node.length(); | ||
| return nodeLength < 33 ? (node.load(0), nodeLength) : (node.readBytes32(), 32); | ||
| } | ||
|
|
||
| function _emptyBytesMemory() private pure returns (bytes memory result) { | ||
| assembly ("memory-safe") { | ||
| result := 0x60 // mload(0x60) is always 0 | ||
| } | ||
| } | ||
ernestognw marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.