DA compression for fuel-tx types #670
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
6 tasks
xgreenx
reviewed
Feb 26, 2024
Collaborator
xgreenx
left a comment
xgreenx
left a comment
There was a problem hiding this comment.
Thank you nice PR!=) It is cool that Compact can be derived almost for all types.
I feel like we put too many details about the implementation of the algorithm and how keys are created, and it seems we can simplify that part a lot to be agnostic. But the main with compaction looks good; I will review it one more time, more precisely when PR is in final shape.
BTW, compilation fails for me:
bvrooman
reviewed
Mar 4, 2024
Contributor
bvrooman
left a comment
bvrooman
left a comment
There was a problem hiding this comment.
Reviewed 6/41 so far, will continue review at the next chance I get
Co-authored-by: Brandon Vrooman <[email protected]>
Co-authored-by: Brandon Vrooman <[email protected]>
xgreenx
previously approved these changes
Sep 9, 2024
rafal-ch
reviewed
Sep 9, 2024
Co-authored-by: Rafał Chabowski <[email protected]>
Co-authored-by: Rafał Chabowski <[email protected]>
xgreenx
approved these changes
Sep 10, 2024
This was referenced Sep 11, 2024
Merged
xgreenx
added a commit
to FuelLabs/fuel-core
that referenced
this pull request
Oct 5, 2024
Related #1605. VM PR FuelLabs/fuel-vm#670. This PR adds DA compression crate for Fuel blocks, performed upon block creation. The compressed blocks are stored into the offchain database and can be fetched using the GraphQL API. ## Note for reviewers To keep this reasonably compact, decompression support is not included in this PR, and will be done as a follow-up. As a result, the full data roundtrip testing is not part of this PR. There's no proof here that compression of full blocks is reversible. ## TODO #### Features - [x] Temporal registry db support - [x] Optimize temporal registry eviction implementation - [x] Implement TxId ↔ TxPointer lookups - [x] Integrate with the block committer (GraphQL interface, probably) #### Tests - [x] compressed blocks are available from non-block-producer nodes - [ ] e2e test for the full decompression cycle (moved to a follow-up) ## Follow-up issues - Sync the node from L1: #2208 - Decompression roudntrip tests #2238 - Figure out which cache eviction algorithm/behavior is wanted: #2231 - Figure out if we need to remove the compressed blocks from the db after a while - Merkle roots for fraud proofs #2232 --------- Co-authored-by: Aaryamann Challani <[email protected]> Co-authored-by: Green Baneling <[email protected]> Co-authored-by: Rafał Chabowski <[email protected]>
kuroki-yosuke
added a commit
to kuroki-yosuke/core-fuel
that referenced
this pull request
Apr 5, 2025
Related FuelLabs/fuel-core#1605. VM PR FuelLabs/fuel-vm#670. This PR adds DA compression crate for Fuel blocks, performed upon block creation. The compressed blocks are stored into the offchain database and can be fetched using the GraphQL API. ## Note for reviewers To keep this reasonably compact, decompression support is not included in this PR, and will be done as a follow-up. As a result, the full data roundtrip testing is not part of this PR. There's no proof here that compression of full blocks is reversible. ## TODO #### Features - [x] Temporal registry db support - [x] Optimize temporal registry eviction implementation - [x] Implement TxId ↔ TxPointer lookups - [x] Integrate with the block committer (GraphQL interface, probably) #### Tests - [x] compressed blocks are available from non-block-producer nodes - [ ] e2e test for the full decompression cycle (moved to a follow-up) ## Follow-up issues - Sync the node from L1: FuelLabs/fuel-core#2208 - Decompression roudntrip tests FuelLabs/fuel-core#2238 - Figure out which cache eviction algorithm/behavior is wanted: FuelLabs/fuel-core#2231 - Figure out if we need to remove the compressed blocks from the db after a while - Merkle roots for fraud proofs FuelLabs/fuel-core#2232 --------- Co-authored-by: Aaryamann Challani <[email protected]> Co-authored-by: Green Baneling <[email protected]> Co-authored-by: Rafał Chabowski <[email protected]>
ChapmaBeerbohm
added a commit
to ChapmaBeerbohm/holo-kit
that referenced
this pull request
Sep 26, 2025
Related FuelLabs/fuel-core#1605. VM PR FuelLabs/fuel-vm#670. This PR adds DA compression crate for Fuel blocks, performed upon block creation. The compressed blocks are stored into the offchain database and can be fetched using the GraphQL API. ## Note for reviewers To keep this reasonably compact, decompression support is not included in this PR, and will be done as a follow-up. As a result, the full data roundtrip testing is not part of this PR. There's no proof here that compression of full blocks is reversible. ## TODO #### Features - [x] Temporal registry db support - [x] Optimize temporal registry eviction implementation - [x] Implement TxId ↔ TxPointer lookups - [x] Integrate with the block committer (GraphQL interface, probably) #### Tests - [x] compressed blocks are available from non-block-producer nodes - [ ] e2e test for the full decompression cycle (moved to a follow-up) ## Follow-up issues - Sync the node from L1: FuelLabs/fuel-core#2208 - Decompression roudntrip tests FuelLabs/fuel-core#2238 - Figure out which cache eviction algorithm/behavior is wanted: FuelLabs/fuel-core#2231 - Figure out if we need to remove the compressed blocks from the db after a while - Merkle roots for fraud proofs FuelLabs/fuel-core#2232 --------- Co-authored-by: Aaryamann Challani <[email protected]> Co-authored-by: Green Baneling <[email protected]> Co-authored-by: Rafał Chabowski <[email protected]>
GeorgeBake
added a commit
to GeorgeBake/pipeline-cli
that referenced
this pull request
Sep 29, 2025
Related FuelLabs/fuel-core#1605. VM PR FuelLabs/fuel-vm#670. This PR adds DA compression crate for Fuel blocks, performed upon block creation. The compressed blocks are stored into the offchain database and can be fetched using the GraphQL API. ## Note for reviewers To keep this reasonably compact, decompression support is not included in this PR, and will be done as a follow-up. As a result, the full data roundtrip testing is not part of this PR. There's no proof here that compression of full blocks is reversible. ## TODO #### Features - [x] Temporal registry db support - [x] Optimize temporal registry eviction implementation - [x] Implement TxId ↔ TxPointer lookups - [x] Integrate with the block committer (GraphQL interface, probably) #### Tests - [x] compressed blocks are available from non-block-producer nodes - [ ] e2e test for the full decompression cycle (moved to a follow-up) ## Follow-up issues - Sync the node from L1: FuelLabs/fuel-core#2208 - Decompression roudntrip tests FuelLabs/fuel-core#2238 - Figure out which cache eviction algorithm/behavior is wanted: FuelLabs/fuel-core#2231 - Figure out if we need to remove the compressed blocks from the db after a while - Merkle roots for fraud proofs FuelLabs/fuel-core#2232 --------- Co-authored-by: Aaryamann Challani <[email protected]> Co-authored-by: Green Baneling <[email protected]> Co-authored-by: Rafał Chabowski <[email protected]>
BugSeeker84
added a commit
to BugSeeker84/fuel-core
that referenced
this pull request
Nov 16, 2025
Related FuelLabs/fuel-core#1605. VM PR FuelLabs/fuel-vm#670. This PR adds DA compression crate for Fuel blocks, performed upon block creation. The compressed blocks are stored into the offchain database and can be fetched using the GraphQL API. ## Note for reviewers To keep this reasonably compact, decompression support is not included in this PR, and will be done as a follow-up. As a result, the full data roundtrip testing is not part of this PR. There's no proof here that compression of full blocks is reversible. ## TODO #### Features - [x] Temporal registry db support - [x] Optimize temporal registry eviction implementation - [x] Implement TxId ↔ TxPointer lookups - [x] Integrate with the block committer (GraphQL interface, probably) #### Tests - [x] compressed blocks are available from non-block-producer nodes - [ ] e2e test for the full decompression cycle (moved to a follow-up) ## Follow-up issues - Sync the node from L1: FuelLabs/fuel-core#2208 - Decompression roudntrip tests FuelLabs/fuel-core#2238 - Figure out which cache eviction algorithm/behavior is wanted: FuelLabs/fuel-core#2231 - Figure out if we need to remove the compressed blocks from the db after a while - Merkle roots for fraud proofs FuelLabs/fuel-core#2232 --------- Co-authored-by: Aaryamann Challani <[email protected]> Co-authored-by: Green Baneling <[email protected]> Co-authored-by: Rafał Chabowski <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of FuelLabs/fuel-core#1605. See the corresponding fuel-core PR as well: FuelLabs/fuel-core#1609
This PR adds
Compactderive macro andfuel-compressionmachinery, that are used to convertTransactionand contained types intoCompressedTransactionandCompressed*. The compact types are equivalent to the original types, except that malleable fields are removed, and often-repeated types likeAssetIdandAddressare converted to three-byte keys to a separate database, called temporal storage.UtxoIds (which are used only once) are converted to useTxPointerinstead.