-
Notifications
You must be signed in to change notification settings - Fork 2.9k
fault_proving(merklized_registry): add merklized registry tables #2719
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
3e61ebc to
18ecbea
Compare
| let mut init_structured_storage = init_storage.write_transaction(); | ||
|
|
||
| let mut rng = &mut StdRng::seed_from_u64(1234); | ||
| let mut rng = &mut StdRng::seed_from_u64(31337); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this change is needed because with 1234 as the seed, it generates conflicting keys :/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's interesting. Well I guess the three-byte keys result in a pretty high collision risk. Still fun that we've encountered it 😅
This makes me wonder if there's any attack vector where a block producer overrides certain compression keys. Spontaneously it doesn't sound like you would be able to do anything dangerous in that case but it could be worth thinking about.
|
do we need to merklize |
netrome
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me. Some questions and comments, but nothing blocking.
| let mut init_structured_storage = init_storage.write_transaction(); | ||
|
|
||
| let mut rng = &mut StdRng::seed_from_u64(1234); | ||
| let mut rng = &mut StdRng::seed_from_u64(31337); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's interesting. Well I guess the three-byte keys result in a pretty high collision risk. Still fun that we've encountered it 😅
This makes me wonder if there's any attack vector where a block producer overrides certain compression keys. Spontaneously it doesn't sound like you would be able to do anything dangerous in that case but it could be worth thinking about.
| AssetsInfo = 25, | ||
| /// Index of the coins that are available to spend. | ||
| CoinsToSpend = 26, | ||
| /// See [`DaCompressionTemporalRegistryAddressV2`](da_compression::v2::DaCompressionTemporalRegistryAddressV2) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So we have merkleized tables for Address, AssetId, ContractId, ScriptCode and PredicateCode. Why don't we need to merkleize the RegistryIndex, Timestamps and EvictorCache tables? Are those irrelevant during decompression?
846282b to
a9767a6
Compare
netrome
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice stuff!
a9767a6 to
684089f
Compare
MitchTurner
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Just the one comment on naming.
| pub mod timestamps; | ||
|
|
||
| #[cfg(feature = "fault-proving")] | ||
| pub mod v2; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be named something more specific to the feature. I'm not sure I understand what v2 means if it's always going to be hidden behind the fault-proving flag (I assume we are always going to keep it behind the flag because only the fault-proving nodes are going to want to track the extra state).
Perhaps we name it fault-proving-extra-storage or something specific like that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it won't always be behind the fault-proving flag :) we plan to remove the flag once the feature is done, but you're right, da-compression is enabled only on a few nodes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
addressed in 4560027
I like the previous naming better though :/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm willing to hear you out. If the flag is getting removed in the future, then maybe versioning like that makes more sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh okay, will revert the commit with the name change then :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
reverted in e54635b
This reverts commit 4560027.
## Version 0.42.0 ### Breaking - [2648](#2648): Add feature-flagged field to block header `fault_proving_header` that contains a commitment to all transaction ids. - [2678](#2678): Removed public accessors for `BlockHeader` fields and replaced with methods instead, moved `tx_id_commitment` to the application header of `BlockHeaderV2`. - [2746](#2746): Breaking changes to the CLI arguments: - To disable `random-walk` just don't specify it. Before it required to use `--random-walk 0`. - Next CLI arguments were renamed: - `relayer-min-duration-s` -> `relayer-min-duration` - `relayer-eth-sync-call-freq-s` -> `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq-s` -> `relayer-eth-sync-log-freq` - Default value for the `heartbeat-idle-duration` was changed from `1s` to `100ms`. So information about new block will be propagated faster by default. - All CLI arguments below use time(like `100ms`, `1s`, `1d`, etc.) use as a flag argument instead of number of seconds: - `random-walk` - `connection-idle-timeout` - `info-interval` - `identify-interval` - `request-timeout` - `connection-keep-alive` - `heartbeat-send-duration` - `heartbeat-idle-duration` - `heartbeat-check-interval` - `heartbeat-max-avg-interval` - `heartbeat-max-time-since-last` - `relayer-min-duration` - `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq` - [2840](#2840): CLI argument `vm-backtrace` is deprecated and does nothing. It will be removed in a future version of `fuel-core`. The `extra_tx_checks` field was renamed into `forbid_fake_coins` that affects JSON based serialization/deserialization. Renamed `extra_tx_checks_default` field into `forbid_fake_coins_default`. ### Added - [2150](#2150): Upgraded `libp2p` to `0.54.1` and introduced `ConnectionLimiter` to limit pending incoming/outgoing connections. - [2491](#2491): Storage read replays of historical blocks for execution tracing. Only available behind `--historical-execution` flag. - [2619](#2619): Add possibility to submit list of changes to rocksdb. - [2666](#2666): Added two new CLI arguments to control the GraphQL queries consistency: `--graphql-block-height-tolerance` (default: `10`) and `--graphql-block-height-min-timeout` (default: `30s`). If a request requires a specific block height and the node is slightly behind, it will wait instead of failing. - [2682](#2682): Added GraphQL APIs to get contract storage and balances for current and past blocks. - [2719](#2719): Merklized DA compression temporal registry tables. - [2722](#2722): Service definition for state root service. - [2724](#2724): Explicit error type for merkleized storage. - [2726](#2726): Add a new gossip-sub message for transaction preconfirmations - [2731](#2731): Include `TemporalRegistry` trait implementations for v2 tables. - [2733](#2733): Add a pending pool transaction that allow transaction to wait a bit of time if an input is missing instead of direct delete. - [2742](#2742): Added API crate for merkle root service. - [2756](#2756): Add new service for managing pre-confirmations - [2769](#2769): Added a new `assembleTx` GraphQL endpoint. The endpoint can be used to assemble the transaction based on the provided requirements. - The returned transaction contains: - Input coins to cover `required_balances` - Input coins to cover the fee of the transaction based on the gas price from `block_horizon` - `Change` or `Destroy` outputs for all assets from the inputs - `Variable` outputs in the case they are required during the execution - `Contract` inputs and outputs in the case they are required during the execution - Reserved witness slots for signed coins filled with `64` zeroes - Set script gas limit(unless `script` is empty) - Estimated predicates, if `estimate_predicates == true` - Returns an error if: - The number of required balances exceeds the maximum number of inputs allowed. - The fee address index is out of bounds. - The same asset has multiple change policies(either the receiver of the change is different, or one of the policies states about the destruction of the token while the other does not). The `Change` output from the transaction also count as a `ChangePolicy`. - The number of excluded coin IDs exceeds the maximum number of inputs allowed. - Required assets have multiple entries. - If accounts don't have sufficient amounts to cover the transaction requirements in assets. - If a constructed transaction breaks the rules defined by consensus parameters. - [2780](#2780): Add implementations for the pre-confirmation signing task - [2784](#2784): Integrate the pre conf signature task into the main consensus task - [2788](#2788): Scaffold dedicated compression service. - [2799](#2799): Add a transaction waiter to the executor to wait for potential new transactions inside the block production window. Add a channel to send preconfirmation created by executor to the other modules Added a new CLI arguments: - `--production-timeout` to control the block production timeout in the case if block producer stuck. - `--poa-open-period` set the block production mode to `Open`. The `Open` mode starts the production of the next block immediately after the previous block. The block is open until the `period` passed. The period is a duration represented by `100ms`, `1s`, `1m`, etc. The manual block production is disabled if this production mode is used. - [2802](#2802): Add a new cache with outputs extracted from the pool for the duration of the block. - [2824](#2824): Introduce new `Try`-like methods for the `TaskNextAction` - [2840](#2840): Added a new CLI arguments: - `assemble-tx-dry-run-limit` - The max number how many times script can be executed during `assemble_tx` GraphQL request. Default value is `3` times. - `assemble-tx-estimate-predicates-limit` - The max number how many times predicates can be estimated during `assemble_tx` GraphQL request. Default values is `10` times. - [2841](#2841): Following endpoints allow estimation of predicates on submission of the transaction via new `estimatePredicates` argument: - `submit` - `submit_and_await` - `submit_and_await_status` The change is backward compatible with all SDKs. The change is not forward-compatible with Rust SDK in the case of the `estiamte_predicates` flag set. - [2844](#2844): Implement DA compression in `fuel-core-compression-service`. - [2845](#2845): New status to manage the pre confirmation status send in `TxUpdateSender`. - [2855](#2855): Add an expiration interval check for pending pool and refactor extracted_outputs to not rely on block creation/process sequence. - [2856](#2856): Add generic logic for managing the signatures and delegate keys for pre-confirmations signatures - [2862](#2862): Derive `enum_iterator::Sequence` and `strum_macros::{EnumCount, IntoStaticStr}` for MerkleizedColumn. ### Changed - [2388](#2388): Rework the P2P service codecs to avoid unnecessary coupling between components. The refactoring makes it explicit that the Gossipsub and RequestResponse codecs only share encoding/decoding functionalities from the Postcard codec. It also makes handling Gossipsub and RequestResponse messages completely independent of each other. - [2460](#2460): The type of the `max_response_size`for the postcard codec used in `RequestResponse` protocols has been changed from `usize` to `u64`. - [2473](#2473): Graphql requests and responses make use of a new `extensions` object to specify request/response metadata. A request `extensions` object can contain an integer-valued `required_fuel_block_height` field. When specified, the request will return an error unless the node's current fuel block height is at least the value specified in the `required_fuel_block_height` field. All graphql responses now contain an integer-valued `current_fuel_block_height` field in the `extensions` object, which contains the block height of the last block processed by the node. - [2618](#2618): Parallelize block/transaction changes creation in Importer - [2653](#2653): Added cleaner error for wasm-executor upon failed deserialization. - [2656](#2656): Migrate test helper function `create_contract` to `fuel_core_types::test_helpers::create_contract`, and refactor test in proof_system/global_merkle_root crate to use this function. - [2659](#2659): Replace `derivative` crate with `educe` crate. - [2705](#2705): Update the default value for `--max-block-size` and `--max-transmit-size` to 50 MB - [2715](#2715): Each GraphQL response contains `current_consensus_parameters_version` and `current_stf_version` in the `extensions` section. - [2723](#2723): Change the way we are building the changelog to avoids conflicts. - [2725](#2725): New txpool worker to remove lock contention - [2752](#2752): Extended the `TransactionStatus` to support pre-confirmations. - [2761](#2761): Renamed `ConsensusParametersProvider` to `ChainStateInfoProvider` because it is now providing more than just info about consensus parameters. - [2767](#2767): Updated fuel-vm to v0.60.0, see [release notes](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.60.0). - [2781](#2781): Deprecate `dryRun` mutation. Use `dryRun` query instead. - [2791](#2791): Added `TxStatusManager` service which serves as a single source of truth regarding the current statuses of transactions - [2793](#2793): Moved common merkle storage trait implementations to `fuel-core-storage` and made it easier to setup a set of columns that need merkleization. - [2799](#2799): Change the block production to not be trigger after an interval but directly after the creation of last block and let the executor run for the block time window. - [2800](#2800): Implement P2P adapter for preconfirmation broadcasting - [2802](#2802): Change new txs notifier to be notified only on executable transactions - [2811](#2811): When the state rewind window of 7d was triggered, the `is_migration_in_progress` was repeatedly called, resulting in multiple iterations over the empty ModificationsHistoryV1 table. Iteration was slow because compaction didn't have a chance to clean up V1 table. We removed iteration from the migration process. - [2824](#2824): Improve conditions where `Error`s in the `PreConfirmationSignatureTask` stop the service - [2840](#2840): Removed `log_backtrace` logic from the executor. It is not needed anymore with the existence of the local debugger for the transactions. - [2865](#2865): Consider the following transaction statuses as final: `Success`, `Failure`, `SqueezedOut`, `PreConfirmationSqueezedOut`. All other statuses will be considered transient. ### Fixed - [2646](#2646): Improved performance of fetching block height by caching it when the view is created. - [2682](#2682): Fixed the issue with RPC consistency feature for the subscriptions(without the fix first we perform the logic of the query, and only after verify the required height). - [2730](#2730): Fixed RocksDB closing issue that potentially could panic. - [2743](#2743): Allow discovery of the peers when slots for functional connections are consumed. Reserved nodes are not affected by the limitation on connections anymore. - [2746](#2746): Fixed flaky part in the e2e tests and version compatibility tests. Speed up compatibility tests execution time. Decreased the default time between block height propagation throw the network. - [2758](#2758): Made `tx_id_commitment` feature flagged in `fuel-core-client`. - [2832](#2832): - Trigger block production only when all other sub services are started. - Fix relayer syncing issue causing block production to slow down occasionally. - [2840](#2840): Fixed `fuel-core-client` receipt deserialization in the case if the `ContractId` is zero. ### Removed - [2863](#2863): Removed everything related to the state root service, as it has been moved to another repo. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: AurelienFT <[email protected]>
## Version 0.42.0 ### Breaking - [2648](FuelLabs/fuel-core#2648): Add feature-flagged field to block header `fault_proving_header` that contains a commitment to all transaction ids. - [2678](FuelLabs/fuel-core#2678): Removed public accessors for `BlockHeader` fields and replaced with methods instead, moved `tx_id_commitment` to the application header of `BlockHeaderV2`. - [2746](FuelLabs/fuel-core#2746): Breaking changes to the CLI arguments: - To disable `random-walk` just don't specify it. Before it required to use `--random-walk 0`. - Next CLI arguments were renamed: - `relayer-min-duration-s` -> `relayer-min-duration` - `relayer-eth-sync-call-freq-s` -> `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq-s` -> `relayer-eth-sync-log-freq` - Default value for the `heartbeat-idle-duration` was changed from `1s` to `100ms`. So information about new block will be propagated faster by default. - All CLI arguments below use time(like `100ms`, `1s`, `1d`, etc.) use as a flag argument instead of number of seconds: - `random-walk` - `connection-idle-timeout` - `info-interval` - `identify-interval` - `request-timeout` - `connection-keep-alive` - `heartbeat-send-duration` - `heartbeat-idle-duration` - `heartbeat-check-interval` - `heartbeat-max-avg-interval` - `heartbeat-max-time-since-last` - `relayer-min-duration` - `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq` - [2840](FuelLabs/fuel-core#2840): CLI argument `vm-backtrace` is deprecated and does nothing. It will be removed in a future version of `fuel-core`. The `extra_tx_checks` field was renamed into `forbid_fake_coins` that affects JSON based serialization/deserialization. Renamed `extra_tx_checks_default` field into `forbid_fake_coins_default`. ### Added - [2150](FuelLabs/fuel-core#2150): Upgraded `libp2p` to `0.54.1` and introduced `ConnectionLimiter` to limit pending incoming/outgoing connections. - [2491](FuelLabs/fuel-core#2491): Storage read replays of historical blocks for execution tracing. Only available behind `--historical-execution` flag. - [2619](FuelLabs/fuel-core#2619): Add possibility to submit list of changes to rocksdb. - [2666](FuelLabs/fuel-core#2666): Added two new CLI arguments to control the GraphQL queries consistency: `--graphql-block-height-tolerance` (default: `10`) and `--graphql-block-height-min-timeout` (default: `30s`). If a request requires a specific block height and the node is slightly behind, it will wait instead of failing. - [2682](FuelLabs/fuel-core#2682): Added GraphQL APIs to get contract storage and balances for current and past blocks. - [2719](FuelLabs/fuel-core#2719): Merklized DA compression temporal registry tables. - [2722](FuelLabs/fuel-core#2722): Service definition for state root service. - [2724](FuelLabs/fuel-core#2724): Explicit error type for merkleized storage. - [2726](FuelLabs/fuel-core#2726): Add a new gossip-sub message for transaction preconfirmations - [2731](FuelLabs/fuel-core#2731): Include `TemporalRegistry` trait implementations for v2 tables. - [2733](FuelLabs/fuel-core#2733): Add a pending pool transaction that allow transaction to wait a bit of time if an input is missing instead of direct delete. - [2742](FuelLabs/fuel-core#2742): Added API crate for merkle root service. - [2756](FuelLabs/fuel-core#2756): Add new service for managing pre-confirmations - [2769](FuelLabs/fuel-core#2769): Added a new `assembleTx` GraphQL endpoint. The endpoint can be used to assemble the transaction based on the provided requirements. - The returned transaction contains: - Input coins to cover `required_balances` - Input coins to cover the fee of the transaction based on the gas price from `block_horizon` - `Change` or `Destroy` outputs for all assets from the inputs - `Variable` outputs in the case they are required during the execution - `Contract` inputs and outputs in the case they are required during the execution - Reserved witness slots for signed coins filled with `64` zeroes - Set script gas limit(unless `script` is empty) - Estimated predicates, if `estimate_predicates == true` - Returns an error if: - The number of required balances exceeds the maximum number of inputs allowed. - The fee address index is out of bounds. - The same asset has multiple change policies(either the receiver of the change is different, or one of the policies states about the destruction of the token while the other does not). The `Change` output from the transaction also count as a `ChangePolicy`. - The number of excluded coin IDs exceeds the maximum number of inputs allowed. - Required assets have multiple entries. - If accounts don't have sufficient amounts to cover the transaction requirements in assets. - If a constructed transaction breaks the rules defined by consensus parameters. - [2780](FuelLabs/fuel-core#2780): Add implementations for the pre-confirmation signing task - [2784](FuelLabs/fuel-core#2784): Integrate the pre conf signature task into the main consensus task - [2788](FuelLabs/fuel-core#2788): Scaffold dedicated compression service. - [2799](FuelLabs/fuel-core#2799): Add a transaction waiter to the executor to wait for potential new transactions inside the block production window. Add a channel to send preconfirmation created by executor to the other modules Added a new CLI arguments: - `--production-timeout` to control the block production timeout in the case if block producer stuck. - `--poa-open-period` set the block production mode to `Open`. The `Open` mode starts the production of the next block immediately after the previous block. The block is open until the `period` passed. The period is a duration represented by `100ms`, `1s`, `1m`, etc. The manual block production is disabled if this production mode is used. - [2802](FuelLabs/fuel-core#2802): Add a new cache with outputs extracted from the pool for the duration of the block. - [2824](FuelLabs/fuel-core#2824): Introduce new `Try`-like methods for the `TaskNextAction` - [2840](FuelLabs/fuel-core#2840): Added a new CLI arguments: - `assemble-tx-dry-run-limit` - The max number how many times script can be executed during `assemble_tx` GraphQL request. Default value is `3` times. - `assemble-tx-estimate-predicates-limit` - The max number how many times predicates can be estimated during `assemble_tx` GraphQL request. Default values is `10` times. - [2841](FuelLabs/fuel-core#2841): Following endpoints allow estimation of predicates on submission of the transaction via new `estimatePredicates` argument: - `submit` - `submit_and_await` - `submit_and_await_status` The change is backward compatible with all SDKs. The change is not forward-compatible with Rust SDK in the case of the `estiamte_predicates` flag set. - [2844](FuelLabs/fuel-core#2844): Implement DA compression in `fuel-core-compression-service`. - [2845](FuelLabs/fuel-core#2845): New status to manage the pre confirmation status send in `TxUpdateSender`. - [2855](FuelLabs/fuel-core#2855): Add an expiration interval check for pending pool and refactor extracted_outputs to not rely on block creation/process sequence. - [2856](FuelLabs/fuel-core#2856): Add generic logic for managing the signatures and delegate keys for pre-confirmations signatures - [2862](FuelLabs/fuel-core#2862): Derive `enum_iterator::Sequence` and `strum_macros::{EnumCount, IntoStaticStr}` for MerkleizedColumn. ### Changed - [2388](FuelLabs/fuel-core#2388): Rework the P2P service codecs to avoid unnecessary coupling between components. The refactoring makes it explicit that the Gossipsub and RequestResponse codecs only share encoding/decoding functionalities from the Postcard codec. It also makes handling Gossipsub and RequestResponse messages completely independent of each other. - [2460](FuelLabs/fuel-core#2460): The type of the `max_response_size`for the postcard codec used in `RequestResponse` protocols has been changed from `usize` to `u64`. - [2473](FuelLabs/fuel-core#2473): Graphql requests and responses make use of a new `extensions` object to specify request/response metadata. A request `extensions` object can contain an integer-valued `required_fuel_block_height` field. When specified, the request will return an error unless the node's current fuel block height is at least the value specified in the `required_fuel_block_height` field. All graphql responses now contain an integer-valued `current_fuel_block_height` field in the `extensions` object, which contains the block height of the last block processed by the node. - [2618](FuelLabs/fuel-core#2618): Parallelize block/transaction changes creation in Importer - [2653](FuelLabs/fuel-core#2653): Added cleaner error for wasm-executor upon failed deserialization. - [2656](FuelLabs/fuel-core#2656): Migrate test helper function `create_contract` to `fuel_core_types::test_helpers::create_contract`, and refactor test in proof_system/global_merkle_root crate to use this function. - [2659](FuelLabs/fuel-core#2659): Replace `derivative` crate with `educe` crate. - [2705](FuelLabs/fuel-core#2705): Update the default value for `--max-block-size` and `--max-transmit-size` to 50 MB - [2715](FuelLabs/fuel-core#2715): Each GraphQL response contains `current_consensus_parameters_version` and `current_stf_version` in the `extensions` section. - [2723](FuelLabs/fuel-core#2723): Change the way we are building the changelog to avoids conflicts. - [2725](FuelLabs/fuel-core#2725): New txpool worker to remove lock contention - [2752](FuelLabs/fuel-core#2752): Extended the `TransactionStatus` to support pre-confirmations. - [2761](FuelLabs/fuel-core#2761): Renamed `ConsensusParametersProvider` to `ChainStateInfoProvider` because it is now providing more than just info about consensus parameters. - [2767](FuelLabs/fuel-core#2767): Updated fuel-vm to v0.60.0, see [release notes](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.60.0). - [2781](FuelLabs/fuel-core#2781): Deprecate `dryRun` mutation. Use `dryRun` query instead. - [2791](FuelLabs/fuel-core#2791): Added `TxStatusManager` service which serves as a single source of truth regarding the current statuses of transactions - [2793](FuelLabs/fuel-core#2793): Moved common merkle storage trait implementations to `fuel-core-storage` and made it easier to setup a set of columns that need merkleization. - [2799](FuelLabs/fuel-core#2799): Change the block production to not be trigger after an interval but directly after the creation of last block and let the executor run for the block time window. - [2800](FuelLabs/fuel-core#2800): Implement P2P adapter for preconfirmation broadcasting - [2802](FuelLabs/fuel-core#2802): Change new txs notifier to be notified only on executable transactions - [2811](FuelLabs/fuel-core#2811): When the state rewind window of 7d was triggered, the `is_migration_in_progress` was repeatedly called, resulting in multiple iterations over the empty ModificationsHistoryV1 table. Iteration was slow because compaction didn't have a chance to clean up V1 table. We removed iteration from the migration process. - [2824](FuelLabs/fuel-core#2824): Improve conditions where `Error`s in the `PreConfirmationSignatureTask` stop the service - [2840](FuelLabs/fuel-core#2840): Removed `log_backtrace` logic from the executor. It is not needed anymore with the existence of the local debugger for the transactions. - [2865](FuelLabs/fuel-core#2865): Consider the following transaction statuses as final: `Success`, `Failure`, `SqueezedOut`, `PreConfirmationSqueezedOut`. All other statuses will be considered transient. ### Fixed - [2646](FuelLabs/fuel-core#2646): Improved performance of fetching block height by caching it when the view is created. - [2682](FuelLabs/fuel-core#2682): Fixed the issue with RPC consistency feature for the subscriptions(without the fix first we perform the logic of the query, and only after verify the required height). - [2730](FuelLabs/fuel-core#2730): Fixed RocksDB closing issue that potentially could panic. - [2743](FuelLabs/fuel-core#2743): Allow discovery of the peers when slots for functional connections are consumed. Reserved nodes are not affected by the limitation on connections anymore. - [2746](FuelLabs/fuel-core#2746): Fixed flaky part in the e2e tests and version compatibility tests. Speed up compatibility tests execution time. Decreased the default time between block height propagation throw the network. - [2758](FuelLabs/fuel-core#2758): Made `tx_id_commitment` feature flagged in `fuel-core-client`. - [2832](FuelLabs/fuel-core#2832): - Trigger block production only when all other sub services are started. - Fix relayer syncing issue causing block production to slow down occasionally. - [2840](FuelLabs/fuel-core#2840): Fixed `fuel-core-client` receipt deserialization in the case if the `ContractId` is zero. ### Removed - [2863](FuelLabs/fuel-core#2863): Removed everything related to the state root service, as it has been moved to another repo. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: AurelienFT <[email protected]>
## Version 0.42.0 ### Breaking - [2648](FuelLabs/fuel-core#2648): Add feature-flagged field to block header `fault_proving_header` that contains a commitment to all transaction ids. - [2678](FuelLabs/fuel-core#2678): Removed public accessors for `BlockHeader` fields and replaced with methods instead, moved `tx_id_commitment` to the application header of `BlockHeaderV2`. - [2746](FuelLabs/fuel-core#2746): Breaking changes to the CLI arguments: - To disable `random-walk` just don't specify it. Before it required to use `--random-walk 0`. - Next CLI arguments were renamed: - `relayer-min-duration-s` -> `relayer-min-duration` - `relayer-eth-sync-call-freq-s` -> `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq-s` -> `relayer-eth-sync-log-freq` - Default value for the `heartbeat-idle-duration` was changed from `1s` to `100ms`. So information about new block will be propagated faster by default. - All CLI arguments below use time(like `100ms`, `1s`, `1d`, etc.) use as a flag argument instead of number of seconds: - `random-walk` - `connection-idle-timeout` - `info-interval` - `identify-interval` - `request-timeout` - `connection-keep-alive` - `heartbeat-send-duration` - `heartbeat-idle-duration` - `heartbeat-check-interval` - `heartbeat-max-avg-interval` - `heartbeat-max-time-since-last` - `relayer-min-duration` - `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq` - [2840](FuelLabs/fuel-core#2840): CLI argument `vm-backtrace` is deprecated and does nothing. It will be removed in a future version of `fuel-core`. The `extra_tx_checks` field was renamed into `forbid_fake_coins` that affects JSON based serialization/deserialization. Renamed `extra_tx_checks_default` field into `forbid_fake_coins_default`. ### Added - [2150](FuelLabs/fuel-core#2150): Upgraded `libp2p` to `0.54.1` and introduced `ConnectionLimiter` to limit pending incoming/outgoing connections. - [2491](FuelLabs/fuel-core#2491): Storage read replays of historical blocks for execution tracing. Only available behind `--historical-execution` flag. - [2619](FuelLabs/fuel-core#2619): Add possibility to submit list of changes to rocksdb. - [2666](FuelLabs/fuel-core#2666): Added two new CLI arguments to control the GraphQL queries consistency: `--graphql-block-height-tolerance` (default: `10`) and `--graphql-block-height-min-timeout` (default: `30s`). If a request requires a specific block height and the node is slightly behind, it will wait instead of failing. - [2682](FuelLabs/fuel-core#2682): Added GraphQL APIs to get contract storage and balances for current and past blocks. - [2719](FuelLabs/fuel-core#2719): Merklized DA compression temporal registry tables. - [2722](FuelLabs/fuel-core#2722): Service definition for state root service. - [2724](FuelLabs/fuel-core#2724): Explicit error type for merkleized storage. - [2726](FuelLabs/fuel-core#2726): Add a new gossip-sub message for transaction preconfirmations - [2731](FuelLabs/fuel-core#2731): Include `TemporalRegistry` trait implementations for v2 tables. - [2733](FuelLabs/fuel-core#2733): Add a pending pool transaction that allow transaction to wait a bit of time if an input is missing instead of direct delete. - [2742](FuelLabs/fuel-core#2742): Added API crate for merkle root service. - [2756](FuelLabs/fuel-core#2756): Add new service for managing pre-confirmations - [2769](FuelLabs/fuel-core#2769): Added a new `assembleTx` GraphQL endpoint. The endpoint can be used to assemble the transaction based on the provided requirements. - The returned transaction contains: - Input coins to cover `required_balances` - Input coins to cover the fee of the transaction based on the gas price from `block_horizon` - `Change` or `Destroy` outputs for all assets from the inputs - `Variable` outputs in the case they are required during the execution - `Contract` inputs and outputs in the case they are required during the execution - Reserved witness slots for signed coins filled with `64` zeroes - Set script gas limit(unless `script` is empty) - Estimated predicates, if `estimate_predicates == true` - Returns an error if: - The number of required balances exceeds the maximum number of inputs allowed. - The fee address index is out of bounds. - The same asset has multiple change policies(either the receiver of the change is different, or one of the policies states about the destruction of the token while the other does not). The `Change` output from the transaction also count as a `ChangePolicy`. - The number of excluded coin IDs exceeds the maximum number of inputs allowed. - Required assets have multiple entries. - If accounts don't have sufficient amounts to cover the transaction requirements in assets. - If a constructed transaction breaks the rules defined by consensus parameters. - [2780](FuelLabs/fuel-core#2780): Add implementations for the pre-confirmation signing task - [2784](FuelLabs/fuel-core#2784): Integrate the pre conf signature task into the main consensus task - [2788](FuelLabs/fuel-core#2788): Scaffold dedicated compression service. - [2799](FuelLabs/fuel-core#2799): Add a transaction waiter to the executor to wait for potential new transactions inside the block production window. Add a channel to send preconfirmation created by executor to the other modules Added a new CLI arguments: - `--production-timeout` to control the block production timeout in the case if block producer stuck. - `--poa-open-period` set the block production mode to `Open`. The `Open` mode starts the production of the next block immediately after the previous block. The block is open until the `period` passed. The period is a duration represented by `100ms`, `1s`, `1m`, etc. The manual block production is disabled if this production mode is used. - [2802](FuelLabs/fuel-core#2802): Add a new cache with outputs extracted from the pool for the duration of the block. - [2824](FuelLabs/fuel-core#2824): Introduce new `Try`-like methods for the `TaskNextAction` - [2840](FuelLabs/fuel-core#2840): Added a new CLI arguments: - `assemble-tx-dry-run-limit` - The max number how many times script can be executed during `assemble_tx` GraphQL request. Default value is `3` times. - `assemble-tx-estimate-predicates-limit` - The max number how many times predicates can be estimated during `assemble_tx` GraphQL request. Default values is `10` times. - [2841](FuelLabs/fuel-core#2841): Following endpoints allow estimation of predicates on submission of the transaction via new `estimatePredicates` argument: - `submit` - `submit_and_await` - `submit_and_await_status` The change is backward compatible with all SDKs. The change is not forward-compatible with Rust SDK in the case of the `estiamte_predicates` flag set. - [2844](FuelLabs/fuel-core#2844): Implement DA compression in `fuel-core-compression-service`. - [2845](FuelLabs/fuel-core#2845): New status to manage the pre confirmation status send in `TxUpdateSender`. - [2855](FuelLabs/fuel-core#2855): Add an expiration interval check for pending pool and refactor extracted_outputs to not rely on block creation/process sequence. - [2856](FuelLabs/fuel-core#2856): Add generic logic for managing the signatures and delegate keys for pre-confirmations signatures - [2862](FuelLabs/fuel-core#2862): Derive `enum_iterator::Sequence` and `strum_macros::{EnumCount, IntoStaticStr}` for MerkleizedColumn. ### Changed - [2388](FuelLabs/fuel-core#2388): Rework the P2P service codecs to avoid unnecessary coupling between components. The refactoring makes it explicit that the Gossipsub and RequestResponse codecs only share encoding/decoding functionalities from the Postcard codec. It also makes handling Gossipsub and RequestResponse messages completely independent of each other. - [2460](FuelLabs/fuel-core#2460): The type of the `max_response_size`for the postcard codec used in `RequestResponse` protocols has been changed from `usize` to `u64`. - [2473](FuelLabs/fuel-core#2473): Graphql requests and responses make use of a new `extensions` object to specify request/response metadata. A request `extensions` object can contain an integer-valued `required_fuel_block_height` field. When specified, the request will return an error unless the node's current fuel block height is at least the value specified in the `required_fuel_block_height` field. All graphql responses now contain an integer-valued `current_fuel_block_height` field in the `extensions` object, which contains the block height of the last block processed by the node. - [2618](FuelLabs/fuel-core#2618): Parallelize block/transaction changes creation in Importer - [2653](FuelLabs/fuel-core#2653): Added cleaner error for wasm-executor upon failed deserialization. - [2656](FuelLabs/fuel-core#2656): Migrate test helper function `create_contract` to `fuel_core_types::test_helpers::create_contract`, and refactor test in proof_system/global_merkle_root crate to use this function. - [2659](FuelLabs/fuel-core#2659): Replace `derivative` crate with `educe` crate. - [2705](FuelLabs/fuel-core#2705): Update the default value for `--max-block-size` and `--max-transmit-size` to 50 MB - [2715](FuelLabs/fuel-core#2715): Each GraphQL response contains `current_consensus_parameters_version` and `current_stf_version` in the `extensions` section. - [2723](FuelLabs/fuel-core#2723): Change the way we are building the changelog to avoids conflicts. - [2725](FuelLabs/fuel-core#2725): New txpool worker to remove lock contention - [2752](FuelLabs/fuel-core#2752): Extended the `TransactionStatus` to support pre-confirmations. - [2761](FuelLabs/fuel-core#2761): Renamed `ConsensusParametersProvider` to `ChainStateInfoProvider` because it is now providing more than just info about consensus parameters. - [2767](FuelLabs/fuel-core#2767): Updated fuel-vm to v0.60.0, see [release notes](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.60.0). - [2781](FuelLabs/fuel-core#2781): Deprecate `dryRun` mutation. Use `dryRun` query instead. - [2791](FuelLabs/fuel-core#2791): Added `TxStatusManager` service which serves as a single source of truth regarding the current statuses of transactions - [2793](FuelLabs/fuel-core#2793): Moved common merkle storage trait implementations to `fuel-core-storage` and made it easier to setup a set of columns that need merkleization. - [2799](FuelLabs/fuel-core#2799): Change the block production to not be trigger after an interval but directly after the creation of last block and let the executor run for the block time window. - [2800](FuelLabs/fuel-core#2800): Implement P2P adapter for preconfirmation broadcasting - [2802](FuelLabs/fuel-core#2802): Change new txs notifier to be notified only on executable transactions - [2811](FuelLabs/fuel-core#2811): When the state rewind window of 7d was triggered, the `is_migration_in_progress` was repeatedly called, resulting in multiple iterations over the empty ModificationsHistoryV1 table. Iteration was slow because compaction didn't have a chance to clean up V1 table. We removed iteration from the migration process. - [2824](FuelLabs/fuel-core#2824): Improve conditions where `Error`s in the `PreConfirmationSignatureTask` stop the service - [2840](FuelLabs/fuel-core#2840): Removed `log_backtrace` logic from the executor. It is not needed anymore with the existence of the local debugger for the transactions. - [2865](FuelLabs/fuel-core#2865): Consider the following transaction statuses as final: `Success`, `Failure`, `SqueezedOut`, `PreConfirmationSqueezedOut`. All other statuses will be considered transient. ### Fixed - [2646](FuelLabs/fuel-core#2646): Improved performance of fetching block height by caching it when the view is created. - [2682](FuelLabs/fuel-core#2682): Fixed the issue with RPC consistency feature for the subscriptions(without the fix first we perform the logic of the query, and only after verify the required height). - [2730](FuelLabs/fuel-core#2730): Fixed RocksDB closing issue that potentially could panic. - [2743](FuelLabs/fuel-core#2743): Allow discovery of the peers when slots for functional connections are consumed. Reserved nodes are not affected by the limitation on connections anymore. - [2746](FuelLabs/fuel-core#2746): Fixed flaky part in the e2e tests and version compatibility tests. Speed up compatibility tests execution time. Decreased the default time between block height propagation throw the network. - [2758](FuelLabs/fuel-core#2758): Made `tx_id_commitment` feature flagged in `fuel-core-client`. - [2832](FuelLabs/fuel-core#2832): - Trigger block production only when all other sub services are started. - Fix relayer syncing issue causing block production to slow down occasionally. - [2840](FuelLabs/fuel-core#2840): Fixed `fuel-core-client` receipt deserialization in the case if the `ContractId` is zero. ### Removed - [2863](FuelLabs/fuel-core#2863): Removed everything related to the state root service, as it has been moved to another repo. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: AurelienFT <[email protected]>
## Version 0.42.0 ### Breaking - [2648](FuelLabs/fuel-core#2648): Add feature-flagged field to block header `fault_proving_header` that contains a commitment to all transaction ids. - [2678](FuelLabs/fuel-core#2678): Removed public accessors for `BlockHeader` fields and replaced with methods instead, moved `tx_id_commitment` to the application header of `BlockHeaderV2`. - [2746](FuelLabs/fuel-core#2746): Breaking changes to the CLI arguments: - To disable `random-walk` just don't specify it. Before it required to use `--random-walk 0`. - Next CLI arguments were renamed: - `relayer-min-duration-s` -> `relayer-min-duration` - `relayer-eth-sync-call-freq-s` -> `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq-s` -> `relayer-eth-sync-log-freq` - Default value for the `heartbeat-idle-duration` was changed from `1s` to `100ms`. So information about new block will be propagated faster by default. - All CLI arguments below use time(like `100ms`, `1s`, `1d`, etc.) use as a flag argument instead of number of seconds: - `random-walk` - `connection-idle-timeout` - `info-interval` - `identify-interval` - `request-timeout` - `connection-keep-alive` - `heartbeat-send-duration` - `heartbeat-idle-duration` - `heartbeat-check-interval` - `heartbeat-max-avg-interval` - `heartbeat-max-time-since-last` - `relayer-min-duration` - `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq` - [2840](FuelLabs/fuel-core#2840): CLI argument `vm-backtrace` is deprecated and does nothing. It will be removed in a future version of `fuel-core`. The `extra_tx_checks` field was renamed into `forbid_fake_coins` that affects JSON based serialization/deserialization. Renamed `extra_tx_checks_default` field into `forbid_fake_coins_default`. ### Added - [2150](FuelLabs/fuel-core#2150): Upgraded `libp2p` to `0.54.1` and introduced `ConnectionLimiter` to limit pending incoming/outgoing connections. - [2491](FuelLabs/fuel-core#2491): Storage read replays of historical blocks for execution tracing. Only available behind `--historical-execution` flag. - [2619](FuelLabs/fuel-core#2619): Add possibility to submit list of changes to rocksdb. - [2666](FuelLabs/fuel-core#2666): Added two new CLI arguments to control the GraphQL queries consistency: `--graphql-block-height-tolerance` (default: `10`) and `--graphql-block-height-min-timeout` (default: `30s`). If a request requires a specific block height and the node is slightly behind, it will wait instead of failing. - [2682](FuelLabs/fuel-core#2682): Added GraphQL APIs to get contract storage and balances for current and past blocks. - [2719](FuelLabs/fuel-core#2719): Merklized DA compression temporal registry tables. - [2722](FuelLabs/fuel-core#2722): Service definition for state root service. - [2724](FuelLabs/fuel-core#2724): Explicit error type for merkleized storage. - [2726](FuelLabs/fuel-core#2726): Add a new gossip-sub message for transaction preconfirmations - [2731](FuelLabs/fuel-core#2731): Include `TemporalRegistry` trait implementations for v2 tables. - [2733](FuelLabs/fuel-core#2733): Add a pending pool transaction that allow transaction to wait a bit of time if an input is missing instead of direct delete. - [2742](FuelLabs/fuel-core#2742): Added API crate for merkle root service. - [2756](FuelLabs/fuel-core#2756): Add new service for managing pre-confirmations - [2769](FuelLabs/fuel-core#2769): Added a new `assembleTx` GraphQL endpoint. The endpoint can be used to assemble the transaction based on the provided requirements. - The returned transaction contains: - Input coins to cover `required_balances` - Input coins to cover the fee of the transaction based on the gas price from `block_horizon` - `Change` or `Destroy` outputs for all assets from the inputs - `Variable` outputs in the case they are required during the execution - `Contract` inputs and outputs in the case they are required during the execution - Reserved witness slots for signed coins filled with `64` zeroes - Set script gas limit(unless `script` is empty) - Estimated predicates, if `estimate_predicates == true` - Returns an error if: - The number of required balances exceeds the maximum number of inputs allowed. - The fee address index is out of bounds. - The same asset has multiple change policies(either the receiver of the change is different, or one of the policies states about the destruction of the token while the other does not). The `Change` output from the transaction also count as a `ChangePolicy`. - The number of excluded coin IDs exceeds the maximum number of inputs allowed. - Required assets have multiple entries. - If accounts don't have sufficient amounts to cover the transaction requirements in assets. - If a constructed transaction breaks the rules defined by consensus parameters. - [2780](FuelLabs/fuel-core#2780): Add implementations for the pre-confirmation signing task - [2784](FuelLabs/fuel-core#2784): Integrate the pre conf signature task into the main consensus task - [2788](FuelLabs/fuel-core#2788): Scaffold dedicated compression service. - [2799](FuelLabs/fuel-core#2799): Add a transaction waiter to the executor to wait for potential new transactions inside the block production window. Add a channel to send preconfirmation created by executor to the other modules Added a new CLI arguments: - `--production-timeout` to control the block production timeout in the case if block producer stuck. - `--poa-open-period` set the block production mode to `Open`. The `Open` mode starts the production of the next block immediately after the previous block. The block is open until the `period` passed. The period is a duration represented by `100ms`, `1s`, `1m`, etc. The manual block production is disabled if this production mode is used. - [2802](FuelLabs/fuel-core#2802): Add a new cache with outputs extracted from the pool for the duration of the block. - [2824](FuelLabs/fuel-core#2824): Introduce new `Try`-like methods for the `TaskNextAction` - [2840](FuelLabs/fuel-core#2840): Added a new CLI arguments: - `assemble-tx-dry-run-limit` - The max number how many times script can be executed during `assemble_tx` GraphQL request. Default value is `3` times. - `assemble-tx-estimate-predicates-limit` - The max number how many times predicates can be estimated during `assemble_tx` GraphQL request. Default values is `10` times. - [2841](FuelLabs/fuel-core#2841): Following endpoints allow estimation of predicates on submission of the transaction via new `estimatePredicates` argument: - `submit` - `submit_and_await` - `submit_and_await_status` The change is backward compatible with all SDKs. The change is not forward-compatible with Rust SDK in the case of the `estiamte_predicates` flag set. - [2844](FuelLabs/fuel-core#2844): Implement DA compression in `fuel-core-compression-service`. - [2845](FuelLabs/fuel-core#2845): New status to manage the pre confirmation status send in `TxUpdateSender`. - [2855](FuelLabs/fuel-core#2855): Add an expiration interval check for pending pool and refactor extracted_outputs to not rely on block creation/process sequence. - [2856](FuelLabs/fuel-core#2856): Add generic logic for managing the signatures and delegate keys for pre-confirmations signatures - [2862](FuelLabs/fuel-core#2862): Derive `enum_iterator::Sequence` and `strum_macros::{EnumCount, IntoStaticStr}` for MerkleizedColumn. ### Changed - [2388](FuelLabs/fuel-core#2388): Rework the P2P service codecs to avoid unnecessary coupling between components. The refactoring makes it explicit that the Gossipsub and RequestResponse codecs only share encoding/decoding functionalities from the Postcard codec. It also makes handling Gossipsub and RequestResponse messages completely independent of each other. - [2460](FuelLabs/fuel-core#2460): The type of the `max_response_size`for the postcard codec used in `RequestResponse` protocols has been changed from `usize` to `u64`. - [2473](FuelLabs/fuel-core#2473): Graphql requests and responses make use of a new `extensions` object to specify request/response metadata. A request `extensions` object can contain an integer-valued `required_fuel_block_height` field. When specified, the request will return an error unless the node's current fuel block height is at least the value specified in the `required_fuel_block_height` field. All graphql responses now contain an integer-valued `current_fuel_block_height` field in the `extensions` object, which contains the block height of the last block processed by the node. - [2618](FuelLabs/fuel-core#2618): Parallelize block/transaction changes creation in Importer - [2653](FuelLabs/fuel-core#2653): Added cleaner error for wasm-executor upon failed deserialization. - [2656](FuelLabs/fuel-core#2656): Migrate test helper function `create_contract` to `fuel_core_types::test_helpers::create_contract`, and refactor test in proof_system/global_merkle_root crate to use this function. - [2659](FuelLabs/fuel-core#2659): Replace `derivative` crate with `educe` crate. - [2705](FuelLabs/fuel-core#2705): Update the default value for `--max-block-size` and `--max-transmit-size` to 50 MB - [2715](FuelLabs/fuel-core#2715): Each GraphQL response contains `current_consensus_parameters_version` and `current_stf_version` in the `extensions` section. - [2723](FuelLabs/fuel-core#2723): Change the way we are building the changelog to avoids conflicts. - [2725](FuelLabs/fuel-core#2725): New txpool worker to remove lock contention - [2752](FuelLabs/fuel-core#2752): Extended the `TransactionStatus` to support pre-confirmations. - [2761](FuelLabs/fuel-core#2761): Renamed `ConsensusParametersProvider` to `ChainStateInfoProvider` because it is now providing more than just info about consensus parameters. - [2767](FuelLabs/fuel-core#2767): Updated fuel-vm to v0.60.0, see [release notes](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.60.0). - [2781](FuelLabs/fuel-core#2781): Deprecate `dryRun` mutation. Use `dryRun` query instead. - [2791](FuelLabs/fuel-core#2791): Added `TxStatusManager` service which serves as a single source of truth regarding the current statuses of transactions - [2793](FuelLabs/fuel-core#2793): Moved common merkle storage trait implementations to `fuel-core-storage` and made it easier to setup a set of columns that need merkleization. - [2799](FuelLabs/fuel-core#2799): Change the block production to not be trigger after an interval but directly after the creation of last block and let the executor run for the block time window. - [2800](FuelLabs/fuel-core#2800): Implement P2P adapter for preconfirmation broadcasting - [2802](FuelLabs/fuel-core#2802): Change new txs notifier to be notified only on executable transactions - [2811](FuelLabs/fuel-core#2811): When the state rewind window of 7d was triggered, the `is_migration_in_progress` was repeatedly called, resulting in multiple iterations over the empty ModificationsHistoryV1 table. Iteration was slow because compaction didn't have a chance to clean up V1 table. We removed iteration from the migration process. - [2824](FuelLabs/fuel-core#2824): Improve conditions where `Error`s in the `PreConfirmationSignatureTask` stop the service - [2840](FuelLabs/fuel-core#2840): Removed `log_backtrace` logic from the executor. It is not needed anymore with the existence of the local debugger for the transactions. - [2865](FuelLabs/fuel-core#2865): Consider the following transaction statuses as final: `Success`, `Failure`, `SqueezedOut`, `PreConfirmationSqueezedOut`. All other statuses will be considered transient. ### Fixed - [2646](FuelLabs/fuel-core#2646): Improved performance of fetching block height by caching it when the view is created. - [2682](FuelLabs/fuel-core#2682): Fixed the issue with RPC consistency feature for the subscriptions(without the fix first we perform the logic of the query, and only after verify the required height). - [2730](FuelLabs/fuel-core#2730): Fixed RocksDB closing issue that potentially could panic. - [2743](FuelLabs/fuel-core#2743): Allow discovery of the peers when slots for functional connections are consumed. Reserved nodes are not affected by the limitation on connections anymore. - [2746](FuelLabs/fuel-core#2746): Fixed flaky part in the e2e tests and version compatibility tests. Speed up compatibility tests execution time. Decreased the default time between block height propagation throw the network. - [2758](FuelLabs/fuel-core#2758): Made `tx_id_commitment` feature flagged in `fuel-core-client`. - [2832](FuelLabs/fuel-core#2832): - Trigger block production only when all other sub services are started. - Fix relayer syncing issue causing block production to slow down occasionally. - [2840](FuelLabs/fuel-core#2840): Fixed `fuel-core-client` receipt deserialization in the case if the `ContractId` is zero. ### Removed - [2863](FuelLabs/fuel-core#2863): Removed everything related to the state root service, as it has been moved to another repo. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: AurelienFT <[email protected]>
## Version 0.42.0 ### Breaking - [2648](FuelLabs/fuel-core#2648): Add feature-flagged field to block header `fault_proving_header` that contains a commitment to all transaction ids. - [2678](FuelLabs/fuel-core#2678): Removed public accessors for `BlockHeader` fields and replaced with methods instead, moved `tx_id_commitment` to the application header of `BlockHeaderV2`. - [2746](FuelLabs/fuel-core#2746): Breaking changes to the CLI arguments: - To disable `random-walk` just don't specify it. Before it required to use `--random-walk 0`. - Next CLI arguments were renamed: - `relayer-min-duration-s` -> `relayer-min-duration` - `relayer-eth-sync-call-freq-s` -> `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq-s` -> `relayer-eth-sync-log-freq` - Default value for the `heartbeat-idle-duration` was changed from `1s` to `100ms`. So information about new block will be propagated faster by default. - All CLI arguments below use time(like `100ms`, `1s`, `1d`, etc.) use as a flag argument instead of number of seconds: - `random-walk` - `connection-idle-timeout` - `info-interval` - `identify-interval` - `request-timeout` - `connection-keep-alive` - `heartbeat-send-duration` - `heartbeat-idle-duration` - `heartbeat-check-interval` - `heartbeat-max-avg-interval` - `heartbeat-max-time-since-last` - `relayer-min-duration` - `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq` - [2840](FuelLabs/fuel-core#2840): CLI argument `vm-backtrace` is deprecated and does nothing. It will be removed in a future version of `fuel-core`. The `extra_tx_checks` field was renamed into `forbid_fake_coins` that affects JSON based serialization/deserialization. Renamed `extra_tx_checks_default` field into `forbid_fake_coins_default`. ### Added - [2150](FuelLabs/fuel-core#2150): Upgraded `libp2p` to `0.54.1` and introduced `ConnectionLimiter` to limit pending incoming/outgoing connections. - [2491](FuelLabs/fuel-core#2491): Storage read replays of historical blocks for execution tracing. Only available behind `--historical-execution` flag. - [2619](FuelLabs/fuel-core#2619): Add possibility to submit list of changes to rocksdb. - [2666](FuelLabs/fuel-core#2666): Added two new CLI arguments to control the GraphQL queries consistency: `--graphql-block-height-tolerance` (default: `10`) and `--graphql-block-height-min-timeout` (default: `30s`). If a request requires a specific block height and the node is slightly behind, it will wait instead of failing. - [2682](FuelLabs/fuel-core#2682): Added GraphQL APIs to get contract storage and balances for current and past blocks. - [2719](FuelLabs/fuel-core#2719): Merklized DA compression temporal registry tables. - [2722](FuelLabs/fuel-core#2722): Service definition for state root service. - [2724](FuelLabs/fuel-core#2724): Explicit error type for merkleized storage. - [2726](FuelLabs/fuel-core#2726): Add a new gossip-sub message for transaction preconfirmations - [2731](FuelLabs/fuel-core#2731): Include `TemporalRegistry` trait implementations for v2 tables. - [2733](FuelLabs/fuel-core#2733): Add a pending pool transaction that allow transaction to wait a bit of time if an input is missing instead of direct delete. - [2742](FuelLabs/fuel-core#2742): Added API crate for merkle root service. - [2756](FuelLabs/fuel-core#2756): Add new service for managing pre-confirmations - [2769](FuelLabs/fuel-core#2769): Added a new `assembleTx` GraphQL endpoint. The endpoint can be used to assemble the transaction based on the provided requirements. - The returned transaction contains: - Input coins to cover `required_balances` - Input coins to cover the fee of the transaction based on the gas price from `block_horizon` - `Change` or `Destroy` outputs for all assets from the inputs - `Variable` outputs in the case they are required during the execution - `Contract` inputs and outputs in the case they are required during the execution - Reserved witness slots for signed coins filled with `64` zeroes - Set script gas limit(unless `script` is empty) - Estimated predicates, if `estimate_predicates == true` - Returns an error if: - The number of required balances exceeds the maximum number of inputs allowed. - The fee address index is out of bounds. - The same asset has multiple change policies(either the receiver of the change is different, or one of the policies states about the destruction of the token while the other does not). The `Change` output from the transaction also count as a `ChangePolicy`. - The number of excluded coin IDs exceeds the maximum number of inputs allowed. - Required assets have multiple entries. - If accounts don't have sufficient amounts to cover the transaction requirements in assets. - If a constructed transaction breaks the rules defined by consensus parameters. - [2780](FuelLabs/fuel-core#2780): Add implementations for the pre-confirmation signing task - [2784](FuelLabs/fuel-core#2784): Integrate the pre conf signature task into the main consensus task - [2788](FuelLabs/fuel-core#2788): Scaffold dedicated compression service. - [2799](FuelLabs/fuel-core#2799): Add a transaction waiter to the executor to wait for potential new transactions inside the block production window. Add a channel to send preconfirmation created by executor to the other modules Added a new CLI arguments: - `--production-timeout` to control the block production timeout in the case if block producer stuck. - `--poa-open-period` set the block production mode to `Open`. The `Open` mode starts the production of the next block immediately after the previous block. The block is open until the `period` passed. The period is a duration represented by `100ms`, `1s`, `1m`, etc. The manual block production is disabled if this production mode is used. - [2802](FuelLabs/fuel-core#2802): Add a new cache with outputs extracted from the pool for the duration of the block. - [2824](FuelLabs/fuel-core#2824): Introduce new `Try`-like methods for the `TaskNextAction` - [2840](FuelLabs/fuel-core#2840): Added a new CLI arguments: - `assemble-tx-dry-run-limit` - The max number how many times script can be executed during `assemble_tx` GraphQL request. Default value is `3` times. - `assemble-tx-estimate-predicates-limit` - The max number how many times predicates can be estimated during `assemble_tx` GraphQL request. Default values is `10` times. - [2841](FuelLabs/fuel-core#2841): Following endpoints allow estimation of predicates on submission of the transaction via new `estimatePredicates` argument: - `submit` - `submit_and_await` - `submit_and_await_status` The change is backward compatible with all SDKs. The change is not forward-compatible with Rust SDK in the case of the `estiamte_predicates` flag set. - [2844](FuelLabs/fuel-core#2844): Implement DA compression in `fuel-core-compression-service`. - [2845](FuelLabs/fuel-core#2845): New status to manage the pre confirmation status send in `TxUpdateSender`. - [2855](FuelLabs/fuel-core#2855): Add an expiration interval check for pending pool and refactor extracted_outputs to not rely on block creation/process sequence. - [2856](FuelLabs/fuel-core#2856): Add generic logic for managing the signatures and delegate keys for pre-confirmations signatures - [2862](FuelLabs/fuel-core#2862): Derive `enum_iterator::Sequence` and `strum_macros::{EnumCount, IntoStaticStr}` for MerkleizedColumn. ### Changed - [2388](FuelLabs/fuel-core#2388): Rework the P2P service codecs to avoid unnecessary coupling between components. The refactoring makes it explicit that the Gossipsub and RequestResponse codecs only share encoding/decoding functionalities from the Postcard codec. It also makes handling Gossipsub and RequestResponse messages completely independent of each other. - [2460](FuelLabs/fuel-core#2460): The type of the `max_response_size`for the postcard codec used in `RequestResponse` protocols has been changed from `usize` to `u64`. - [2473](FuelLabs/fuel-core#2473): Graphql requests and responses make use of a new `extensions` object to specify request/response metadata. A request `extensions` object can contain an integer-valued `required_fuel_block_height` field. When specified, the request will return an error unless the node's current fuel block height is at least the value specified in the `required_fuel_block_height` field. All graphql responses now contain an integer-valued `current_fuel_block_height` field in the `extensions` object, which contains the block height of the last block processed by the node. - [2618](FuelLabs/fuel-core#2618): Parallelize block/transaction changes creation in Importer - [2653](FuelLabs/fuel-core#2653): Added cleaner error for wasm-executor upon failed deserialization. - [2656](FuelLabs/fuel-core#2656): Migrate test helper function `create_contract` to `fuel_core_types::test_helpers::create_contract`, and refactor test in proof_system/global_merkle_root crate to use this function. - [2659](FuelLabs/fuel-core#2659): Replace `derivative` crate with `educe` crate. - [2705](FuelLabs/fuel-core#2705): Update the default value for `--max-block-size` and `--max-transmit-size` to 50 MB - [2715](FuelLabs/fuel-core#2715): Each GraphQL response contains `current_consensus_parameters_version` and `current_stf_version` in the `extensions` section. - [2723](FuelLabs/fuel-core#2723): Change the way we are building the changelog to avoids conflicts. - [2725](FuelLabs/fuel-core#2725): New txpool worker to remove lock contention - [2752](FuelLabs/fuel-core#2752): Extended the `TransactionStatus` to support pre-confirmations. - [2761](FuelLabs/fuel-core#2761): Renamed `ConsensusParametersProvider` to `ChainStateInfoProvider` because it is now providing more than just info about consensus parameters. - [2767](FuelLabs/fuel-core#2767): Updated fuel-vm to v0.60.0, see [release notes](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.60.0). - [2781](FuelLabs/fuel-core#2781): Deprecate `dryRun` mutation. Use `dryRun` query instead. - [2791](FuelLabs/fuel-core#2791): Added `TxStatusManager` service which serves as a single source of truth regarding the current statuses of transactions - [2793](FuelLabs/fuel-core#2793): Moved common merkle storage trait implementations to `fuel-core-storage` and made it easier to setup a set of columns that need merkleization. - [2799](FuelLabs/fuel-core#2799): Change the block production to not be trigger after an interval but directly after the creation of last block and let the executor run for the block time window. - [2800](FuelLabs/fuel-core#2800): Implement P2P adapter for preconfirmation broadcasting - [2802](FuelLabs/fuel-core#2802): Change new txs notifier to be notified only on executable transactions - [2811](FuelLabs/fuel-core#2811): When the state rewind window of 7d was triggered, the `is_migration_in_progress` was repeatedly called, resulting in multiple iterations over the empty ModificationsHistoryV1 table. Iteration was slow because compaction didn't have a chance to clean up V1 table. We removed iteration from the migration process. - [2824](FuelLabs/fuel-core#2824): Improve conditions where `Error`s in the `PreConfirmationSignatureTask` stop the service - [2840](FuelLabs/fuel-core#2840): Removed `log_backtrace` logic from the executor. It is not needed anymore with the existence of the local debugger for the transactions. - [2865](FuelLabs/fuel-core#2865): Consider the following transaction statuses as final: `Success`, `Failure`, `SqueezedOut`, `PreConfirmationSqueezedOut`. All other statuses will be considered transient. ### Fixed - [2646](FuelLabs/fuel-core#2646): Improved performance of fetching block height by caching it when the view is created. - [2682](FuelLabs/fuel-core#2682): Fixed the issue with RPC consistency feature for the subscriptions(without the fix first we perform the logic of the query, and only after verify the required height). - [2730](FuelLabs/fuel-core#2730): Fixed RocksDB closing issue that potentially could panic. - [2743](FuelLabs/fuel-core#2743): Allow discovery of the peers when slots for functional connections are consumed. Reserved nodes are not affected by the limitation on connections anymore. - [2746](FuelLabs/fuel-core#2746): Fixed flaky part in the e2e tests and version compatibility tests. Speed up compatibility tests execution time. Decreased the default time between block height propagation throw the network. - [2758](FuelLabs/fuel-core#2758): Made `tx_id_commitment` feature flagged in `fuel-core-client`. - [2832](FuelLabs/fuel-core#2832): - Trigger block production only when all other sub services are started. - Fix relayer syncing issue causing block production to slow down occasionally. - [2840](FuelLabs/fuel-core#2840): Fixed `fuel-core-client` receipt deserialization in the case if the `ContractId` is zero. ### Removed - [2863](FuelLabs/fuel-core#2863): Removed everything related to the state root service, as it has been moved to another repo. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: AurelienFT <[email protected]>
Linked Issues/PRs
part 1/4 of #2568
Description
Implementation of various V2 temporal registry tables with Merklized encoding.
New modules and features for fault-proofing:
crates/fuel-core/src/graphql_api/storage/da_compression.rs: Added a new modulev2under theda_compressiondirectory to support fault-proofing.crates/fuel-core/src/graphql_api/storage/da_compression/v2.rs: Introduced new modules foraddress,asset_id,contract_id,predicate_code, andscript_codeto handle fault-proofing.Updates to
Columnenum:crates/fuel-core/src/graphql_api/storage.rs: Added new entries to theColumnenum to support the new data compression types for fault-proofing.Implementation of V2 tables with Merklized encoding:
crates/fuel-core/src/graphql_api/storage/da_compression/v2/address.rs: ImplementedDaCompressionTemporalRegistryAddressV2and related structures with Merklized encoding.crates/fuel-core/src/graphql_api/storage/da_compression/v2/asset_id.rs: ImplementedDaCompressionTemporalRegistryAssetIdV2and related structures with Merklized encoding.crates/fuel-core/src/graphql_api/storage/da_compression/v2/contract_id.rs: ImplementedDaCompressionTemporalRegistryContractIdV2and related structures with Merklized encoding.crates/fuel-core/src/graphql_api/storage/da_compression/v2/predicate_code.rs: ImplementedDaCompressionTemporalRegistryPredicateCodeV2and related structures with Merklized encoding.crates/fuel-core/src/graphql_api/storage/da_compression/v2/script_code.rs: ImplementedDaCompressionTemporalRegistryScriptCodeV2and related structures with Merklized encoding.Other changes:
crates/fuel-core/Cargo.toml: Added new fault-proofing dependencies forfuel-core-importerandfuel-core-poa.crates/storage/src/blueprint/merklized.rs: Updated the seed value inbasic_merklelized_storage_testsmacro for consistency in tests. [1] [2]Checklist
Before requesting review
After merging, notify other teams
[Add or remove entries as needed]