RPC Consistency Proposal #2473

acerone85 · 2024-12-05T12:09:34Z

Linked Issues/PRs

Related issue: #1897
Closes #2605

See https://www.notion.so/fuellabs/RPC-Consistency-Proposal-V2-13b2f2293f31809bbce0d93a4c28d633 for the proposal specs.

This PR implements a variation of the proposal in the link, due to techincal limitations of the HTTP protocol when using graphql subscriptions.

for context: Headers cannot be used, the reason being that subscriptions will return multiple graphql responses in a single HTTP response (and therefore have a single HeaderMap), while we require that each item in the response has its associated CURRENT_FUEL_BLOCK_HEIGHT field. Moreover the Response metadata for subscriptions is returned before the actual response body (as it is standard with HTTP).

in the end graphql has a (very undocumented) way to add an extensions field to responses, at the same level of data/errors field. This does not require to tweak anything at the HTTP level.

We will implement an improvement where the server node will wait for a configurable number of blocks before returning a PRECONDITION FAILED response, in a follow-up PR.

Entrypoint for reviews:

https://github.com/FuelLabs/fuel-core/pull/2473/files#diff-9caf873d3167ab9485e7882d621b5147e98b2478cc6462095b88374e2f914f3aR428 for the graphql handler
https://github.com/FuelLabs/fuel-core/pull/2473/files#diff-9caf873d3167ab9485e7882d621b5147e98b2478cc6462095b88374e2f914f3aR428 for the extension definition

Manual testing

TODO:

Tests

Description

Checklist

~~Breaking changes are clearly marked as such in the PR description and changelog~~
New behavior is reflected in tests
The specification matches the implemented behavior (link update PR if changes are needed)

Before requesting review

I have reviewed the code myself
~~I have created follow-up issues caused by this PR and linked them here~~

After merging, notify other teams

[Add or remove entries as needed]

Rust SDK
Sway compiler
Platform documentation (for out-of-organization contributors, the person merging the PR will do this)
Someone else?

crates/client/src/client.rs

crates/fuel-core/src/graphql_api/worker_service.rs

netrome

Nice stuff! I have some thoughts and comments, but no showstoppers.

crates/client/src/client.rs

crates/fuel-core/src/graphql_api/worker_service.rs

crates/fuel-core/src/graphql_api/api_service.rs

Co-authored-by: Mårten Blankfors <[email protected]>

crates/fuel-core/src/graphql_api/required_fuel_block_height_extension.rs

crates/fuel-core/src/graphql_api/api_service.rs

crates/fuel-core/src/graphql_api/required_fuel_block_height_extension.rs

Torres-ssf · 2024-12-13T17:15:30Z

Hi @acerone85,

Could you kindly confirm if the response header for subscriptions is also intended to include current_fuel_block_height?

acerone85 · 2024-12-14T18:11:56Z

Hi @acerone85,

Could you kindly confirm if the response header for subscriptions is also intended to include current_fuel_block_height?

In the current implementation there is no response header for the current_fuel_block_height. Would it be desirable to have one?

…nto 1897-rpc-consistency-proposal # Conflicts: # CHANGELOG.md

acerone85 · 2025-02-12T13:56:27Z

crates/client/src/reqwest_ext.rs

+/// #
+/// # #[derive(cynic::QueryFragment)]
+/// # #[cynic(
+/// #     schema_path = "../schemas/starwars.schema.graphql",


would be nice to have an example related to how we use the reqwest_ext module, but not a big deal at this point :)

acerone85 · 2025-02-12T13:58:38Z

crates/client/src/reqwest_ext.rs

+#[cfg(not(target_arch = "wasm32"))]
+type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + Send + 'a>>;
+
+#[cfg(target_arch = "wasm32")]
+type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + 'a>>;


curious to know why the Send trait bound is not available/needed on the wasm32 architecture

TIL: It seems like it's a general assumption that the wasm32-unknown-unknown target is single-threaded https://users.rust-lang.org/t/multithreading-in-wasm-how-did-that-come-out/87518 - although there seem to be work to implement multi-threading in WASM coming up in the future. Interesting.

(and if the target is single-threaded you should not have to require Send anywhere)

acerone85 · 2025-02-12T14:04:52Z

crates/client/src/reqwest_ext.rs

+impl<ResponseData, Errors> CynicReqwestBuilder<ResponseData, Errors> {
+    pub fn new(builder: reqwest::RequestBuilder) -> Self {
+        Self {
+            builder,
+            _marker: std::marker::PhantomData,
+        }
+    }
+}


nit: is impl<ResponseData, Errors> From<reqwest::RequestBuilder>for CynicReqwestBuilder<ResponseData, Errors> more idiomatic?

acerone85 · 2025-02-12T14:16:44Z

crates/client/src/reqwest_ext.rs

+            return Err(response);
+        };
+
+        Ok(deserred)


just wanted to check, we return Ok(_) even if the status is not 2xx?

(I think it makes sense if we reserve the Error status for actual connection/deserialization errors, but wanted to check)

acerone85 · 2025-02-12T14:20:41Z

crates/client/src/reqwest_ext.rs

+#[derive(Debug, Clone, serde::Deserialize)]
+pub struct ExtensionsResponse {
+    pub required_fuel_block_height: Option<BlockHeight>,
+    pub current_fuel_block_height: Option<BlockHeight>,


nit: I think we require that this field is set in all graphql responses

acerone85 · 2025-02-12T14:23:01Z

crates/client/src/reqwest_ext.rs

+
+#[derive(Debug, Clone, serde::Deserialize)]
+pub struct ExtensionsResponse {
+    pub required_fuel_block_height: Option<BlockHeight>,


we don't set this field in the response in the graphql extension. But happy to add it if needed

acerone85 · 2025-02-12T15:39:56Z

crates/client/src/client.rs

+    ) -> &mut Self {
+        match &mut self.require_height {
+            ConsistencyPolicy::Auto { height } => {
+                *height.lock().expect("Mutex poisoned") = new_height;


should we use parking_lot Mutex, or use directly a AtomicU32 and avoid a Mutex altogether?

Why do we need locks for this at all? We have a mutable reference to self here already.

So I see that we need some type of interior mutability since the query function takes an immutable self receiver. However, I think it would be cleaner to change the signature of the query function to take a mutable self receiver.

Since that's a breaking change, how about we leave this as-is for now but create a follow-up to simplify this which we can include in the next breaking release?

netrome

The code seems correct to me, but I'm pretty annoyed by the Option<Arc<Mutex>> and would love to get rid of it as soon as possible. I understand that we might need some sort of interior mutability to preserve backwards compatibility in the client though.

netrome · 2025-02-12T19:33:43Z

crates/client/src/client.rs

+    ) -> &mut Self {
+        match &mut self.require_height {
+            ConsistencyPolicy::Auto { height } => {
+                *height.lock().expect("Mutex poisoned") = new_height;


Why do we need locks for this at all? We have a mutable reference to self here already.

netrome · 2025-02-12T19:39:47Z

crates/client/src/client.rs

+    fn decode_response<R, E>(
+        response: FuelGraphQlResponse<R, E>,
+        inner_required_height: Option<Arc<Mutex<Option<BlockHeight>>>>,
+    ) -> io::Result<R>


While this is okay, I think it would be cleaner to separate the height extraction to a separate extract_required_height function.

netrome · 2025-02-12T19:42:36Z

crates/client/src/client.rs

+    ) -> &mut Self {
+        match &mut self.require_height {
+            ConsistencyPolicy::Auto { height } => {
+                *height.lock().expect("Mutex poisoned") = new_height;


So I see that we need some type of interior mutability since the query function takes an immutable self receiver. However, I think it would be cleaner to change the signature of the query function to take a mutable self receiver.

Since that's a breaking change, how about we leave this as-is for now but create a follow-up to simplify this which we can include in the next breaking release?

netrome · 2025-02-12T19:46:50Z

crates/client/src/reqwest_ext.rs

+#[cfg(not(target_arch = "wasm32"))]
+type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + Send + 'a>>;
+
+#[cfg(target_arch = "wasm32")]
+type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + 'a>>;


TIL: It seems like it's a general assumption that the wasm32-unknown-unknown target is single-threaded https://users.rust-lang.org/t/multithreading-in-wasm-how-did-that-come-out/87518 - although there seem to be work to implement multi-threading in WASM coming up in the future. Interesting.

netrome · 2025-02-12T19:47:38Z

crates/client/src/reqwest_ext.rs

+#[cfg(not(target_arch = "wasm32"))]
+type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + Send + 'a>>;
+
+#[cfg(target_arch = "wasm32")]
+type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + 'a>>;


(and if the target is single-threaded you should not have to require Send anywhere)

## Version 0.42.0 ### Breaking - [2648](#2648): Add feature-flagged field to block header `fault_proving_header` that contains a commitment to all transaction ids. - [2678](#2678): Removed public accessors for `BlockHeader` fields and replaced with methods instead, moved `tx_id_commitment` to the application header of `BlockHeaderV2`. - [2746](#2746): Breaking changes to the CLI arguments: - To disable `random-walk` just don't specify it. Before it required to use `--random-walk 0`. - Next CLI arguments were renamed: - `relayer-min-duration-s` -> `relayer-min-duration` - `relayer-eth-sync-call-freq-s` -> `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq-s` -> `relayer-eth-sync-log-freq` - Default value for the `heartbeat-idle-duration` was changed from `1s` to `100ms`. So information about new block will be propagated faster by default. - All CLI arguments below use time(like `100ms`, `1s`, `1d`, etc.) use as a flag argument instead of number of seconds: - `random-walk` - `connection-idle-timeout` - `info-interval` - `identify-interval` - `request-timeout` - `connection-keep-alive` - `heartbeat-send-duration` - `heartbeat-idle-duration` - `heartbeat-check-interval` - `heartbeat-max-avg-interval` - `heartbeat-max-time-since-last` - `relayer-min-duration` - `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq` - [2840](#2840): CLI argument `vm-backtrace` is deprecated and does nothing. It will be removed in a future version of `fuel-core`. The `extra_tx_checks` field was renamed into `forbid_fake_coins` that affects JSON based serialization/deserialization. Renamed `extra_tx_checks_default` field into `forbid_fake_coins_default`. ### Added - [2150](#2150): Upgraded `libp2p` to `0.54.1` and introduced `ConnectionLimiter` to limit pending incoming/outgoing connections. - [2491](#2491): Storage read replays of historical blocks for execution tracing. Only available behind `--historical-execution` flag. - [2619](#2619): Add possibility to submit list of changes to rocksdb. - [2666](#2666): Added two new CLI arguments to control the GraphQL queries consistency: `--graphql-block-height-tolerance` (default: `10`) and `--graphql-block-height-min-timeout` (default: `30s`). If a request requires a specific block height and the node is slightly behind, it will wait instead of failing. - [2682](#2682): Added GraphQL APIs to get contract storage and balances for current and past blocks. - [2719](#2719): Merklized DA compression temporal registry tables. - [2722](#2722): Service definition for state root service. - [2724](#2724): Explicit error type for merkleized storage. - [2726](#2726): Add a new gossip-sub message for transaction preconfirmations - [2731](#2731): Include `TemporalRegistry` trait implementations for v2 tables. - [2733](#2733): Add a pending pool transaction that allow transaction to wait a bit of time if an input is missing instead of direct delete. - [2742](#2742): Added API crate for merkle root service. - [2756](#2756): Add new service for managing pre-confirmations - [2769](#2769): Added a new `assembleTx` GraphQL endpoint. The endpoint can be used to assemble the transaction based on the provided requirements. - The returned transaction contains: - Input coins to cover `required_balances` - Input coins to cover the fee of the transaction based on the gas price from `block_horizon` - `Change` or `Destroy` outputs for all assets from the inputs - `Variable` outputs in the case they are required during the execution - `Contract` inputs and outputs in the case they are required during the execution - Reserved witness slots for signed coins filled with `64` zeroes - Set script gas limit(unless `script` is empty) - Estimated predicates, if `estimate_predicates == true` - Returns an error if: - The number of required balances exceeds the maximum number of inputs allowed. - The fee address index is out of bounds. - The same asset has multiple change policies(either the receiver of the change is different, or one of the policies states about the destruction of the token while the other does not). The `Change` output from the transaction also count as a `ChangePolicy`. - The number of excluded coin IDs exceeds the maximum number of inputs allowed. - Required assets have multiple entries. - If accounts don't have sufficient amounts to cover the transaction requirements in assets. - If a constructed transaction breaks the rules defined by consensus parameters. - [2780](#2780): Add implementations for the pre-confirmation signing task - [2784](#2784): Integrate the pre conf signature task into the main consensus task - [2788](#2788): Scaffold dedicated compression service. - [2799](#2799): Add a transaction waiter to the executor to wait for potential new transactions inside the block production window. Add a channel to send preconfirmation created by executor to the other modules Added a new CLI arguments: - `--production-timeout` to control the block production timeout in the case if block producer stuck. - `--poa-open-period` set the block production mode to `Open`. The `Open` mode starts the production of the next block immediately after the previous block. The block is open until the `period` passed. The period is a duration represented by `100ms`, `1s`, `1m`, etc. The manual block production is disabled if this production mode is used. - [2802](#2802): Add a new cache with outputs extracted from the pool for the duration of the block. - [2824](#2824): Introduce new `Try`-like methods for the `TaskNextAction` - [2840](#2840): Added a new CLI arguments: - `assemble-tx-dry-run-limit` - The max number how many times script can be executed during `assemble_tx` GraphQL request. Default value is `3` times. - `assemble-tx-estimate-predicates-limit` - The max number how many times predicates can be estimated during `assemble_tx` GraphQL request. Default values is `10` times. - [2841](#2841): Following endpoints allow estimation of predicates on submission of the transaction via new `estimatePredicates` argument: - `submit` - `submit_and_await` - `submit_and_await_status` The change is backward compatible with all SDKs. The change is not forward-compatible with Rust SDK in the case of the `estiamte_predicates` flag set. - [2844](#2844): Implement DA compression in `fuel-core-compression-service`. - [2845](#2845): New status to manage the pre confirmation status send in `TxUpdateSender`. - [2855](#2855): Add an expiration interval check for pending pool and refactor extracted_outputs to not rely on block creation/process sequence. - [2856](#2856): Add generic logic for managing the signatures and delegate keys for pre-confirmations signatures - [2862](#2862): Derive `enum_iterator::Sequence` and `strum_macros::{EnumCount, IntoStaticStr}` for MerkleizedColumn. ### Changed - [2388](#2388): Rework the P2P service codecs to avoid unnecessary coupling between components. The refactoring makes it explicit that the Gossipsub and RequestResponse codecs only share encoding/decoding functionalities from the Postcard codec. It also makes handling Gossipsub and RequestResponse messages completely independent of each other. - [2460](#2460): The type of the `max_response_size`for the postcard codec used in `RequestResponse` protocols has been changed from `usize` to `u64`. - [2473](#2473): Graphql requests and responses make use of a new `extensions` object to specify request/response metadata. A request `extensions` object can contain an integer-valued `required_fuel_block_height` field. When specified, the request will return an error unless the node's current fuel block height is at least the value specified in the `required_fuel_block_height` field. All graphql responses now contain an integer-valued `current_fuel_block_height` field in the `extensions` object, which contains the block height of the last block processed by the node. - [2618](#2618): Parallelize block/transaction changes creation in Importer - [2653](#2653): Added cleaner error for wasm-executor upon failed deserialization. - [2656](#2656): Migrate test helper function `create_contract` to `fuel_core_types::test_helpers::create_contract`, and refactor test in proof_system/global_merkle_root crate to use this function. - [2659](#2659): Replace `derivative` crate with `educe` crate. - [2705](#2705): Update the default value for `--max-block-size` and `--max-transmit-size` to 50 MB - [2715](#2715): Each GraphQL response contains `current_consensus_parameters_version` and `current_stf_version` in the `extensions` section. - [2723](#2723): Change the way we are building the changelog to avoids conflicts. - [2725](#2725): New txpool worker to remove lock contention - [2752](#2752): Extended the `TransactionStatus` to support pre-confirmations. - [2761](#2761): Renamed `ConsensusParametersProvider` to `ChainStateInfoProvider` because it is now providing more than just info about consensus parameters. - [2767](#2767): Updated fuel-vm to v0.60.0, see [release notes](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.60.0). - [2781](#2781): Deprecate `dryRun` mutation. Use `dryRun` query instead. - [2791](#2791): Added `TxStatusManager` service which serves as a single source of truth regarding the current statuses of transactions - [2793](#2793): Moved common merkle storage trait implementations to `fuel-core-storage` and made it easier to setup a set of columns that need merkleization. - [2799](#2799): Change the block production to not be trigger after an interval but directly after the creation of last block and let the executor run for the block time window. - [2800](#2800): Implement P2P adapter for preconfirmation broadcasting - [2802](#2802): Change new txs notifier to be notified only on executable transactions - [2811](#2811): When the state rewind window of 7d was triggered, the `is_migration_in_progress` was repeatedly called, resulting in multiple iterations over the empty ModificationsHistoryV1 table. Iteration was slow because compaction didn't have a chance to clean up V1 table. We removed iteration from the migration process. - [2824](#2824): Improve conditions where `Error`s in the `PreConfirmationSignatureTask` stop the service - [2840](#2840): Removed `log_backtrace` logic from the executor. It is not needed anymore with the existence of the local debugger for the transactions. - [2865](#2865): Consider the following transaction statuses as final: `Success`, `Failure`, `SqueezedOut`, `PreConfirmationSqueezedOut`. All other statuses will be considered transient. ### Fixed - [2646](#2646): Improved performance of fetching block height by caching it when the view is created. - [2682](#2682): Fixed the issue with RPC consistency feature for the subscriptions(without the fix first we perform the logic of the query, and only after verify the required height). - [2730](#2730): Fixed RocksDB closing issue that potentially could panic. - [2743](#2743): Allow discovery of the peers when slots for functional connections are consumed. Reserved nodes are not affected by the limitation on connections anymore. - [2746](#2746): Fixed flaky part in the e2e tests and version compatibility tests. Speed up compatibility tests execution time. Decreased the default time between block height propagation throw the network. - [2758](#2758): Made `tx_id_commitment` feature flagged in `fuel-core-client`. - [2832](#2832): - Trigger block production only when all other sub services are started. - Fix relayer syncing issue causing block production to slow down occasionally. - [2840](#2840): Fixed `fuel-core-client` receipt deserialization in the case if the `ContractId` is zero. ### Removed - [2863](#2863): Removed everything related to the state root service, as it has been moved to another repo. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: AurelienFT <[email protected]>

## Version 0.42.0 ### Breaking - [2648](FuelLabs/fuel-core#2648): Add feature-flagged field to block header `fault_proving_header` that contains a commitment to all transaction ids. - [2678](FuelLabs/fuel-core#2678): Removed public accessors for `BlockHeader` fields and replaced with methods instead, moved `tx_id_commitment` to the application header of `BlockHeaderV2`. - [2746](FuelLabs/fuel-core#2746): Breaking changes to the CLI arguments: - To disable `random-walk` just don't specify it. Before it required to use `--random-walk 0`. - Next CLI arguments were renamed: - `relayer-min-duration-s` -> `relayer-min-duration` - `relayer-eth-sync-call-freq-s` -> `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq-s` -> `relayer-eth-sync-log-freq` - Default value for the `heartbeat-idle-duration` was changed from `1s` to `100ms`. So information about new block will be propagated faster by default. - All CLI arguments below use time(like `100ms`, `1s`, `1d`, etc.) use as a flag argument instead of number of seconds: - `random-walk` - `connection-idle-timeout` - `info-interval` - `identify-interval` - `request-timeout` - `connection-keep-alive` - `heartbeat-send-duration` - `heartbeat-idle-duration` - `heartbeat-check-interval` - `heartbeat-max-avg-interval` - `heartbeat-max-time-since-last` - `relayer-min-duration` - `relayer-eth-sync-call-freq` - `relayer-eth-sync-log-freq` - [2840](FuelLabs/fuel-core#2840): CLI argument `vm-backtrace` is deprecated and does nothing. It will be removed in a future version of `fuel-core`. The `extra_tx_checks` field was renamed into `forbid_fake_coins` that affects JSON based serialization/deserialization. Renamed `extra_tx_checks_default` field into `forbid_fake_coins_default`. ### Added - [2150](FuelLabs/fuel-core#2150): Upgraded `libp2p` to `0.54.1` and introduced `ConnectionLimiter` to limit pending incoming/outgoing connections. - [2491](FuelLabs/fuel-core#2491): Storage read replays of historical blocks for execution tracing. Only available behind `--historical-execution` flag. - [2619](FuelLabs/fuel-core#2619): Add possibility to submit list of changes to rocksdb. - [2666](FuelLabs/fuel-core#2666): Added two new CLI arguments to control the GraphQL queries consistency: `--graphql-block-height-tolerance` (default: `10`) and `--graphql-block-height-min-timeout` (default: `30s`). If a request requires a specific block height and the node is slightly behind, it will wait instead of failing. - [2682](FuelLabs/fuel-core#2682): Added GraphQL APIs to get contract storage and balances for current and past blocks. - [2719](FuelLabs/fuel-core#2719): Merklized DA compression temporal registry tables. - [2722](FuelLabs/fuel-core#2722): Service definition for state root service. - [2724](FuelLabs/fuel-core#2724): Explicit error type for merkleized storage. - [2726](FuelLabs/fuel-core#2726): Add a new gossip-sub message for transaction preconfirmations - [2731](FuelLabs/fuel-core#2731): Include `TemporalRegistry` trait implementations for v2 tables. - [2733](FuelLabs/fuel-core#2733): Add a pending pool transaction that allow transaction to wait a bit of time if an input is missing instead of direct delete. - [2742](FuelLabs/fuel-core#2742): Added API crate for merkle root service. - [2756](FuelLabs/fuel-core#2756): Add new service for managing pre-confirmations - [2769](FuelLabs/fuel-core#2769): Added a new `assembleTx` GraphQL endpoint. The endpoint can be used to assemble the transaction based on the provided requirements. - The returned transaction contains: - Input coins to cover `required_balances` - Input coins to cover the fee of the transaction based on the gas price from `block_horizon` - `Change` or `Destroy` outputs for all assets from the inputs - `Variable` outputs in the case they are required during the execution - `Contract` inputs and outputs in the case they are required during the execution - Reserved witness slots for signed coins filled with `64` zeroes - Set script gas limit(unless `script` is empty) - Estimated predicates, if `estimate_predicates == true` - Returns an error if: - The number of required balances exceeds the maximum number of inputs allowed. - The fee address index is out of bounds. - The same asset has multiple change policies(either the receiver of the change is different, or one of the policies states about the destruction of the token while the other does not). The `Change` output from the transaction also count as a `ChangePolicy`. - The number of excluded coin IDs exceeds the maximum number of inputs allowed. - Required assets have multiple entries. - If accounts don't have sufficient amounts to cover the transaction requirements in assets. - If a constructed transaction breaks the rules defined by consensus parameters. - [2780](FuelLabs/fuel-core#2780): Add implementations for the pre-confirmation signing task - [2784](FuelLabs/fuel-core#2784): Integrate the pre conf signature task into the main consensus task - [2788](FuelLabs/fuel-core#2788): Scaffold dedicated compression service. - [2799](FuelLabs/fuel-core#2799): Add a transaction waiter to the executor to wait for potential new transactions inside the block production window. Add a channel to send preconfirmation created by executor to the other modules Added a new CLI arguments: - `--production-timeout` to control the block production timeout in the case if block producer stuck. - `--poa-open-period` set the block production mode to `Open`. The `Open` mode starts the production of the next block immediately after the previous block. The block is open until the `period` passed. The period is a duration represented by `100ms`, `1s`, `1m`, etc. The manual block production is disabled if this production mode is used. - [2802](FuelLabs/fuel-core#2802): Add a new cache with outputs extracted from the pool for the duration of the block. - [2824](FuelLabs/fuel-core#2824): Introduce new `Try`-like methods for the `TaskNextAction` - [2840](FuelLabs/fuel-core#2840): Added a new CLI arguments: - `assemble-tx-dry-run-limit` - The max number how many times script can be executed during `assemble_tx` GraphQL request. Default value is `3` times. - `assemble-tx-estimate-predicates-limit` - The max number how many times predicates can be estimated during `assemble_tx` GraphQL request. Default values is `10` times. - [2841](FuelLabs/fuel-core#2841): Following endpoints allow estimation of predicates on submission of the transaction via new `estimatePredicates` argument: - `submit` - `submit_and_await` - `submit_and_await_status` The change is backward compatible with all SDKs. The change is not forward-compatible with Rust SDK in the case of the `estiamte_predicates` flag set. - [2844](FuelLabs/fuel-core#2844): Implement DA compression in `fuel-core-compression-service`. - [2845](FuelLabs/fuel-core#2845): New status to manage the pre confirmation status send in `TxUpdateSender`. - [2855](FuelLabs/fuel-core#2855): Add an expiration interval check for pending pool and refactor extracted_outputs to not rely on block creation/process sequence. - [2856](FuelLabs/fuel-core#2856): Add generic logic for managing the signatures and delegate keys for pre-confirmations signatures - [2862](FuelLabs/fuel-core#2862): Derive `enum_iterator::Sequence` and `strum_macros::{EnumCount, IntoStaticStr}` for MerkleizedColumn. ### Changed - [2388](FuelLabs/fuel-core#2388): Rework the P2P service codecs to avoid unnecessary coupling between components. The refactoring makes it explicit that the Gossipsub and RequestResponse codecs only share encoding/decoding functionalities from the Postcard codec. It also makes handling Gossipsub and RequestResponse messages completely independent of each other. - [2460](FuelLabs/fuel-core#2460): The type of the `max_response_size`for the postcard codec used in `RequestResponse` protocols has been changed from `usize` to `u64`. - [2473](FuelLabs/fuel-core#2473): Graphql requests and responses make use of a new `extensions` object to specify request/response metadata. A request `extensions` object can contain an integer-valued `required_fuel_block_height` field. When specified, the request will return an error unless the node's current fuel block height is at least the value specified in the `required_fuel_block_height` field. All graphql responses now contain an integer-valued `current_fuel_block_height` field in the `extensions` object, which contains the block height of the last block processed by the node. - [2618](FuelLabs/fuel-core#2618): Parallelize block/transaction changes creation in Importer - [2653](FuelLabs/fuel-core#2653): Added cleaner error for wasm-executor upon failed deserialization. - [2656](FuelLabs/fuel-core#2656): Migrate test helper function `create_contract` to `fuel_core_types::test_helpers::create_contract`, and refactor test in proof_system/global_merkle_root crate to use this function. - [2659](FuelLabs/fuel-core#2659): Replace `derivative` crate with `educe` crate. - [2705](FuelLabs/fuel-core#2705): Update the default value for `--max-block-size` and `--max-transmit-size` to 50 MB - [2715](FuelLabs/fuel-core#2715): Each GraphQL response contains `current_consensus_parameters_version` and `current_stf_version` in the `extensions` section. - [2723](FuelLabs/fuel-core#2723): Change the way we are building the changelog to avoids conflicts. - [2725](FuelLabs/fuel-core#2725): New txpool worker to remove lock contention - [2752](FuelLabs/fuel-core#2752): Extended the `TransactionStatus` to support pre-confirmations. - [2761](FuelLabs/fuel-core#2761): Renamed `ConsensusParametersProvider` to `ChainStateInfoProvider` because it is now providing more than just info about consensus parameters. - [2767](FuelLabs/fuel-core#2767): Updated fuel-vm to v0.60.0, see [release notes](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.60.0). - [2781](FuelLabs/fuel-core#2781): Deprecate `dryRun` mutation. Use `dryRun` query instead. - [2791](FuelLabs/fuel-core#2791): Added `TxStatusManager` service which serves as a single source of truth regarding the current statuses of transactions - [2793](FuelLabs/fuel-core#2793): Moved common merkle storage trait implementations to `fuel-core-storage` and made it easier to setup a set of columns that need merkleization. - [2799](FuelLabs/fuel-core#2799): Change the block production to not be trigger after an interval but directly after the creation of last block and let the executor run for the block time window. - [2800](FuelLabs/fuel-core#2800): Implement P2P adapter for preconfirmation broadcasting - [2802](FuelLabs/fuel-core#2802): Change new txs notifier to be notified only on executable transactions - [2811](FuelLabs/fuel-core#2811): When the state rewind window of 7d was triggered, the `is_migration_in_progress` was repeatedly called, resulting in multiple iterations over the empty ModificationsHistoryV1 table. Iteration was slow because compaction didn't have a chance to clean up V1 table. We removed iteration from the migration process. - [2824](FuelLabs/fuel-core#2824): Improve conditions where `Error`s in the `PreConfirmationSignatureTask` stop the service - [2840](FuelLabs/fuel-core#2840): Removed `log_backtrace` logic from the executor. It is not needed anymore with the existence of the local debugger for the transactions. - [2865](FuelLabs/fuel-core#2865): Consider the following transaction statuses as final: `Success`, `Failure`, `SqueezedOut`, `PreConfirmationSqueezedOut`. All other statuses will be considered transient. ### Fixed - [2646](FuelLabs/fuel-core#2646): Improved performance of fetching block height by caching it when the view is created. - [2682](FuelLabs/fuel-core#2682): Fixed the issue with RPC consistency feature for the subscriptions(without the fix first we perform the logic of the query, and only after verify the required height). - [2730](FuelLabs/fuel-core#2730): Fixed RocksDB closing issue that potentially could panic. - [2743](FuelLabs/fuel-core#2743): Allow discovery of the peers when slots for functional connections are consumed. Reserved nodes are not affected by the limitation on connections anymore. - [2746](FuelLabs/fuel-core#2746): Fixed flaky part in the e2e tests and version compatibility tests. Speed up compatibility tests execution time. Decreased the default time between block height propagation throw the network. - [2758](FuelLabs/fuel-core#2758): Made `tx_id_commitment` feature flagged in `fuel-core-client`. - [2832](FuelLabs/fuel-core#2832): - Trigger block production only when all other sub services are started. - Fix relayer syncing issue causing block production to slow down occasionally. - [2840](FuelLabs/fuel-core#2840): Fixed `fuel-core-client` receipt deserialization in the case if the `ContractId` is zero. ### Removed - [2863](FuelLabs/fuel-core#2863): Removed everything related to the state root service, as it has been moved to another repo. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: AurelienFT <[email protected]>

acerone85 added 2 commits December 5, 2024 11:30

Start structuring middleware

a8f5f41

Use a static atomic to keep track of the current block height

778dd7c

acerone85 requested a review from Voxelot December 5, 2024 12:09

acerone85 requested review from Dentosal, MitchTurner and xgreenx as code owners December 5, 2024 12:09

acerone85 linked an issue Dec 5, 2024 that may be closed by this pull request

RPC Consistency Proposal #1897

Closed

acerone85 requested a review from a team December 5, 2024 12:09

acerone85 added 2 commits December 5, 2024 12:13

Add changelog entry

d1b1c33

Add tests and fix middleware

b77506c

acerone85 self-assigned this Dec 5, 2024

acerone85 commented Dec 5, 2024

View reviewed changes

crates/client/src/client.rs Outdated Show resolved Hide resolved

acerone85 commented Dec 5, 2024

View reviewed changes

crates/fuel-core/src/graphql_api/worker_service.rs Outdated Show resolved Hide resolved

netrome previously approved these changes Dec 5, 2024

View reviewed changes

Update crates/fuel-core/src/graphql_api/api_service.rs

b764cd4

Co-authored-by: Mårten Blankfors <[email protected]>

acerone85 dismissed netrome’s stale review via b764cd4 December 6, 2024 13:34

acerone85 and others added 6 commits December 6, 2024 14:18

Set headers in fuel client

d9091e6

AxumRequest/AxumResponse -> http::Request/http::Response

90c411f

Fix compilation

625bf8f

Extension instead of Axum middleware

085d173

Merge branch 'master' into 1897-rpc-consistency-proposal

a98ef9b

Add response header

c95f09a

acerone85 commented Dec 12, 2024

View reviewed changes

crates/fuel-core/src/graphql_api/required_fuel_block_height_extension.rs Show resolved Hide resolved

acerone85 commented Dec 12, 2024

View reviewed changes

crates/fuel-core/src/graphql_api/api_service.rs Outdated Show resolved Hide resolved

acerone85 commented Dec 12, 2024

View reviewed changes

crates/fuel-core/src/graphql_api/required_fuel_block_height_extension.rs Outdated Show resolved Hide resolved

acerone85 commented Dec 12, 2024

View reviewed changes

crates/fuel-core/src/graphql_api/required_fuel_block_height_extension.rs Outdated Show resolved Hide resolved

acerone85 commented Dec 12, 2024

View reviewed changes

crates/fuel-core/src/graphql_api/required_fuel_block_height_extension.rs Outdated Show resolved Hide resolved

Better handling of response

6099651

xgreenx added 3 commits February 11, 2025 15:46

Merge remote-tracking branch 'origin/1897-rpc-consistency-proposal' i…

703bc24

…nto 1897-rpc-consistency-proposal # Conflicts: # CHANGELOG.md

Fixed CHANGELOG.md

511431f

Implement required block height support on the fuel-core-client side

48d508d

xgreenx previously approved these changes Feb 11, 2025

View reviewed changes

xgreenx and others added 2 commits February 11, 2025 18:39

Merge branch 'master' into 1897-rpc-consistency-proposal

461fbe7

Make CI happy

af2484a

xgreenx dismissed their stale review via af2484a February 12, 2025 00:04

xgreenx added 2 commits February 11, 2025 20:32

Added more tests

44cffec

Use random determenistic port of compatibility tests

5cb1639

xgreenx approved these changes Feb 12, 2025

View reviewed changes

Merge branch 'master' into 1897-rpc-consistency-proposal

eb25459

acerone85 commented Feb 12, 2025

View reviewed changes

netrome approved these changes Feb 12, 2025

View reviewed changes

Merge branch 'master' into 1897-rpc-consistency-proposal

0f6f319

acerone85 enabled auto-merge (squash) February 12, 2025 20:18

acerone85 merged commit f73a12e into master Feb 12, 2025
33 checks passed

acerone85 deleted the 1897-rpc-consistency-proposal branch February 12, 2025 20:43

This was referenced Mar 18, 2025

Release v0.42.0 #2877

Merged

test #2878

Closed

RPC Consistency Proposal #2473

RPC Consistency Proposal #2473

Uh oh!

Conversation

acerone85 commented Dec 5, 2024 • edited by xgreenx Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Linked Issues/PRs

Manual testing

TODO:

Description

Checklist

Before requesting review

After merging, notify other teams

Uh oh!

Uh oh!

Uh oh!

netrome left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Torres-ssf commented Dec 13, 2024

Uh oh!

acerone85 commented Dec 14, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

netrome left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

acerone85 commented Dec 5, 2024 •

edited by xgreenx

Loading