Refactor/rollup node refactor #351

frisitano · 2025-10-07T07:09:06Z

No description provided.

codspeed-hq · 2025-10-08T18:45:00Z

CodSpeed Performance Report

Merging #351 will degrade performances by 97.16%

_{Comparing refactor/rollup-node-refactor (6ff7d55) with main (6852bf6)}

Summary

⚡ 1 improvement
❌ 1 regression

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	`BASE`	`HEAD`	Change
❌	`pipeline_derive_in_file_blobs`	27.4 ms	965.2 ms	-97.16%
⚡	`pipeline_derive_s3_blobs`	16,914.8 ms	79.3 ms	×210

jonastheis

This PR is great! Simplifies the readability and concepts in the flow of the code so much! imo it's much easier to reason about the state of the node than before.

A few things:

we should add an in-depth description of the changes, new features, simplifications -> this will also allow us to systematically evaluate whether we have everything tested or need to add some tests later. + it will help with reviewing
I left a bunch of comments inline.
I'm a bit concerned about performance in some cases but we need to evaluate with benchmarks
I think this PR addresses a few issues at once, we should link that to the description above and then close these issues accordingly:
- #294
- #269
- maybe?: #245
- maybe?: #174

jonastheis · 2025-10-08T02:03:51Z

crates/network/src/manager.rs

 impl<
        N: FullNetwork<Primitives = ScrollNetworkPrimitives>,
        CS: ScrollHardforks + EthChainSpec + Send + Sync + 'static,
-    > Stream for ScrollNetworkManager<N, CS>


Why change this from a Stream to a Future?

Previously the rollup node manager would drive the ScrollNetworkManager future, which would yield NetworkManagerEvent's. Now we spawn the ScrollNetworkManager as a separate task and use channels to send events to the ChainOrchestrator. It's a slightly different architecture but achieves a similar goal. As such we don't need a stream on the ScrollNetworkManager as it doesn't yeild events anymore.

jonastheis · 2025-10-08T02:05:34Z

crates/node/src/args.rs

+    ChainOrchestrator, ChainOrchestratorConfig, ChainOrchestratorHandle, Consensus, NoopConsensus,
+    SystemContractConsensus,
 };
+// use rollup_node_manager::{


jonastheis · 2025-10-08T02:06:15Z

crates/node/src/args.rs

-                number: 0,
-            });
-        }
+        // if let Some(block_info) = startup_safe_block {


why is this commented out?

I've removed it, we no longer need it as now we include the block number associated with derived attributes which allows us to do our reconciliation. Previously we were relying on the safe_block_numer to do the association which was messy and error prone.

jonastheis · 2025-10-08T02:07:27Z

crates/node/src/args.rs

-            self.sequencer_args.allow_empty_blocks,
-        );
+        let engine = Engine::new(Arc::new(engine_api), fcs);
+        // let engine = EngineDriver::new(


why commented?

jonastheis · 2025-10-08T23:47:00Z

crates/database/db/src/operations.rs

-            .stream(self.get_connection())
-            .await?
-            .map(|res| Ok(res.map(Into::into)?)))
+            Some(L1MessageKey::BlockNumber(block_number)) => {


there is a lot of stuff happening in this function and it would be great to add some comments as to what on a high-level is happening in each branch and why.

jonastheis · 2025-10-09T03:42:26Z

crates/chain-orchestrator/src/lib.rs

-
-            return Err(ChainOrchestratorError::ChainInconsistency);
+    // /// Wraps a pending chain orchestrator future, metering the completion of it.
+    // pub fn handle_metered(


why is this commented out?

jonastheis · 2025-10-09T04:02:45Z

crates/chain-orchestrator/src/lib.rs

-        soft_limit: usize,
-    }
+        // If the block number is greater than the current head we attempt to extend the chain.
+        let mut new_headers = if received_block_number > self.engine.fcs().head_block_info().number


Suggested change

let mut new_headers = if received_block_number > self.engine.fcs().head_block_info().number

let mut new_headers = if received_block_number > current_head_number

jonastheis · 2025-10-09T04:15:11Z

crates/chain-orchestrator/src/lib.rs

+                .ok_or(ChainOrchestratorError::L2BlockNotFoundInL2Client(received_block_number))?;
+
+            if current_chain_block.header.hash_slow() == received_block_hash {
+                tracing::debug!(target: "scroll::chain_orchestrator", ?received_block_hash, ?received_block_number, "Received block from peer that is already in the chain");


sure we only want to log this in debug?

jonastheis · 2025-10-09T04:15:33Z

crates/chain-orchestrator/src/lib.rs

+            // Assert that we are not reorging below the safe head.
+            let current_safe_info = self.engine.fcs().safe_block_info();
+            if received_block_number <= current_safe_info.number {
+                tracing::debug!(target: "scroll::chain_orchestrator", ?received_block_hash, ?received_block_number, current_safe_info = ?self.engine.fcs().safe_block_info(), "Received block from peer that would reorg below the safe head - ignoring");


sure we only want to log this in debug?

jonastheis · 2025-10-09T04:27:47Z

crates/chain-orchestrator/src/lib.rs

-        let mut bytes = [0u8; 1024];
-        rand::rng().fill(bytes.as_mut_slice());
-        let mut u = Unstructured::new(&bytes);
+            // Check if the parent hash of the received block is in the chain.


Isn't this just a reorg of depth 1? Shouldn't this case also be handled by the reorg logic below? I think the code flow here could be a bit better to make it clearer which conditions are met and which path is taken. especially in the reorg case and with the fork-choice condition if block_with_peer.block.header.timestamp <= current_head.header.timestamp {

It's not a reorg of depth one, it's a reorg of arbitrary depth, i.e. the new chain has length one, but the depth is arbitrary. As a consequence of this comment, I added a check to ensure that the depth would not result in a safe block reorg. You are correct that we could delegate this to the reorg logic below, but it seems inefficient and a waste, as we already have all the information we need to reconcile the reorg. With some refactoring, I agree we could combine this condition and the reorg logic below in a more readable and efficient manner. For now, I think it's pragmatic to keep it as is.

jonastheis · 2025-10-09T06:05:11Z

crates/chain-orchestrator/src/lib.rs

+        // If the received block number has a block number greater than the current head by more
+        // than the optimistic sync threshold, we optimistically sync the chain.
+        if received_block_number > current_head_number + self.config.optimistic_sync_threshold() {
+            tracing::trace!(target: "scroll::chain_orchestrator", ?received_block_number, ?current_head_number, "Received new block from peer with block number greater than current head by more than the optimistic sync threshold");


here we start optimistic sync but also do the other consolidation. is that intended?

good catch, fixed.

jonastheis · 2025-10-09T07:58:45Z

crates/chain-orchestrator/src/lib.rs

-                // Safe head should be the highest block from batch index <= 100
-                assert_eq!(safe_head, Some(block_1.block_info));
+        // Persist the mapping of L1 messages to L2 blocks such that we can react to L1 reorgs.
+        let blocks = chain.iter().map(|block| block.into()).collect::<Vec<_>>();


Is this a valid operation in optimistic sync mode? what if the L1 messages contained in the chain are garbage?

I've updated the logic such that now we only persist and gossip blocks if they have been validated and we have fully synced L1 / L2 and consolidated the chain.

jonastheis · 2025-10-09T07:59:21Z

crates/chain-orchestrator/src/lib.rs

+
+        // If we were previously in L2 syncing mode and the FCS update resulted in a valid state, we
+        // transition the L2 sync state to synced and consolidate the chain.
+        if result.is_valid() && self.sync_state.l2().is_syncing() {


do we need to check if the result is valid? above we already check whether it is invalid and return

Yes, because there is also the case that it could be Syncing, so in that case, we will want to defer until a later point at which we've fully synced.

jonastheis · 2025-10-09T08:00:48Z

crates/chain-orchestrator/src/lib.rs

+        // Persist the signature for the block and notify the network manager of a successful
+        // import.
+        let tx = self.database.tx_mut().await?;
+        tx.insert_signature(chain_head_hash, block_with_peer.signature).await?;


don't we already persist the signature in handle_block_from_peer

Good catch, I've removed persisting the signature here

jonastheis · 2025-10-09T09:02:18Z

crates/chain-orchestrator/src/lib.rs

+
+            // If the received and expected L1 messages do not match return an error.
+            if message_hash != expected_hash {
+                self.notify(ChainOrchestratorEvent::L1MessageMismatch {


How do we currently react to this event?

The event itself is exclusively used for testing.

greged93

Great refactor, this is soooo much easier to read and nicer to go through then the previous state of the orchestrator and even node in general!

Left some inline comments and a small nit.

greged93 · 2025-10-09T10:39:27Z

crates/chain-orchestrator/src/consolidation.rs

+            if block_matches_attributes(
+                &attributes.attributes,
+                &current_block,
+                current_block.parent_hash,


I think this can go, this check was used before in order to check that the block we received from the L2 was the child block of the safe head in the Engine Driver. Here all we are doing is check block.parent_hash == block.parent_hash.

greged93 · 2025-10-09T14:47:32Z

crates/database/db/src/operations.rs

+                BlockConsolidationOutcome::Consolidated(block_info) => {
+                    self.insert_block(block_info, outcome.batch_info).await?;
+                }
+                BlockConsolidationOutcome::Skipped(block_info) => {
+                    // No action needed, the block has already been previously consolidated however
+                    // we will insert it again defensively
+                    self.insert_block(block_info, outcome.batch_info).await?;
+                }


nit: this can collapsed into one arm

greged93 · 2025-10-09T15:36:47Z

crates/engine/src/engine.rs

+        let result =
+            self.client.fork_choice_updated_v1(fcs.get_alloy_fcs(), Some(attributes)).await?;


small note here: I think this works in the case of Reth because payloads built from attributes are automatically inserted here.

One concern we might have which isn't handled here but mentioned in the Op stack docs, is the case where the data from the batch contains invalid transaction data and the execution node fails to build a payload. I believe in this case, the result we get here would be valid, but trying to call get_payload(id) would return an error.

This is an important nuance. This will have implications in the Reorg branch of the consolidation logic:

rollup-node/crates/chain-orchestrator/src/lib.rs

Lines 425 to 469 in 44424d6

BlockConsolidationAction::Reorg(attributes) => {

tracing::info!(target: "scroll::chain_orchestrator", block_number = ?attributes.block_number, "Reorging chain to derived block");

// We reorg the head to the safe block and then build the payload for the

// attributes.

let head = *self.engine.fcs().safe_block_info();

if head.number != attributes.block_number - 1 {

return Err(ChainOrchestratorError::InvalidBatchReorg {

batch_info,

safe_block_number: head.number,

derived_block_number: attributes.block_number,

});

}

let fcu = self.engine.build_payload(Some(head), attributes.attributes).await?;

let payload = self

.engine

.get_payload(fcu.payload_id.expect("payload_id can not be None"))

.await?;

let block: ScrollBlock = try_into_block(

ExecutionData { payload: payload.into(), sidecar: Default::default() },

self.config.chain_spec().clone(),

)

.expect("block must be valid");

let result = self.engine.new_payload(&block).await?;

if result.is_invalid() {

return Err(ChainOrchestratorError::InvalidBatch(

(&block).into(),

batch_info,

));

}

// Update the forkchoice state to the new head.

let block_info: L2BlockInfoWithL1Messages = (&block).into();

self.engine

.update_fcs(

Some(block_info.block_info),

Some(block_info.block_info),

Some(block_info.block_info),

)

.await?;

reorg_results.push(block_info.clone());

BlockConsolidationOutcome::Reorged(block_info)

}

};

Whilst this is an important nuance, I consider accounting for corrupt transaction data to be out of scope of this PR due to the fact that batch submission is permissioned (in the happy case). I propose that we create an issue to track this and address this in a future PR, possibly in the context of a ninfallible derivation pipeline (which we currently don't have).

What do you think?

I propose that we create an issue to track this and address this in a future PR, possibly in the context of a ninfallible derivation pipeline (which we currently don't have).

Agreed, let's track it and leave as is for now.

crates/providers/src/l1/message.rs

greged93 · 2025-10-09T15:54:04Z

crates/sequencer/src/lib.rs

+        // If there is an inflight payload building job, poll it.
+        if let Some(payload_building_job) = this.payload_building_job.as_mut() {
+            match payload_building_job.future.as_mut().poll(cx) {
+                Poll::Ready(payload_id) => {
+                    this.payload_building_job = None;
+                    return Poll::Ready(Some(SequencerEvent::PayloadReady(payload_id)));
+                }
+                Poll::Pending => {}
+            }


Should the payload_building_job have higher priority in the polling order? If the payload is ready and the trigger as well, the current order means we decide to skip the next slot. If we invert them, we would return the payload to the chain orchestrator, and would catch the trigger on the next polling (might be a little late, but at least we won't completely miss it).

greged93

One additional comment

greged93 · 2025-10-13T10:25:54Z

crates/chain-orchestrator/src/lib.rs

+            // Persist the signature for the block and notify the network manager of a successful
+            // import.
+            let tx = self.database.tx_mut().await?;
+            tx.insert_signature(chain_head_hash, block_with_peer.signature).await?;
+            tx.commit().await?;


I think the signature is already persisted in handle_block_from_peer, which is the only place where this method is called.

greged93 · 2025-10-13T10:49:59Z

crates/engine/src/engine.rs

+        let result =
+            self.client.fork_choice_updated_v1(fcs.get_alloy_fcs(), Some(attributes)).await?;


I propose that we create an issue to track this and address this in a future PR, possibly in the context of a ninfallible derivation pipeline (which we currently don't have).

Agreed, let's track it and leave as is for now.

greged93

no further comment, lgtm!

frisitano added 3 commits October 5, 2025 00:24

atomic rollup node

8bf7f2d

Merge branch 'main' into refactor/rollup-node-refactor

e24e516

atomic rollup node

c3f113e

frisitano marked this pull request as draft October 7, 2025 07:10

frisitano added 5 commits October 8, 2025 08:47

atomic rollup node

3708f68

lint

40b6716

add status rpc

5b06475

fork choice fix and test coverage

6351f28

add logs and optimise block handling in fork choice logic

355998a

frisitano marked this pull request as ready for review October 8, 2025 17:15

frisitano added 3 commits October 9, 2025 02:09

update rpc and default optimistic sync threshold

62edb0b

Merge branch 'main' into refactor/rollup-node-refactor

17d0e65

commit merge files

46ebde7

frisitano mentioned this pull request Oct 8, 2025

[lint] Enable cargo docs for rollup-node-chain-orchestrator #353

Open

jonastheis reviewed Oct 9, 2025

View reviewed changes

frisitano requested a review from greged93 October 9, 2025 05:36

jonastheis reviewed Oct 9, 2025

View reviewed changes

fix derivation pipeline persisting L1 mesages and address comments

4309fb1

greged93 reviewed Oct 9, 2025

View reviewed changes

frisitano added 4 commits October 11, 2025 18:40

add semaphore for database read transaction limiting

d7fa9ec

add L1 message finalized with depth inclusion rule

52f5b24

introduce DerivationPipelineWorker

b82f7e0

add derivation pipeline worker concurrency limits

44424d6

greged93 reviewed Oct 13, 2025

View reviewed changes

address feedback

6ff7d55

frisitano requested a review from greged93 October 13, 2025 12:12

greged93 approved these changes Oct 13, 2025

View reviewed changes

frisitano merged commit 5472bfd into main Oct 13, 2025
14 of 15 checks passed

frisitano deleted the refactor/rollup-node-refactor branch October 13, 2025 13:47

	let mut new_headers = if received_block_number > self.engine.fcs().head_block_info().number
	let mut new_headers = if received_block_number > current_head_number

		let result =
		self.client.fork_choice_updated_v1(fcs.get_alloy_fcs(), Some(attributes)).await?;

	BlockConsolidationAction::Reorg(attributes) => {
	tracing::info!(target: "scroll::chain_orchestrator", block_number = ?attributes.block_number, "Reorging chain to derived block");
	// We reorg the head to the safe block and then build the payload for the
	// attributes.
	let head = *self.engine.fcs().safe_block_info();
	if head.number != attributes.block_number - 1 {
	return Err(ChainOrchestratorError::InvalidBatchReorg {
	batch_info,
	safe_block_number: head.number,
	derived_block_number: attributes.block_number,
	});
	}
	let fcu = self.engine.build_payload(Some(head), attributes.attributes).await?;
	let payload = self
	.engine
	.get_payload(fcu.payload_id.expect("payload_id can not be None"))
	.await?;
	let block: ScrollBlock = try_into_block(
	ExecutionData { payload: payload.into(), sidecar: Default::default() },
	self.config.chain_spec().clone(),
	)
	.expect("block must be valid");

	let result = self.engine.new_payload(&block).await?;
	if result.is_invalid() {
	return Err(ChainOrchestratorError::InvalidBatch(
	(&block).into(),
	batch_info,
	));
	}

	// Update the forkchoice state to the new head.
	let block_info: L2BlockInfoWithL1Messages = (&block).into();
	self.engine
	.update_fcs(
	Some(block_info.block_info),
	Some(block_info.block_info),
	Some(block_info.block_info),
	)
	.await?;

	reorg_results.push(block_info.clone());
	BlockConsolidationOutcome::Reorged(block_info)
	}
	};

Refactor/rollup node refactor #351

Refactor/rollup node refactor #351

Uh oh!

Conversation

frisitano commented Oct 7, 2025

Uh oh!

codspeed-hq bot commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging #351 will degrade performances by 97.16%

Summary

Benchmarks breakdown

Uh oh!

jonastheis left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

greged93 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codspeed-hq bot commented Oct 8, 2025 •

edited

Loading