Bridges - Add improved congestion control mechanism by bkontur · Pull Request #6231 · paritytech/polkadot-sdk

bkontur · 2024-10-25T08:51:14Z

Closes: #5551
Closes: #5550

Context

Before permissionless lanes, bridges only supported hard-coded, static lanes. The congestion mechanism was based on sending Transact(report_bridge_status(is_congested)) from pallet-xcm-bridge-hub to pallet-xcm-bridge-hub-router. Depending on is_congested, we adjusted the fee factor to increase or decrease fees. This congestion mechanism relied on monitoring XCMP queues, which could cause issues like suspending the entire XCMP queue rather than just the affected bridge.

Additionally, we are progressing with deploying bridge message pallets/routing directly on AssetHub, where we don’t interact with XCMP to perform ExportXcm locally.

Description

This PR introduces two new pallets pallet-xcm-bridge and pallet-xcm-bridge-router. These pallets are designed to improve congestion handling for bridges. The main objective of this PR is to enhance the routing and management of messages between chains.

This PR re-introduces and improves congestion for bridges:

Enhanced Bridge Congestion Mechanism: The bridge queue mechanism has been restructured to operate independently of XCMP, with a refined protocol for congestion detection and suspension management (congested_threshold and stop_threshold).
Bridge-Specific Channel Suspension: pallet-xcm-bridge and pallet-xcm-bridge-router now use BridgeId to identify specific bridges, enabling selective suspension and resumption of individual bridge channels.
Dynamic Congestion Detection: pallet-xcm-bridge now includes callbacks for fn suspend_bridge and fn resume_bridge based on congestion status:
- For sibling chains, the pallet-xcm-bridges exporter can send xcm::Transact(update_bridge_status(bridge_id, is_congested)) using the stored callback information.
- For local chain deployments, the pallet-xcm-bridge's exporter manages state directly.
New Stop Threshold: A stop_threshold limit in pallet-xcm-bridge enables or disables ExportXcm::validate, providing a fallback mechanism when the router does not adhere to the suspend signal.
Flexible Message Routing: pallet-xcm-bridge-router has support for message routing for both sibling chains (ExportMessage - ViaRemoteBridgeExporter) and local deployment (ExportXcm - ViaLocalBridgeExporter).

Open questions

when the router receives is_congested = false with fn do_update_bridge_status(, can we directly remove it (its fee factor) from Bridges and the next message fee wont be affected by increased fee factor or slowly start decreasing fee factor on idle (this is how it is implemented now)? Original comment
Report congestion detection uses bool update_bridge_status(bool), but shouldn't we better send the enum BridgeState {Opened, SoftSuspended, HardSuspended, Closed}? And on HardSuspended stop the pallet-xcm-bridge-router?

bkontur · 2024-10-25T11:25:35Z

bot fmt
/cmd prdoc --audience runtime_dev --bump patch

prdoc/pr_6231.prdoc

bkontur · 2024-10-26T20:30:05Z

bot fmt

bkontur · 2024-10-28T15:04:20Z

bot fmt

bkontur · 2024-11-05T12:40:39Z

bot fmt

bkontur · 2024-11-07T15:57:44Z

/cmd bench --runtime asset-hub-westend asset-hub-rococo --pallet pallet_xcm_bridge_hub_router

bkontur · 2024-11-07T16:18:32Z

bot bench cumulus-assets --runtime=asset-hub-westend --pallet=pallet_xcm_bridge_hub_router
bot bench cumulus-assets --runtime=asset-hub-rococo --pallet=pallet_xcm_bridge_hub_router

bkontur · 2024-11-07T16:51:47Z

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-assets --runtime=asset-hub-westend --pallet=pallet_xcm_bridge_hub_router
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-assets --runtime=asset-hub-rococo --pallet=pallet_xcm_bridge_hub_router

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages

bkontur · 2024-11-07T21:51:26Z

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --subcommand=xcm --runtime=bridge-hub-rococo --pallet=pallet_xcm_benchmarks::generic
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --subcommand=xcm --runtime=bridge-hub-westend --pallet=pallet_xcm_benchmarks::generic

bkontur · 2024-11-08T23:55:04Z

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_messages
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_xcm_bridge_hub
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_xcm_bridge_hub

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --subcommand=xcm --runtime=bridge-hub-rococo --pallet=pallet_xcm_benchmarks::generic
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-bridge-hubs --subcommand=xcm --runtime=bridge-hub-westend --pallet=pallet_xcm_benchmarks::generic

bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-assets --runtime=asset-hub-westend --pallet=pallet_xcm_bridge_hub_router
bot bench -v PIPELINE_SCRIPTS_REF=bko-fix cumulus-assets --runtime=asset-hub-rococo --pallet=pallet_xcm_bridge_hub_router

bkontur · 2025-05-07T13:00:10Z

The 2 comments so far seem like they can be addressed separately, so if you think they make sense, I would propose, for the beginning to address them in a separate PR which precedes this. And to rebase this PR on top of that one. After that I will continue with the review.

@serban300 I created separate issue for those comments and they are up to you :)

Personally, I would prefer to review and merge this PR incrementally since it's very large. My concern is that reviewing 9k lines at once can lead to missing things.

Let me try to split this big PR, but till that time, please you could help with review for other smaller ones:

Setup permissionless lanes for AssetHubs #6675
Sets up bridging for AssetHubs and enables relaying of multiple parachains on BridgeHubs.
Setup permissionless lanes for AssetHubs - heads/state-roots store #8324
Introduces pallet-bridge-proof-root-store and deploys it on AssetHubs.
Setup permissionless lanes for AssetHubs - added OnNewHead to pallet-bridge-parachains #8325
Adds the OnNewHead hook to pallet-bridge-parachains.
Setup permissionless lanes for AssetHubs - heads/state-roots sync #8326
Introduces the pallet-bridge-proof-root-sync mechanism and deploys it on BridgeHubs.
Setup permissionless lanes for AssetHubs - local dispatch on AH #8368
Implements MessageQueueRouterFor for SendXcm, allowing messages from pallet-bridge-messages to be dispatched for local chain execution via pallet-message-queue.

Especially, the 8324/8325/8326/8368 could possibly go directly to the master (+backports)

Fixing #8215 based on #8185: Improve try-state for pallet-xcm-bridge It removes try_as and uses try_into implementation instead. After adding test case where VersionedLocation* to have (XCM_VERSION - 1) is , the test case fails. ![after-adding-test-before-fix](https://github.com/user-attachments/assets/0672c677-fc50-4b1e-b636-7b6f23e4092d) After adding the fix, by removing try_as and replacing with try_into, test case passed: ![after-adding-fix](https://github.com/user-attachments/assets/18452881-23e5-4563-b08f-47d4e0ef4be3)

Fixing #8215 based on #8185: Improve try-state for pallet-xcm-bridge It removes try_as and uses try_into implementation instead. --------- Co-authored-by: Branislav Kontur <[email protected]>

bkontur · 2025-05-26T11:36:22Z

/cmd fmt

prdoc/pr_6231.prdoc

…stion

serban300 · 2025-07-17T13:48:30Z

bridges/modules/xcm-bridge-router/src/lib.rs

+					let message_size_factor =
+						FixedU128::from_u32(message_size.saturating_div(1024))
+							.saturating_mul(MESSAGE_SIZE_FEE_BASE);
+					let total_factor = EXPONENTIAL_FEE_BASE.saturating_add(message_size_factor);
+
+					let previous_factor = bridge_state.delivery_fee_factor;
+					bridge_state.delivery_fee_factor =
+						bridge_state.delivery_fee_factor.saturating_mul(total_factor);


Will integrate the refactored FeeTracker here after we sort out all the other comments related to this pallet.

bridges/modules/xcm-bridge-router/src/lib.rs

serban300 · 2025-07-17T14:06:48Z

bridges/modules/xcm-bridge-router/src/lib.rs

+
+			// handle congestion and fee factor (if detected)
+			Bridges::<T, I>::mutate_exists(&bridge_id, |bridge_state| match bridge_state {
+				Some(ref mut bridge_state) if bridge_state.is_congested => {


Is is_congested actually needed ? Looks like we could just increase the fee factor here and set it to MINIMAL_DELIVERY_FEE_FACTOR on do_update_bridge_status(is_congested: false).

I'm just wondering if it would be good to have it for the future or for easier tracking of the bridge status.

Is is_congested actually needed ? Looks like we could just increase the fee factor here and set it to MINIMAL_DELIVERY_FEE_FACTOR on do_update_bridge_status(is_congested: false).

I'm just wondering if it would be good to have it for the future or for easier tracking of the bridge status.

If you check description, there is an open question about changing that bool to BridgeState.

Report congestion detection uses bool update_bridge_status(bool), but shouldn't we better send the enum BridgeState {Opened, SoftSuspended, HardSuspended, Closed}? And on HardSuspended stop the pallet-xcm-bridge-router?

So, if we want to stop the router (and prevent dropping), we need to change bool to BridgeState. But if we don't want to stop the router and only rely on increasing the fee factor, then yes — we don't even need to store is_congested.

@serban300 what do you think and/or suggest?

Hmmm I don't know what would be best. Thinking.

@serban300

cc: @serban300 --------- Co-authored-by: Serban Iorga <[email protected]>

@serban300

The relevant change, which allows us to do dynamic pricing in `pallet-xcm-bridge` based on the router configuration and bridge status: ```rust - /// Price of single message export to the bridged consensus (`Self::BridgedNetwork`). - type MessageExportPrice: Get<Assets>; + /// Price of a single message export to the bridged consensus (`Self::BridgedNetwork`). + type MessageExportPrice: PriceForMessageDelivery<Id = BridgeId>; ``` Basically, with message bridging on AssetHub, the same dynamic price (based on the bridge congestion status) is calculated for exporting either via: - local routing using `ToRococoOverAssetHubRococoXcmRouter` / `ToWestendOverAssetHubWestendXcmRouter` (AH<>AH transfer) - executing `ExportMessage` from a sibling chain. It is just backport of xcm-bridge/xcm-bridge-router stuff from this commit: 8ada805 (where you can see also the real usage and setup) cc: @serban300

Fixing comments in PR #6231 - changed DeliveryFeeFactorUpdated #6231 (comment) - changed Bridges with ValueQuery with defaults #6231 (comment) - updated message_size to be u32 #6231 (comment) --------- Co-authored-by: Branislav Kontur <[email protected]>

paritytech-workflow-stopper · 2025-07-21T22:45:56Z

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/16429704449
Failed job name: fmt

xcm-bridge-router simplifications

bkontur added the T15-bridges This PR/Issue is related to bridges. label Oct 25, 2024

bkontur self-assigned this Oct 25, 2024

bkontur force-pushed the bko-bridges-congestion branch 2 times, most recently from 659be89 to b48b8a5 Compare October 25, 2024 21:14

bkontur commented Oct 26, 2024

View reviewed changes

prdoc/pr_6231.prdoc Show resolved Hide resolved

bkontur force-pushed the bko-bridges-congestion branch from a663bc2 to cbc6ae7 Compare October 26, 2024 20:29

bkontur force-pushed the bko-bridges-congestion branch from c78f9bc to 501a5c0 Compare October 28, 2024 15:01

bkontur force-pushed the bko-bridges-congestion branch 7 times, most recently from c78e707 to 152389a Compare November 5, 2024 12:33

bkontur force-pushed the bko-bridges-congestion branch 3 times, most recently from edd9c5c to 38f1bb3 Compare November 7, 2024 13:33

bkontur force-pushed the bko-bridges-congestion branch from f06433a to d329dec Compare November 7, 2024 16:44

bkontur force-pushed the bko-bridges-congestion branch from 21baa7f to c00ff6b Compare November 8, 2024 17:44

bkontur added the A4-needs-backport label Nov 11, 2024

Nits

ad0931c

bkontur mentioned this pull request May 7, 2025

Refactor and reuse QueueConfigData instead of CongestionLimits for pallet-xcm-bridge #8463

Open

bkontur mentioned this pull request May 9, 2025

FeeTracker deduplications #8477

Merged

paritytech-review-bot bot requested a review from a team May 22, 2025 10:26

bkontur mentioned this pull request May 22, 2025

[Bridges] Setup permissionless lanes for AssetHubs bkontur/runtimes#14

Open

2 tasks

rosarp and others added 2 commits May 22, 2025 21:57

Bridges: Fix - Improve try-state for pallet-xcm-bridge (#8618)

f3b23c7

Fixing #8215 based on #8185: Improve try-state for pallet-xcm-bridge It removes try_as and uses try_into implementation instead. --------- Co-authored-by: Branislav Kontur <[email protected]>

Merge remote-tracking branch 'origin/master' into bko-bridges-congestion

6f2ee64

github-actions bot and others added 5 commits May 26, 2025 11:39

Update from github-actions[bot] running command 'fmt'

c94caf0

Revert BHs unrelated changes

230bd04

prdoc suggestions

e64ba53

licence headers

3aaf599

Merge remote-tracking branch 'origin/master' into bko-bridges-congestion

8f83793

bkontur commented May 28, 2025

View reviewed changes

prdoc/pr_6231.prdoc Outdated Show resolved Hide resolved

bkontur added 2 commits May 28, 2025 16:04

Update prdoc/pr_6231.prdoc

6529de6

Merge remote-tracking branch 'origin/master' into bko-bridges-congestion

49410f9

bkontur mentioned this pull request Jul 15, 2025

Add benchmarks for pallet-xcm-bridge-hub #5550

Closed

Merge remote-tracking branch 'upstream/master' into bko-bridges-conge…

07faafc

…stion

serban300 reviewed Jul 17, 2025

View reviewed changes

WIP: apply_dynamic_fee_factor with asset: &mut Asset (#9259)

fd1c1ba

cc: @serban300 --------- Co-authored-by: Serban Iorga <[email protected]>

paritytech-review-bot bot requested a review from a team July 18, 2025 11:21

serban300 mentioned this pull request Jul 18, 2025

WIP: Allow dynamic fees setup for xcm-bridge #9260

Merged

rosarp mentioned this pull request Jul 18, 2025

PR#6231 fixed according to comments #9268

Merged

rosarp and others added 2 commits July 22, 2025 00:00

Merge remote-tracking branch 'origin/master' into bko-bridges-congestion

ecd955a

serban300 and others added 2 commits July 30, 2025 18:13

Fix build

8e1efac

xcm-bridge-router simplifications (#9397)

9d65de2

xcm-bridge-router simplifications

Conversation

bkontur commented Oct 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Description

Open questions

Uh oh!

bkontur commented Oct 25, 2024

Uh oh!

Uh oh!

bkontur commented Oct 26, 2024

Uh oh!

bkontur commented Oct 28, 2024

Uh oh!

bkontur commented Nov 5, 2024

Uh oh!

bkontur commented Nov 7, 2024

Uh oh!

bkontur commented Nov 7, 2024

Uh oh!

bkontur commented Nov 7, 2024

Uh oh!

bkontur commented Nov 7, 2024

Uh oh!

bkontur commented Nov 8, 2024

Uh oh!

bkontur commented May 7, 2025

Uh oh!

bkontur commented May 26, 2025

Uh oh!

Uh oh!

serban300 Jul 17, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

serban300 Jul 17, 2025

Choose a reason for hiding this comment

Uh oh!

bkontur Jul 18, 2025

Choose a reason for hiding this comment

Uh oh!

serban300 Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

paritytech-workflow-stopper bot commented Jul 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

bkontur commented Oct 25, 2024 •

edited

Loading