Skip to content

fix: dropping lagging pubsub subscribers#1814

Merged
librelois merged 3 commits intomasterfrom
elois/reorg-backpressure
Feb 10, 2026
Merged

fix: dropping lagging pubsub subscribers#1814
librelois merged 3 commits intomasterfrom
elois/reorg-backpressure

Conversation

@librelois
Copy link
Copy Markdown
Member

@librelois librelois commented Feb 9, 2026

⚠️ Breaking changes ⚠️

  • eth_subscribe(newHeads) subscribers can now be dropped when they fall behind and exceed the pending notification cap (default: 512).
  • New node CLI option: --pubsub-max-pending-notifications (default 512) controls that cap.
  • Operators with slow consumers may need to tune this value upward to reduce forced disconnects.

Goal of the changes

Prevent WS/pubsub backlog growth from causing connection exhaustion while preserving Ethereum reorg-compliant newHeads behavior.

What reviewers need to know

  • newHeads reorg emission was refactored to stream notifications lazily instead of building Vec<PubSubResult> per event:
    • client/rpc/src/eth_pubsub.rs
  • Reorg metadata fan-out now uses shared ownership to avoid repeated deep clones:
    • Option<Arc<ReorgInfo<_>>> in mapping-sync notification flow
    • client/mapping-sync/src/lib.rs
    • client/mapping-sync/src/kv/worker.rs
    • client/mapping-sync/src/sql/mod.rs
  • Added backpressure protection for pubsub sinks:
    • lagging subscribers are closed/evicted when pending queue length exceeds configured cap
    • client/mapping-sync/src/lib.rs
  • Added runtime configuration hook and wired it from node CLI:
    • new EthConfiguration.pubsub_max_pending_notifications
    • template/node/src/eth.rs
    • template/node/src/service.rs
  • Drop logs for lagging subscribers are debug level to avoid production log noise.
  • Fixed eth_subscribe(syncing) subscription lifecycle on disconnect:
    • if subscription.send(...) fails, the task now exits (return for initial send, break in loop)
    • prevents disconnected syncing subscribers from leaving a long-lived task behind
    • client/rpc/src/eth_pubsub.rs

@librelois librelois requested a review from sorpaas as a code owner February 9, 2026 16:16
Copy link
Copy Markdown
Contributor

@manuelmauro manuelmauro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Copy Markdown
Collaborator

@arturgontijo arturgontijo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@librelois librelois added this pull request to the merge queue Feb 10, 2026
librelois added a commit to moonbeam-foundation/frontier that referenced this pull request Feb 10, 2026
* fix: dropping lagging pubsub subscribers

* Fix eth_subscribe(syncing) subscription lifecycle on disconnect
librelois added a commit to moonbeam-foundation/frontier that referenced this pull request Feb 10, 2026
* fix: dropping lagging pubsub subscribers

* Fix eth_subscribe(syncing) subscription lifecycle on disconnect
@librelois librelois removed this pull request from the merge queue due to a manual request Feb 10, 2026
@librelois librelois merged commit b2ed493 into master Feb 10, 2026
6 checks passed
@librelois librelois deleted the elois/reorg-backpressure branch February 10, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants