feat(nat): rotate hole-punch coordinators with capped per-attempt timeout

[mickvandijke]

Summary

• Replaces the single-coordinator hole-punch hint with a ranked list (Tier 2): callers supply best-first, the connect loop rotates through them with a short per-attempt timeout (1.5s) for non-final candidates and the strategy's full timeout for the last. Worst-case wait drops from K * 8s to (K-1) * 1.5s + 8s.
• Adds coordinator-side back-pressure (Tier 4 lite): BootstrapCoordinator tracks active relay slots keyed by (initiator_peer_id, target_peer_id) and silently refuses incoming PUNCH_ME_NOW relays once at capacity. Inline garbage-collection sweep reclaims stale slots so ghost sessions cannot leak the counter. Configurable via two new NatTraversalConfig fields: coordinator_max_active_relays (default 32) and coordinator_relay_slot_timeout (default 5s).
• No wire-protocol changes — back-pressure is purely timing-based (silent refusal triggers Tier 2 rotation on the initiator). Backwards-compatible with older peers.

Why

Today every cold-starting peer pins one of the small set of bootstrap nodes (saorsa-2 / saorsa-3) as its hole-punch coordinator, concentrating active coordinator load on infrastructure that is already serving DHT traffic. With this change a single overloaded coordinator stops blocking the dial: the initiator times out fast (1.5s) and rotates to the next preferred candidate, and the coordinator itself sheds excess load by silently refusing relays once it has 32 in flight.

Design choice

Considered an explicit `HOLE_PUNCH_COORDINATOR_BUSY` wire frame for instant initiator notification but the cost — new frame type, codec, parser dispatch, version compatibility — was not justified. Timing-based back-pressure wastes ~1.5s per busy coordinator but ships in roughly 1/3 the code with zero protocol surface and is fully backwards-compatible.

Test plan

• `cargo fmt --all -- --check` clean
• `cargo clippy --all-targets -- -D warnings -D clippy::unwrap_used -D clippy::expect_used` clean
• `cargo test --lib` passes (1480 tests, +10 new)
• 5 new unit tests for `P2pEndpoint::merge_preferred_coordinators`: empty preferred no-op, front insertion in order, dedup of pre-existing entries, non-preferred entries keep their original order, empty candidate list
• 5 new unit tests for `BootstrapCoordinator` back-pressure: under-capacity accept, at-capacity silent refuse with stat increment, slot re-arm without consuming additional capacity, sweep reclaims stale slots, non-relay frames do not consume slots
• No breaking API changes — old `set_hole_punch_preferred_coordinator` is retained as a thin wrapper around the new list form

Companion PR

Second of three PRs landing smarter hole-punch coordinator selection. The first (saorsa-labs/saorsa-core#TBD) added round-aware referrer ranking in the DHT layer and is independent of this one. The third will wire saorsa-core to call the new `set_hole_punch_preferred_coordinators` API once this merges.

🤖 Generated with Claude Code

Greptile Summary

This PR implements two back-pressure improvements to hole-punch coordinator selection: a ranked coordinator list with per-attempt rotation (Tier 2) where non-final candidates get a 1.5s quick timeout before advancing to the next, and a node-wide RelaySlotTable (Tier 4 lite) that caps simultaneous in-flight (initiator_addr, target_peer_id) relay sessions and silently refuses when at capacity. The implementation is correct, well-documented, and well-tested (10 new unit tests), and the previously flagged concern about P2pConfig::to_nat_config() not propagating the new fields has been addressed — both coordinator_max_active_relays and coordinator_relay_slot_idle_timeout flow correctly through NatConfig → to_nat_config() → NatTraversalConfig → RelaySlotTable::new.

Confidence Score: 5/5

Safe to merge — no P0 or P1 issues found; implementation is correct, well-tested, and well-documented.

All 10 changed/new files reviewed thoroughly. The relay slot table logic, rotation bounds, lock ordering, config propagation, and Drop-based cleanup are all correct. Prior review concerns about to_nat_config() propagation were addressed. Remaining observations (duplicate-preferred inefficiency, stale relay_initiator_addr on QUIC migration) are P2 edge cases bounded by the 5s idle timeout and do not affect correctness or reliability in normal operation.

No files require special attention.

Important Files Changed

Filename	Overview
src/relay_slot_table.rs	New node-wide back-pressure table; lock-drop-before-warn pattern is correct, amortized sweep bounds per-frame cost, 6 focused inline tests cover all code paths
src/p2p_endpoint.rs	Adds merge_preferred_coordinators (O(N+M), correct dedup), rotation index arithmetic is bounds-safe, debug_assert guards sync between index and strategy coordinator, 5 new unit tests
src/connection/nat_traversal.rs	BootstrapCoordinator correctly threads relay_slot_table Arc; Drop releases initiator slots; relay_initiator_addr captured once (stale on migration, but bounded by 5s idle timeout)
src/nat_traversal_api.rs	New config fields with serde defaults and validation; single RelaySlotTable Arc shared via clone across both server and client TransportConfig — cap is correctly node-wide
src/unified_config.rs	NatConfig gains both new fields with correct defaults; to_nat_config() propagates them to NatTraversalConfig (addresses prior review concern)
src/config/transport.rs	relay_slot_table: Option<Arc<RelaySlotTable>> added to TransportConfig; defaults to None for Quinn-style fixtures that bypass P2pEndpoint
src/connection/mod.rs	Both NatTraversalState::new call sites correctly thread self.config.relay_slot_table.clone() from the frozen config
src/lib.rs	Exposes pub mod relay_slot_table; no other changes
tests/relay_queue_tests.rs	Existing test struct literals updated with new NatTraversalConfig fields; no logic changes
tests/security_regression_tests.rs	Existing test helpers updated with new NatTraversalConfig fields; no logic changes

Sequence Diagram

sequenceDiagram
    participant I as Initiator
    participant C1 as Coordinator 1 (preferred)
    participant C2 as Coordinator 2 (preferred)
    participant C3 as Coordinator 3 (fallback)
    participant T as Target

    Note over I: set_hole_punch_preferred_coordinators([C1, C2])
    Note over I: preferred_coordinator_count = 2

    I->>C1: PUNCH_ME_NOW (attempt 1)
    Note over C1: try_acquire slot → capacity check
    alt C1 at capacity (Tier 4 back-pressure)
        C1-->>I: silent drop (no reply)
        Note over I: PER_COORDINATOR_QUICK_HOLEPUNCH_TIMEOUT (1.5s)
    else C1 available
        C1->>T: PUNCH_ME_NOW relay
        T-->>I: QUIC connection attempt
    end

    Note over I: Timeout after 1.5s → rotate
    Note over I: current_preferred_coordinator_idx = 0 → 1
    Note over I: is_final_rotation_attempt = true (1+1 >= 2)

    I->>C2: PUNCH_ME_NOW (attempt 2, full strategy timeout ~8s)
    alt C2 available
        C2->>T: PUNCH_ME_NOW relay
        T->>I: QUIC connection (success)
    else C2 fails / times out
        Note over I: should_retry_holepunch()?
        alt retry budget remaining
            I->>C2: retry with C2
        else exhausted
            I->>C3: MASQUE relay fallback
        end
    end

Loading

_{Greploops — Automatically fix all review issues by running /greploops in Claude Code. It iterates: fix, push, re-review, repeat until 5/5 confidence.
Use the Greptile plugin for Claude Code to query reviews, search comments, and manage custom context directly from your terminal.}

_{Reviews (2): Last reviewed commit: "refactor!(nat): node-wide coordinator ba..." | Re-trigger Greptile}